Date and Location 02 Dec 2015, 11am-12pm pacific.
Russell Owen, Simon Krughoff
Jim Bosch, John Swinbank, Paul Price IPAC: Unknown User (jar), Gregory Dubois-Felsmann, Unknown User (xiuqin), Unknown User (ciardi) SQUARE: Frossie Economou NCSA: Greg Daues, Steve Pietrowicz, Margaret Gelman, Matias Carrasco Kind, Hsin-Fang Chiang Architecture; Kian-Tat Lim, Tim Jenness SLAC: Nate Pease, Brian Van Klaveren, Fritz Mueller, Andy Salnikov, Vaikunth Thukral, Unknown User (kelsey), Jacek Becla Goals To start getting on the same page with regards to butler, with focus on what butler really is, what it does, what its roles are, what APIs is has, and most importantly what it will and will not do in the future. Discussion items
It'd be great if each team could cover features you think you "must have" in butler that are not available today, and features that would be nice-to-have but are not critical.
Time Item Who Notes 5min Intro K-T or NateP or Jacek
10min UW Easily specify new outputs from tasks (no need to edit obs_ packages) butler.get should fail immediately if the data is not available Easily iterate over partial IDs for all dataset types Simplify the way IDs are normalized (?), and document the system better; also preferably make it easier to know when you have an ID that has been normalized or not. Handling of bitemporal calibration products including camera descriptions. N.b. calibration products can be lots of different things: objects, images, telemetry data, sky model, etc. Butlerized access to logs, configs, stack version, etc. 10min Princeton DRP Butler Requirements / Feature Requests 10min IPAC SUI / Level 3 and SuperTask feedback on the Butler 10min SQUARE isolate in-memory from IO, architecture for plugging other protocols (like S3) 10min NCSA
Architecture Tim Jenness : I feel strongly that we should have two persistable file format options designed in to the butler (FITS and HDF5 say) so that we can be sure we have a flexible design that is not wedded to a single file format. Nate Pease: (not butler work, has an AFW component tag) Discussion Carry technical discussions via community Next meeting: Tuesday Dec 8, 12:30pm, via bluejeans. Focus: (a) check if created stories adequately cover all requests, (b) finalize prioritization Random notes
Butler (not-done) stories:
Nate Pease, turn the requirements discussed / input received into JIRA stories Identify core architectural issues Design core architectural issues Write a comprehensible high level design document Create interface definition ( we will defer remote butler & URL interface definition)
Application Interface Configuration Interface Back-end interface. Include what can be plugged in, and where.
transport serialization storage Nate Pease and Kian-Tat Lim - propose prioritization of all butler stories Work on top-priority item for each group:
Princeton: (epic S16) ( Serge Monkewitz is working on kind-of-related ) SUI: Multiple repositories: finish defining, create story, work on partial implementation TBD.
Implement (epic W16) (might get broken into 2 stories and delivered over 2 sprints) UW: (epic Butler W16), also "if time": (epic S16) Tuscon: define interface for (epic S16, assigned to BVK) NCSA: (nothing at this time)