Date
Attendees
- Unknown User (danielw)
- Fabrice Jammes
- Serge Monkewitz
Kian-Tat Lim (for metadata discussion)
Goals
- Get together, resolve bottlenecks, make progress faster, plan, design, exchange ideas
Discussion items
John joined group. Will work on Metadata / Data Access / Image Services.
Replanning:
- Two 10-SP metadata related tasks moved from AndyS to John
- DM-1252 (4 SP) moved from AndyS to Daniel, Daniel working on code related to this ticket
- AndyS will work on
- Query Metadata (new epic/stories, see DM-1487)
- DM-621 (loader) - moving this story from Serge
- If we want to add more scope to W15 - likely in the area of distributed loader
Distributed loader for integration tests (DM-998)
- use existing single-node partitioner, then scp csv files produced by partitioner, and ssh <load> on each node
CSS
- Daniel moving css to core/modules/css, includes reorganization of python modules, only czar had python so far
Metadata Store
- reuse code from datarel, can't use datarel as-is, not generic enough
- ideally, computation related to images should happen in afw, not in datarel or metadata store related code (apps team already agreed to deal with computation related to sources)
- research off-the-shelf systems that might be useful (irods, what fermi did etc)
- when importing external data sets, better to "make it butler-compatible first, then ingest"
- metadata store needs to deal with repositories of configs
- might need to capture in provenance which version of butler was used
- need to discuss with SUI team if sql interface to metadata is sufficient, or web api is needed
- different typed of images in one repo or separate? Let butler drive it: one repository per task is best
- meaningful vs obscure file names - still tbd