This page serves as an index for various design sketches, meeting minutes, and developer documentation pages produced as part of Gen3 Middleware (Gen3 Butler, SuperTask) development in S18 and F18. Many of these were originally nested under the Data Butler Working Group page, but were not actually products of that working group.
None of these documents should serve as the final authoritative reference for any middleware components; all relevant content should eventually be migrated to LDM documents or at least technical notes.
- Butler Design Discussion 2018-02-14: Registry / Datastore boundary
- Gen3 Butler Composites Design
- Generation 3 Butler Feature Requests
- Gen3 Middleware Camera Specialization Interfaces
- Gen3 Middleware Software Stack
- Observational Metadata Data Structures and Extraction
- Gen3 Butler Subset/Transfer/Merge for Shared-Nothing Workflow
- Gen3 Automated Test Packaging
- Image and Visit metadata - linking the Gen3 Registry and the Science Data Model
- Commentary on Butler Data Model
- Configuring a Butler
- Datastore Questions
- Butler Major Design Questions, 2019-05-28
- QuantumGraph Generation Algorithm Updates for DM-19988
- Gen3 Butler query Python API
- Gen3 DataId and DatasetRef serialization
- Dataset and Collection Table Reorganization
- Architectural Prototype for the New Gen3 Registry
- Saving per-Quantum provenance and propagating nothing-to-do cases, and The Future
- Useful combinations of pipetask command-line options in BPS