Contents
Change Control
All LPM- and LSE- prefix documents are managed by the LSST Change Control Board using the change control process described in LPM-19. All LDM- prefix documents are managed by the DM Technical Control Team using the DM Discussion and Decision Making Process (in particular noting section 3).
If small issues (problems) relevant to DM are noted in any of these documents, JIRA tickets may be filed against the "Requirements Documents" or "Design Documents" components in the DM project. If larger changes to the documents are thought to be necessary, a JIRA ticket should be filed in the RFC project (with component "DM"); this can be escalated to an LSST Change Request (LCR) if need be.
Some documents are under active development. Links to relevant work areas are provided below.
Revisions to DMSR, DPDD, DM Applications Design
In particular, the DM System Requirements (DMSR, LSE-61), Data Products Definition Document (DPDD, LSE-163), and DM Applications Design (LDM-151) are in the process of a coordinated revision in 2016-2017. See the links below.
Requirements Hierarchy
These documents are written in scientific language with embedded specifications.
Document | Title | Notes |
---|---|---|
LPM-17 | Science Requirements Document | The fundamental, overall science goals of the project |
LSE-163 | LSST Data Products Definition Document | The data products that are the end deliverables to scientists 2016-2017 revision:
|
These documents are more formal. They are written in requirements language with source in Enterprise Architect (EA) and traceability matrices.
Document | Title | Notes |
---|---|---|
LSE-29 | LSST Systems Requirements | The top-level formal system requirements |
LSE-30 | Observatory System Specification | Requirements and budgets based on the high-level design |
LSE-61 | Data Management System Requirements | Flowed-down requirements for the Data Management subsystem 2016-2017 revision:
|
High-Level Operations Plans
These documents describe in prose how various aspects of operating the system will work, including some implicit requirements on the behavior of the system.
Document | Title | Notes |
---|---|---|
Operations Concept Document | In work in EA, not yet published | |
LSE-79 | Commissioning Plan | |
calibration-appendixB.pdf | Calibration Plan | In work, not yet fully published. Appendix B documenting expected raw calibration exposures is available at the given link. |
Document-5373 | Data Access White Paper | A very old document laying out data access policies that has never been formally accepted nor superseded. It informs much of the system design but is not authoritative in any way. Work is ongoing to update this by David Ciardi. |
Interface Control Documents and Support Documents (Collection-5201)
These documents are intended to comply with LSE-151, the Interface Control & Compliance Management Process. The control documents are written in requirements language with source in EA. The support documents, marked with "(S)", tend to be written in prose.
(The full set of LSST system-level ICDs, not just those involving DM, should be available at Collection-2807.)
Document | Title | Notes |
---|---|---|
LSE-69 | Interface between the Camera and Data Management | Camera events and telemetry |
LSE-68 | Camera Data Acquisition Interface | Camera image data |
LSE-130 (S) | Support-Data Exchanges between Data Management and Camera | Camera as-built information provided to DM |
LSE-75 | Control System Interfaces between the Telescope & Data Management | Pointing and WCS feedback to the telescope |
LSE-76 | Infrastructure Interfaces between the Summit Facility and Data Management | Minimal, mostly networking now |
LSE-77 | Infrastructure Interfaces Between Data Management and the Base Facility | Base site (Deprecated and in the process of being formally withdrawn. Replaced by LSE-239.) |
LSE-239 (S) | Base Facility Data Center Design Requirements | Specifications prepared for design/build contractor (Not technically an ICD.) |
LSE-78 (S) | LSST Observatory Network Design | Summit and Base LANs and WAN |
LSE-140 | Auxiliary Instrumentation Interface between Data Management and Telescope | Calibration spectrometer, dome screen, etc. |
LSE-72 | OCS - Data Management Software Communication Interface | DM OCS-visible devices/components and EFD archiving |
LSE-70 (S) | LSST Observatory Control System Communication Architecture and Protocol | OCS middleware communications |
LSE-209 | Software Component to OCS Interface Control Document | States and commands for OCS-visible devices/components |
LSE-74 (S) | System Dictionary and Telemetry Streams | |
LSE-131 | Interface between Data Management and EPO |
DM Design Baseline (Collection-2511)
These documents describe the baseline design of the DM subsystem at a high level. They are written in prose.
Document | Title | Notes |
---|---|---|
LDM-148 | Data Management System Design | |
LDM-132 | Data Management WBS Dictionary | The descriptions in this document tend to be very general; the boundaries are not necessarily clearly drawn. See also the UML use cases below. |
These documents describe the baseline design of the major components of the DM subsystem. They are also written in prose. Some of these are in the process of being converted from Word to reStructured Text managed through GitHub and published through readthedocs.org in more-easily linkable form – although note that the official baseline version will always be in DocuShare.
Document | Title | Notes |
---|---|---|
LDM-151 | Data Management Applications Design | 2016-2017 revision:
|
LDM-152 | Data Management Middleware Design | Design at the application isolation and control level reST source, Web version |
LDM-135 | Data Management Database Design | Design of the distributed and non-distributed databases containing data products and internal management data |
LDM-153 | Data Management Database Baseline Schema | Notional schemas for the various databases |
LDM-129 | Data Management Infrastructure Design | Design at the hardware and system services level |
LDM-130 | LSST SUI Requirements | In work, current published draft is obsolete |
LDM-131 | Data Management SUI Conceptual Design | |
LDM-156 | Moving Object Pipeline System Design | Probably should not be change-controlled |
Detailed DM Operations Documents (Collection-2511)
The following prose document outlines how various components within the DM subsystem are expected to work in operations, especially in conjunction with other parts of the LSST system. This has also been converted to reStructured Text. A more comprehensive document developed by the NCSA team is expected to replace this one soon.
Document | Title | Notes |
---|---|---|
LDM-230 | Automated Operation of the LSST Data Management System | reST source, Web version |
Detailed DM Design Documents
Most of these documents are in Confluence here. Some are still in Trac. One, LDM-156 Moving Object Pipeline System Design, is in DocuShare and change-controlled although it probably should not be.
DM Sizing Model (Collection-2511)
These Excel spreadsheets and accompanying prose explanatory documents model the structure, capabilities, capacities, and costs of the DM system.
Document | Title | Notes |
---|---|---|
LSE-81 | LSST Science and Project Sizing Inputs | Requirements from the project level and science-based models |
LSE-82 | LSST Science and Project Sizing Inputs Explanation | |
LDM-138 | Data Management Compute Sizing Model | Derives computing cycles needed based on models of algorithmic processing and estimates based on prototypes |
LDM-140 | Data Management Compute Sizing Explanation | |
LDM-141 | Data Management Storage Sizing and I/O Model | Derives terabytes and gigabytes per second needed based on models of data product storage and retrieval and estimates based on prototypes |
LDM-139 | Data Management Storage Sizing and I/O Explanation | |
LDM-142 | Network Sizing Model | |
LDM-144 | Site Specific Infrastructure Estimation Model | Derives machine details and costs based on compute and storage and network requirements derived in the above spreadsheets |
LDM-143 | Site Specific Infrastructure Estimation Explanation |
DM UML Use Cases (Collection-2511)
These documents, with source in EA, were used for doing initial cost estimates of the DM system and components. They have not been updated as responsibilities have moved between WBS items or as functions have been reallocated, but they do still represent activities that will occur and may help provide some understanding about what different WBS elements are expected to do.
1 Comment
Zeljko Ivezic
Most excellent start!