Skip to end of metadata
Go to start of metadata


Change Control

All LPM- and LSE- prefix documents are managed by the LSST Change Control Board using the change control process described in LPM-19.  All LDM- prefix documents are managed by the DM Technical Control Team using the DM Discussion and Decision Making Process (in particular noting section 3).

If small issues (problems) relevant to DM are noted in any of these documents, JIRA tickets may be filed against the "Requirements Documents" or "Design Documents" components in the DM project.  If larger changes to the documents are thought to be necessary, a JIRA ticket should be filed in the RFC project (with component "DM"); this can be escalated to an LSST Change Request (LCR) if need be.

Some documents are under active development.  Links to relevant work areas are provided below.

Revisions to DMSR, DPDD, DM Applications Design

In particular, the DM System Requirements (DMSR, LSE-61), Data Products Definition Document (DPDD, LSE-163), and DM Applications Design (LDM-151) are in the process of a coordinated revision in 2016-2017. See the links below.

Requirements Hierarchy

These documents are written in scientific language with embedded specifications.

LPM-17Science Requirements DocumentThe fundamental, overall science goals of the project
LSE-163LSST Data Products Definition Document

The data products that are the end deliverables to scientists

2016-2017 revision:

These documents are more formal.  They are written in requirements language with source in Enterprise Architect (EA) and traceability matrices.

LSE-29LSST Systems RequirementsThe top-level formal system requirements
LSE-30Observatory System SpecificationRequirements and budgets based on the high-level design
LSE-61Data Management System Requirements

Flowed-down requirements for the Data Management subsystem

2016-2017 revision:

  • The source is in the LSST SysML model (currently in transition from EA to MagicDraw)
  • A draft is available at the top of the Docushare history

High-Level Operations Plans

These documents describe in prose how various aspects of operating the system will work, including some implicit requirements on the behavior of the system.


Operations Concept DocumentIn work in EA, not yet published
LSE-79Commissioning Plan
calibration-appendixB.pdfCalibration PlanIn work, not yet fully published. Appendix B documenting expected raw calibration exposures is available at the given link.
Document-5373Data Access White PaperA very old document laying out data access policies that has never been formally accepted nor superseded.  It informs much of the system design but is not authoritative in any way.  Work is ongoing to update this by David Ciardi.

Interface Control Documents and Support Documents (Collection-5201)

These documents are intended to comply with LSE-151, the Interface Control & Compliance Management Process.  The control documents are written in requirements language with source in EA.  The support documents, marked with "(S)", tend to be written in prose.

(The full set of LSST system-level ICDs, not just those involving DM, should be available at Collection-2807.)



LSE-69Interface between the Camera and Data ManagementCamera events and telemetry
LSE-68Camera Data Acquisition InterfaceCamera image data
LSE-130 (S)Support-Data Exchanges between Data Management and CameraCamera as-built information provided to DM

LSE-75Control System Interfaces between the Telescope & Data ManagementPointing and WCS feedback to the telescope
LSE-76Infrastructure Interfaces between the Summit Facility and Data ManagementMinimal, mostly networking now
LSE-77Infrastructure Interfaces Between Data Management and the Base FacilityBase site
(Deprecated and in the process of being formally withdrawn. Replaced by LSE-239.) 
LSE-239 (S)Base Facility Data Center Design RequirementsSpecifications prepared for design/build contractor
(Not technically an ICD.) 
LSE-78 (S)LSST Observatory Network DesignSummit and Base LANs and WAN
LSE-140Auxiliary Instrumentation Interface between Data Management and TelescopeCalibration spectrometer, dome screen, etc.

LSE-72OCS - Data Management Software Communication InterfaceDM OCS-visible devices/components and EFD archiving
LSE-70 (S)LSST Observatory Control System Communication Architecture and ProtocolOCS middleware communications
LSE-209Software Component to OCS Interface Control DocumentStates and commands for OCS-visible devices/components
LSE-74 (S)System Dictionary and Telemetry Streams

LSE-131Interface between Data Management and EPO

DM Design Baseline (Collection-2511)

These documents describe the baseline design of the DM subsystem at a high level.  They are written in prose.

LDM-148Data Management System Design
LDM-132Data Management WBS DictionaryThe descriptions in this document tend to be very general; the boundaries are not necessarily clearly drawn.  See also the UML use cases below.

These documents describe the baseline design of the major components of the DM subsystem.  They are also written in prose.  Some of these are in the process of being converted from Word to reStructured Text managed through GitHub and published through in more-easily linkable form – although note that the official baseline version will always be in DocuShare.

LDM-151Data Management Applications Design

LaTeX source

2016-2017 revision:

LDM-152Data Management Middleware DesignDesign at the application isolation and control level
reST source, Web version
LDM-135Data Management Database Design

Design of the distributed and non-distributed databases containing data products and internal management data

reST source, Web version

LDM-153Data Management Database Baseline Schema

Notional schemas for the various databases

LDM-129Data Management Infrastructure Design

Design at the hardware and system services level

reST source, Web version

LDM-130LSST SUI RequirementsIn work, current published draft is obsolete
LDM-131Data Management SUI Conceptual Design
LDM-156Moving Object Pipeline System DesignProbably should not be change-controlled

Detailed DM Operations Documents (Collection-2511)

The following prose document outlines how various components within the DM subsystem are expected to work in operations, especially in conjunction with other parts of the LSST system.  This has also been converted to reStructured Text.  A more comprehensive document developed by the NCSA team is expected to replace this one soon.

LDM-230Automated Operation of the LSST Data Management SystemreST source, Web version

Detailed DM Design Documents

Most of these documents are in Confluence here.  Some are still in Trac.  One, LDM-156 Moving Object Pipeline System Design, is in DocuShare and change-controlled although it probably should not be.

DM Sizing Model (Collection-2511)

These Excel spreadsheets and accompanying prose explanatory documents model the structure, capabilities, capacities, and costs of the DM system.

LSE-81LSST Science and Project Sizing InputsRequirements from the project level and science-based models
LSE-82LSST Science and Project Sizing Inputs Explanation

LDM-138Data Management Compute Sizing ModelDerives computing cycles needed based on models of algorithmic processing and estimates based on prototypes
LDM-140Data Management Compute Sizing Explanation

LDM-141Data Management Storage Sizing and I/O ModelDerives terabytes and gigabytes per second needed based on models of data product storage and retrieval and estimates based on prototypes
LDM-139Data Management Storage Sizing and I/O Explanation

LDM-142Network Sizing Model

LDM-144Site Specific Infrastructure Estimation ModelDerives machine details and costs based on compute and storage and network requirements derived in the above spreadsheets
LDM-143Site Specific Infrastructure Estimation Explanation

DM UML Use Cases (Collection-2511)

These documents, with source in EA, were used for doing initial cost estimates of the DM system and components.  They have not been updated as responsibilities have moved between WBS items or as functions have been reallocated, but they do still represent activities that will occur and may help provide some understanding about what different WBS elements are expected to do.

LDM-133Data Management UML Domain Model
LDM-134Data Management Applications UML Use Case and Activity Model
LDM-146Data Management Middleware UML Use Case and Activity Model
LDM-244Data Management Science User Interface UML Use Case Model
  • No labels

1 Comment

  1. Most excellent start!