(back to the list of all DM-SST meeting minutes)


11:00 PT



Room System

Phone Dial-in

Short : ls.st/dmsst 

Zoom: https://noirlab-edu.zoom.us/j/97839121776?pwd=K1JPeUpSMXFvSXJSa2xORGkyVk5zdz09

Zoom Meeting ID: 978 3912 1776 
Zoom Password: 512314

Dial closest IP: (east coast) and (west coast) then use the Zoom meeting ID 978 3912 1776  as the dialing extension.

For example: 978 3912 1776 @ or: 3912 1776  Password: 512314

Dial-in numbers:

+1 346 248 7799 (US Toll)

+1 669 900 6833 (US Toll) Meeting ID: 936 2540 1560 International numbers available: https://gemini.zoom.us/u/adcUNrbXzS



  • Jim Bosch (Princeton local talk Roman/Rubin/Euclid)

Discussion items


ItemWhoPre-Meeting NotesNotes and  Action Items
Project/Science Updates

No PST meeting last week due to AMCR

  • Leanne Guy raise the scheduling of the Roman/Rubin town halls at the AAS (Thursday 12:45) . This is a known problem; there are many town halls and this was deemed the least bad option 
Definition of dataset selection requirements: DMS-REQ-293 (P1a) – what degree of dataset-selectability is required to claim this requirement?

Specification: A Dataset may consist of one or more pixel images, a set of records in a file or database, or any other grouping of data that are processed or produced as a logical unit. The DMS shall be able to identify and retrieve complete, consistent datasets for processing.

Discussion: Logical groupings might be pairs of Exposures in a Visit, along with supporting metadata and provenance information, or might be groupings defined in the context of Level3 processing

From 2023-07-10 DM-SST Agenda and Meeting Notes

What is sufficient to claim this requirement?


Original intent was more akin to what is called a "collection" today - not a single image. 

Colin and Robert on summit this week and will try to come up with a proposal

Proposal from Colin: 

  • argues that 'dataset' is a Butler dataset only to define a scope that we can deliver - "so one or more butler datasets"
  • mechanisms 
    • tagged collections based on Butler dimensions
      • this is implemented
    • open-ended - pass a list of data IDS to the Butler generated somehow (e.g EFD ...) 
      • this is possible but a bit hacky and the Butler UI could be improved, but it is possible. 
      • E.g. a CDB query that can be turned into a tagged collection would satisfy this. 
      • File with a list of dataIDs would be nice to have
  • Consistent has a definition in the upper OSS -  - e.g. arising from a coherent set of inputs 
    • Chaining together of collections in the right way is the mechanism for superseding.  
    • Tagged collection is independent of a chain and is only a list of data IDs - does it guarantee that there are no duplicates?
    • See CM  and the reprocessing of dome IDs
  • Butler requirement to process multiple input datasets in parallel is DMS-MWBT-REQ-0082 satisfies this in a notebook with analysis code, but not in a pipetask, The requirement is about querying, not process. 
  • There is a feature request in DMTN-220 for dataId set upload that is the same as this request: https://dmtn-220.lsst.io/#middleware-feature-requests (3.1). DMTN-220 is the source for all these types of Butler requests. 
  • Jim Bosch Follow up on the possibility of investigating further the ability to process 2 collections in parallel.   
  • Jeffrey Carlin Gregory Dubois-Felsmann Colin Slater Define verifications for the consistency of collections part of this requirements - e.g. OSS-118  
  • Gregory Dubois-Felsmann Create ticket for new Butler functionality to take a (simple text)  file of data IDs and pass them to the Butler and retrieve the associated data. Follow up with Jim Bosch    : DM-42298
 "What is the pathway to defining the data products that are required to meet DMS-REQ-0266"

Follow on from DM-SST vF2F , 2023-20-23

DM-41716 - Getting issue details... STATUS

Specification: The DMS shall create an Exposure Catalog containing information for each exposure that includes the exposure date/time and duration, properties of the filter used, dome and telescope pointing and orientation, status of calibration apparatus, airmass and zenith distance, telescope and dome status, environmental information, and information regarding each sensor including an ID, its location in the focal plane, electronic configuration, and WCS.

To schedule discussion for a January SST meeting 
RFC-970: Modifications to LSE-61 requirements

Status of RFC-970 proposed changes - have all the proposals to re-prioritize and/or de-aggregate requirements discussed at the vF2F meeting been addressed? 

RFC has been approved. LCR-3643 has been submitted.

After consultation with K-T, RFC-977 has also been submitted with some minor changes to "facilities"-related requirements.

DM Acceptance test Campaign, Fall 2023 - status and closeout

Closeout and review of Test Report: DMTR-401 / Test Plan: LVV-P106

slides with updates since vF2F


Next meeting  

List of SST tasks (Confluence)

DescriptionDue dateAssigneeTask appears on
  • Robert Lupton Clarify the meaning of time in the object table. 1 sentence description in sdm_schemas, can link to a short DMTN.  Update 2022-02-09: Meeting to resolve this on 2022-02-21  
28 Feb 2022Robert Lupton2018-11-05 DM SST F2F Agenda and Meeting notes
  • Gregory Dubois-Felsmann check if SDM standardization is adequately represented in project documents, and whether DMTN-067 should be required.
31 Mar 2022Gregory Dubois-Felsmann2022-02-14 DM-SST Virtual F2F Agenda and Meeting notes
  • Steve Ritz Work with DM and Camera to develop plans for delivering a shutter position-vs-time model  
16 May 2022Steve Ritz2022-03-14 DM-SST Agenda and Meeting Notes
  • Gregory Dubois-Felsmann  / Leanne Guy Create a TN that outlines the science use cases and options for compressed PVIs from AP. Present the issue at DMLT VF2F and then the PST.   
13 Jun 2022Gregory Dubois-Felsmann2022-05-09 DM-SST Agenda and Meeting Notes
  • Eli Rykoff Report on what information DES used for their decision on using compression?  
25 Jul 2022Eli Rykoff2022-06-06 DM-SST VF2F Agenda and Meeting notes
  • Melissa Graham to draft a proposal for including pz in alert packets  
12 Sep 2022Melissa Graham2022-08-15 DM-SST Agenda and Meeting Notes
26 Sep 2022Steve Ritz2022-09-12 DM-SST Agenda and Meeting Notes
  • Gregory Dubois-Felsmann  start a technote to write down the details for science end user interface to SED corrections in catalogs  
31 Oct 2022Gregory Dubois-Felsmann2022-08-22 DM-SST Agenda and Meeting Notes
31 Dec 2022Leanne Guy2022-06-06 DM-SST VF2F Agenda and Meeting notes
28 Feb 2023Leanne Guy2023-01-23 DM-SST Agenda and Meeting Notes
  • Leanne Guy talk to Steve R about presenting plans for the ShearObject table to PST and SciCollab chairs   
20 Mar 2023Leanne Guy2023-02-27 DM-SST Agenda and Meeting Notes
31 Mar 2023Jim Bosch2023-02-27 DM-SST Agenda and Meeting Notes
  • Leanne Guy  talk to Gregory Dubois-Felsmann to review the original intent of the AFS-related Portal requirements before deciding on a course of action  
29 May 2023Leanne Guy2023-05-01 DM-SST Focus Meeting - Brokers in Commissioning
  • Leanne Guy Prepare to consult the PST on the question of providing compressed PVIs for AP outputs, to cover the period before the data become available in a DR.  
02 Jun 2023Leanne Guy2023-03-27 DM-SST Agenda and Meeting Notes
  • Jim Bosch Incorporate 30-60 day period for raws on disk into the strawman proposal and present to KT  
26 Jun 2023Jim Bosch2023-05-08 DM-SST Agenda and Meeting Notes
  • Parker Fagrelius Patrick Ingraham  how long will it take to do a scan as described? No need to scan the whole WL range but will require additional points outside nominal lambda range.  
30 Jun 2023Parker Fagrelius2023-03-27 DM-SST Agenda and Meeting Notes
31 Jul 2023Colin Slater2023-07-10 DM-SST Agenda and Meeting Notes
  • Eli Rykoff , Leanne Guy  Develop a proposal for what calibration processing, hardware, data we actually need and what will be needed for DR1. This has implications for the ORR and for prioritisation of work in commissioning  
31 Jul 2023Eli Rykoff2023-01-30 DM-SST Agenda and Meeting Notes
  • Yusra AlSayyad will look to see if there is any effort to help on option 1  
28 Aug 2023Yusra AlSayyad2023-08-14 DM-SST Agenda and Meeting Notes
  • Jim Bosch  Provide a physical example of that a  up on cell table would look like fo the Colin Slater / DAX team to review  
31 Aug 2023Jim Bosch2023-02-27 DM-SST Agenda and Meeting Notes