Date & Time 

  11:00 PDT

Location

Browser

Room System

Phone Dial-in

https://bluejeans.com/103664856

  1. Dial: 199.48.152.152 or bjn.vc
  2. Enter Meeting ID: 103664856 -or- use the pairing code

Dial-in numbers:

  • +1 408 740 7256
  • +1 888 240 2560 (US Toll Free)
  • +1 408 317 9253 (Alternate Number)

Meeting ID: 103664856

Attendees

Regrets


Discussion items


ItemWhoPre-meeting notesDiscussion and action items
Project/Science Updates

Leanne Guy

PST Meeting: 

  • Chile still under quarantine. No updates on when the project will be rebaselined. 
  • Robert gave a detailed talk at the PST meeting last week on what has been learnt so far about the sensors: ISR: DM and Camera. 
    • Robert now pushing for a more integrated approach to working on the detectors.
    • FFP image processed with DM science pipelines in the talk should not be shared on social media - a press release is being planned. 

DM-SST vF2F meeting: 

  • Following input from several people, we will not hold a long-format DM-SST vF2F meeting. The 2020-09-14 DM-SST meeting will be a regular meeting 
  • Robert comments that we shouldn't think that we are being delivered a fully characterised and debugged camera; still lots to do looking at the scientific quality of the data
  • Robert will give the PST ISR talk again this week on Wednesday at the regular Princeton pipelines team meeting. 
Issuing persistent references

Gregory Dubois-Felsmann Tim Jenness

Gregory and Tim will present their slides for a AAS hosted workshop on the topic of  issuing DOIs (or other forms of persistent references) by projects/missions and astronomical archives.

The AAS meeting on Friday aims to bring together major  astronomical data archives to discuss the problem of providing persistable references to data, namely DOIs.

Drivers: 

  • What do users really expect in terms of longevity of query results?
  • ApJ are probably going to insist on a DOI being attached to data to identify its source. 
  • Will probably need to find some sort of compromise between storing billions of rows of all query results and not storing billions of rows every time someone mints  DOI.  

Classic use case: 

  • A users does their own photometric redshift analysis with a different algorithm and publishes a catalogue of redshifts for say  10 million objects. Data rights prevent making this table with  coordinates and fluxes, etc world public. Instead, they can produce a public table with with object IDs and photometric redshift numbers. Data rights holders cab access the full information. 

Main points that Gregory and Tim will present: 

  • Rubin has no formal requirements for creating DOIs, and nor any agency mandate to act as a long term archive. The latter has always been expected to evolve in the operations era, giving agencies time to think about what to do. We are not the best DOI issuer. 
  • We do have requirements around the reproducibility of queries over the duration of the survey, i.e  the ability to save a query specification and re-execute it at a later date and get the same result. 
    • ... and that the system to do this should be capable of being scaled to all 11 DRs spinning simultaneously.
  • Usefulness of query specification persistence is debatable, what is emerging in the community is a desire to have collections persisted by a list of object identifiers e.g DOIs for a DR Object catalog
  • Could be useful to address data rights issues, e.g just publish the IDs of a concrete list of objects.  
  • Want to ask that the project develop tools to facilitate the creation of lists of object IDs, to devise a form for persisting and sharing those, and for attaching DOIs to them. This would enable data rights holder to take such a list, and create  a query that reselects the objects in the list. 
  • This small upscope would need an LCR, technically easy to implement, the expense would be in the resources required to run the service – the query specifications are already in scope, issuing the DOIs is what is new. 
  • Possible issues with  limited user file workspace and database workspace resources. 
Plans for photometric redshifts in the DPR catalogs

Melissa Graham

Status review on where we are with photo-z.  Proposed set of milestones for converging on a plan for photo-z.
Updated version of DMTN-049 tickets/DM-6367. 

  • Updated DMTN-049 now focused on building a roadmap and providing a revised timelien to converge on a photo-z algorithm for DR1.
  • Consensus that is it essential that the decision be based on LSST data, and that we provide the community with LSST (commissioning) data for some decent spectroscopic sample in order to validate their algorithms and make their proposal. 
  • Given the compressed commissioning schedule, we may need to reevaluate the choice of algorithm frequently in early operations. 
  • Agreement to push out the decision on a photo-z algorithm until after we have commissioning data, but to start as much as possible before commissioning keep to the proposed new schedule to begin soliciting input from the community. 
  • Make explicit the assumptions going into this proposal, e.g  who is responsible for photo Z's, what are our and the community's responsibilities? Similar to the broker/project responsibilities.  
  • Idea of a Data Challenge was raised, à la PLAsTiCC. This could be interesting once we have commissioning data in-hand; ask the community to demonstrate the performance of their algorithms against a defined dataset. 

Updates to dmtn-049: 

  • Make it clear who is responsible for what in the document, i.e choice of algorithm, implementation, verification, validation, how this will work with in-kind contributions etc. 
  • Make explicit that the choice of photo-z algorithm will be reviewed with each DR. This is this initial process for a decision for an algorithm for DR1. 
  • Add science drivers for joint processing, but make explicit that there is currently no requirement nor commitment to do this. 
AOB

Leanne Guy

  • No meeting next week due to JSR
  • No meeting following week due to labour day 


List of SST tasks (Confluence)

DescriptionDue dateAssigneeTask appears on
  • Robert Lupton Clarify the meaning of time in the object table. 1 sentence description in sdm_schemas, can link to a short DMTN.  Update 2022-02-09: Meeting to resolve this on 2022-02-21  
28 Feb 2022Robert Lupton2018-11-05 DM SST F2F Agenda and Meeting notes
  • Gregory Dubois-Felsmann check if SDM standardization is adequately represented in project documents, and whether DMTN-067 should be required.
31 Mar 2022Gregory Dubois-Felsmann2022-02-14 DM-SST Virtual F2F Agenda and Meeting notes
28 Feb 2023Leanne Guy2023-01-23 DM-SST Agenda and Meeting Notes
  • Leanne Guy talk to Steve R about presenting plans for the ShearObject table to PST and SciCollab chairs   
20 Mar 2023Leanne Guy2023-02-27 DM-SST Agenda and Meeting Notes
31 Mar 2023Jim Bosch2023-02-27 DM-SST Agenda and Meeting Notes
  • Leanne Guy  talk to Gregory Dubois-Felsmann to review the original intent of the AFS-related Portal requirements before deciding on a course of action  
29 May 2023Leanne Guy2023-05-01 DM-SST Focus Meeting - Brokers in Commissioning
  • Leanne Guy Prepare to consult the PST on the question of providing compressed PVIs for AP outputs, to cover the period before the data become available in a DR.  
02 Jun 2023Leanne Guy2023-03-27 DM-SST Agenda and Meeting Notes
  • Jim Bosch Incorporate 30-60 day period for raws on disk into the strawman proposal and present to KT  
26 Jun 2023Jim Bosch2023-05-08 DM-SST Agenda and Meeting Notes
  • Parker Fagrelius Patrick Ingraham  how long will it take to do a scan as described? No need to scan the whole WL range but will require additional points outside nominal lambda range.  
30 Jun 2023Parker Fagrelius2023-03-27 DM-SST Agenda and Meeting Notes
31 Jul 2023Colin Slater2023-07-10 DM-SST Agenda and Meeting Notes
  • Eli Rykoff , Leanne Guy  Develop a proposal for what calibration processing, hardware, data we actually need and what will be needed for DR1. This has implications for the ORR and for prioritisation of work in commissioning  
31 Jul 2023Eli Rykoff2023-01-30 DM-SST Agenda and Meeting Notes
  • Yusra AlSayyad will look to see if there is any effort to help on option 1  
28 Aug 2023Yusra AlSayyad2023-08-14 DM-SST Agenda and Meeting Notes
  • Jim Bosch  Provide a physical example of that a  up on cell table would look like fo the Colin Slater / DAX team to review  
31 Aug 2023Jim Bosch2023-02-27 DM-SST Agenda and Meeting Notes
30 Nov 2023Gregory Dubois-Felsmann2023-10-23 DM-SST vF2F Agenda and Meeting Notes
30 Nov 2023Leanne Guy2023-10-23 DM-SST vF2F Agenda and Meeting Notes
  •  "What is the pathway to defining the data products that are required to meet DMS-REQ-0266" Jeffrey Carlin   
30 Nov 2023Jeffrey Carlin2023-10-23 DM-SST vF2F Agenda and Meeting Notes
  • Jeffrey Carlin follow up with KT on DMS-REQ-0176 and DMS-REQ-0315 to update/disaggregate this for latest base/summit infrastructure split.  
30 Nov 2023Jeffrey Carlin2023-10-23 DM-SST vF2F Agenda and Meeting Notes
  • Jim Bosch Follow up on the possibility of investigating further the ability to process 2 collections in parallel.   
31 Jan 2024Jim Bosch2023-12-04 DM-SST Agenda and Meeting Notes
31 Jan 2024Jeffrey Carlin2023-12-04 DM-SST Agenda and Meeting Notes
Gregory Dubois-Felsmann2023-10-23 DM-SST vF2F Agenda and Meeting Notes