(back to the list of all DM-SST meeting minutes)

Time: 

11:00 PT

Location

Browser

Room System

Phone Dial-in

Short : ls.st/dmsst 

Zoom: https://noirlab-edu.zoom.us/j/97839121776?pwd=K1JPeUpSMXFvSXJSa2xORGkyVk5zdz09

Zoom Meeting ID: 978 3912 1776 
Zoom Password: 512314

Dial closest IP: 162.255.36.11 (east coast) and 162.255.37.11 (west coast) then use the Zoom meeting ID 978 3912 1776  as the dialing extension.

For example: 978 3912 1776 @162.255.37.11 or: 162.255.37.11##978 3912 1776  Password: 512314

Dial-in numbers:

+1 346 248 7799 (US Toll)

+1 669 900 6833 (US Toll) Meeting ID: 936 2540 1560 International numbers available: https://gemini.zoom.us/u/adcUNrbXzS

Attendees

Regrets

Discussion items

 ​

ItemWhoPre-Meeting NotesNotes and  Action Items
Project/Science Updates

Leanne Guy

No PST meeting last week

Many people involved with either the DESC meeting or the Joint Operations Review this week. 

Do we want to have a meeting at the JTM?

DESC session on Wed 10:45 Pacific  for a discussion on Parallel 3.3 Survey Non-uniformity Effects. There might be a discussion on non-uniformity. 

  • SC Chairs PST meeting this week in the PST slot.  AuxTel is the topic. It will be recorded. 
  • If we have an agenda item, it might be good to have a 1-hour meeting. Stretch goals, DR1 definition.  Leanne will think about possibly topics. 


Shear measurement pipeline followup

Jim Bosch 

Follow up discussion on the shear measurement pipeline in DM and discussion of a Rubin metadetect table schema and other implications for DM. 

Goal is to converge on a  conceptual schema to add to the DPDD and to prepare an RFC  

slides from the DESC presentation

DM-38128 - Getting issue details... STATUS has links to draft PR and PDF.

  • New section 4.2.6 describing the shear measurement approach that  was presented last week.
  • Proposing a separate ShearObject table, no matches to the Object table. 
  • Naming conventions, acronyms, camelcase .... can we do better?
  • CTS/JB: this is one column for g,r,i bands  same as  the Object table 
  • JB: Could see table growth if we want quantities per-band as well as averaged across bands.
  • JC: Are we presuming a cell table that this can be joined to?
  • JB: In the same boat as ccdVisit, tract, patch table.  Provenance table is probably the most important. Not sure if this is in the DB or in the Butler provenance system
  • LPG/CTS: Looking like this table will be ~1/4 number of columns as the Object table. 
  • JB: We can bitpack flags but this is not well supported by SQL nor Parquet. Flags dp not come for free. 
  • JC: Table name - does this only imply that it is only useful to a specific science? Can we come up with a different name that implies broader use?
  • JB: It really is only useful to WL and is not useful for the physical sizes or ellipticities of galaxies. Table goal is to present the sky as a background field that gets distorted by a foreground field, not as a table of individual astrophysical objects.
  • LPG: Need to make sure that the ShearObject table is not mistakenly used  by the community - there are columns with similar names as in the Object table. Take the same approach as for SSO tables. ShearObjects focused on a specific science case, just as SSObject tables do.  
  • CTS: Joins to a cell table are not really a worry.  Concerned about provenance tables for the images that go into cells. Would like to see a physical schema for that sooner than later.
  • CTS: Has informed DAX team about this - no one is freaked out.
  • JB: Cell provenance table does not necessarily need to be in the DB, it could be in Butler. 
  • CTS: Some people might wonder why they cannot get the all the relevant data from Qserv and have to go to Butler to get some data. Could result in a bad UX. 
  • JB: If DAX say it is OK to put into Qserv  then OK 
  • CTS: This can go ahead  without a cell table - will want an example later but not urgent
  • CTS: Rollout might be a good topic for a PST science collab chairs talk . Together with the removal of Object extra (MCMC samples) . 
  • LPG:  Make any talk to the PST / SC Chairs general to cover all recent updates to the DPDD. e.g  
    • Astrometry - DR1 goals are different to the final goals 
    • We store deblending outputs better now
    • Cell-based coadds  - discuss this more - what does this mean for firefly
  • MLG: Cell based coadds - are these a new image data products? 
  • JB: Only called out in the ShearObject table - implementation detail of how we build the coadds. This might affect how the file for cell-based coadds will look.  All Object quantities will come from the cell-based coadds.
  • MLG: Why not issue cell-based templates
  • JB: Not clear we will get any benefit and don't want to rock the boat at this stage. Possibly post DR1 but 
  • Jim Bosch Create an RFC the proposed ShearObject table (once ES has had a chance to comment)   RFC-921
  • Leanne Guy talk to Steve R about presenting plans for the ShearObject table to PST and SciCollab chairs   
  • Jim Bosch  Provide a physical example of that a  up on cell table would look like fo the Colin Slater / DAX team to review  
  • Jim Bosch Provide an example of a file containing a cell-based coadd for Gregory Dubois-Felsmann to examine to assess implications for firefly  
Proposal for wrapping up Obs package work 
Leanne Guy

DM-12676 : Obs package refactor has been lingering for a long time at low priority. I propose to try to get a software developer from the in-kin general pool to assist with completing this work


AOB

Leanne Guy

Next meeting is:  




List of SST tasks (Confluence)

DescriptionDue dateAssigneeTask appears on
  • Robert Lupton Clarify the meaning of time in the object table. 1 sentence description in sdm_schemas, can link to a short DMTN.  Update 2022-02-09: Meeting to resolve this on 2022-02-21  
28 Feb 2022Robert Lupton2018-11-05 DM SST F2F Agenda and Meeting notes
  • Gregory Dubois-Felsmann check if SDM standardization is adequately represented in project documents, and whether DMTN-067 should be required.
31 Mar 2022Gregory Dubois-Felsmann2022-02-14 DM-SST Virtual F2F Agenda and Meeting notes
  • Steve Ritz Work with DM and Camera to develop plans for delivering a shutter position-vs-time model  
16 May 2022Steve Ritz2022-03-14 DM-SST Agenda and Meeting Notes
  • Gregory Dubois-Felsmann  / Leanne Guy Create a TN that outlines the science use cases and options for compressed PVIs from AP. Present the issue at DMLT VF2F and then the PST.   
13 Jun 2022Gregory Dubois-Felsmann2022-05-09 DM-SST Agenda and Meeting Notes
  • Eli Rykoff Report on what information DES used for their decision on using compression?  
25 Jul 2022Eli Rykoff2022-06-06 DM-SST VF2F Agenda and Meeting notes
  • Melissa Graham to draft a proposal for including pz in alert packets  
12 Sep 2022Melissa Graham2022-08-15 DM-SST Agenda and Meeting Notes
26 Sep 2022Steve Ritz2022-09-12 DM-SST Agenda and Meeting Notes
  • Gregory Dubois-Felsmann  start a technote to write down the details for science end user interface to SED corrections in catalogs  
31 Oct 2022Gregory Dubois-Felsmann2022-08-22 DM-SST Agenda and Meeting Notes
31 Dec 2022Leanne Guy2022-06-06 DM-SST VF2F Agenda and Meeting notes
28 Feb 2023Leanne Guy2023-01-23 DM-SST Agenda and Meeting Notes
  • Leanne Guy talk to Steve R about presenting plans for the ShearObject table to PST and SciCollab chairs   
20 Mar 2023Leanne Guy2023-02-27 DM-SST Agenda and Meeting Notes
31 Mar 2023Jim Bosch2023-02-27 DM-SST Agenda and Meeting Notes
  • Leanne Guy  talk to Gregory Dubois-Felsmann to review the original intent of the AFS-related Portal requirements before deciding on a course of action  
29 May 2023Leanne Guy2023-05-01 DM-SST Focus Meeting - Brokers in Commissioning
  • Leanne Guy Prepare to consult the PST on the question of providing compressed PVIs for AP outputs, to cover the period before the data become available in a DR.  
02 Jun 2023Leanne Guy2023-03-27 DM-SST Agenda and Meeting Notes
  • Jim Bosch Incorporate 30-60 day period for raws on disk into the strawman proposal and present to KT  
26 Jun 2023Jim Bosch2023-05-08 DM-SST Agenda and Meeting Notes
  • Parker Fagrelius Patrick Ingraham  how long will it take to do a scan as described? No need to scan the whole WL range but will require additional points outside nominal lambda range.  
30 Jun 2023Parker Fagrelius2023-03-27 DM-SST Agenda and Meeting Notes
31 Jul 2023Colin Slater2023-07-10 DM-SST Agenda and Meeting Notes
  • Eli Rykoff , Leanne Guy  Develop a proposal for what calibration processing, hardware, data we actually need and what will be needed for DR1. This has implications for the ORR and for prioritisation of work in commissioning  
31 Jul 2023Eli Rykoff2023-01-30 DM-SST Agenda and Meeting Notes
  • Yusra AlSayyad will look to see if there is any effort to help on option 1  
28 Aug 2023Yusra AlSayyad2023-08-14 DM-SST Agenda and Meeting Notes
  • Jim Bosch  Provide a physical example of that a  up on cell table would look like fo the Colin Slater / DAX team to review  
31 Aug 2023Jim Bosch2023-02-27 DM-SST Agenda and Meeting Notes