2021-06-15 Science Metric Development Agenda and Meeting notes

Leanne Guy

• NBReport - Simon needs a more concrete description of need. Current use case is too broad. RHL: don't want to invent requirements up front. Need to start doing verification first.  RHL: 

• Robert Lupton Scope: What is the role of notebooks in certifying calibration products RHL to come up with a proposal. 10 Aug 2021 

Jeffrey Carlin Keith Bechtol

Current draft of DMTR-311

• w14 for v22.0.0 and backport the scarlet fix
• Gen2/Gen3 comparison request from Yusra

• Little progress - noo full parity btw G2/3 ppls, Y would like metrics calculated on the G2 oprocessed data before OK-ing a release. Eli is writing a script to compare results Slack conversation here: https://lsstc.slack.com/archives/C4JQP6FRS/p1623111517088600, DM-30696 - Getting issue details... STATUS
• TE1/TE2 - rerun with deblending flag. Results look more reasonable

• Jeffrey Carlinask Yusra about proving Gen2/3 results privately and publish Gen3 native in CMR.  22 Jun 2021 

Review of any issues that have come up during the week with faro and metric computation 

• DM-13758 - Getting issue details... STATUS  - do the filtering before assembling the matched catalog.  New ticket created to do in faro - close this one out

Jeffrey Carlin Keith Bechtol

• Morgan running (gen2) w22 - one failed tract. 3718)? cosmos tract. Done, will not reprocess to get a self consistent run. 
• gen3 w_2021_22 processing ticket:  DM-30365 - Getting issue details... STATUS
• w18  - will not be rerun. 

Leanne Guy

• Ticket   DM-30054 - Getting issue details... STATUS  : loading reference catalogs with colour terms applied
• New RC2 dataset created for use with faro by Simon Krughoff - subset of cosmos fields. Details 
• Closeout of the Spring 2021 epic  DM-26993 - Getting issue details... STATUS  and new Fall 2021 epic  DM-30748 - Getting issue details... STATUS
  • Using object table and source table (parquet format)
• Backlog epic:  DM-29525 - Getting issue details... STATUS

• Access to schema of tract object table is possible: See this thread.
  • Can access for the tract object and source tables. 
  • Need to do this because on the RSP, could not load the full tract into memory, so a workaround was needed. Suggestion from Eli to get the parquet schema without loading anything.  
• Simon's new dataset. Minimum of checking. 8 visits from 5 bands in cosmos, just with 6 central chips from HSC. Run that dataset through DRP Step 1 (SFP, JC, FGCM)/2 (warp/coadd)/3(meas FP object creation). Only 4 patches with full coverage. All this with w24 ran to completion with no error in quanta. Eli said FGCM outputs were not terrible. Will be (intention is to) better than validation_hsc - more visits/bands. Still in the ci / testing./reg. issues phase. repo contains only raw/calib and not outputs. Need a shared location at NCSA 

• Leanne Guy Need to test and refine the RC2 dataset and run faro on tit.   31 Aug 2021 

• Keith: For Fall 2021 epic, wearing commissioning hat, I would personally like to prioritize implementation of a collection of metrics to provide basic quality evaluation of individual detectors / visits, in other words, metrics that would be run immediately upon taking on-sky images, and which could be used as part of an on-the-fly analysis system providing feedback to observers.
  • Propose to develop these metrics using HSC RC2
  • Propose to use sourceTable as main catalog input (work in progress DM-26214 - Getting issue details... STATUS )
  • Needs reference catalogs as input (work in progress DM-30732 - Getting issue details... STATUS )
  • Example metrics: number of detected objects (flagged objects), throughout, image quality, sky brightness, matches to reference catalogs, effective depth, anomaly detection
  • Question: Could we apply these metrics to AuxTel imaging to attempt to perform a closed loop test?
  • Part of the thinking here is that I want to start getting more specific about the types of information we need to address the use cases that have been developed, having some example metrics to use to think more concretely about visualizations and generally moving information where it needs to go.
• RHL: Auxtel is one way to do it, can also run on HSC single frame data. Reply DC2 data throught the camera data could also be used.  

Leanne Guy