DRP Metrics Monitoring 2021-03-01

Mondays 12pm - 12:50pm EST

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees:

Agenda:

• Review Action items from last week
• Review w_2021_06  
  • What was noteworthy in this rerun?
    • It was a rough one. Filter problems with multiband. Memory problems with multiband which needed a config parameter to change in order to get it through. More filterName problems in pipe_analysis. Took 3 weeks instead of 1 week. 
    • Slurm issue appears to be there was an update in slurm versions in the most recent maintenance window. 
    • Features:  (scarlet and filter names)
    • Beware: 9813 uses a different config than 9697 and 9615. setSpectral=False.
  • What changed? Annotate
    
    • Investigate rho-stats an ellipticity resids in w_2020_06 post-Scarlet Arun Kannawadi Fred Moolekamp 
    • Redefine what skyObjects and skySources mean when Scarlet is being used. Use flags to plot measurements on the parent? Fred Moolekamp Lauren MacArthur Lee Kelvin
• What do we expect from w_2021_10? 
  • New features:
    • Beware of: ticket that redfines isPrimary which might change metrics next rerun. maybe next
    • Native Gen3 rerun. 
      • For hopes of parity (from upstream to downstream) we need to get the following in by Wednesday:
        • Read Calibs the same way in ISR between Gen2 and Gen3:   DM-28985 - Getting issue details... STATUS
        • PsfWcsSelectImagesTask: DM-20695 (Reviewed. Will merge today) 
        • Apply jointcal/FGCM during makeWarp: DM-17062 or DM-27613 (working today. Need quick review tomorrow)
        • ForcedCcd ForcedCoadd:  DM-19942 
        • TransformObjectTable: DM-22277 Is anything downstream of this ready?
  • What new metric can we expect next time
• Review w04 DC2
• AOB
  • The result of ci_hsc_gen2 TE1 didn't change when scarlet was added on 1/27. Because Scarlet is not on in processCcd. processCcd still uses the SDSS Deblender. validation_data_hsc/validate_drp only runs validate_drp.
• some plots to consider: https://lsst.ncsa.illinois.edu/~emorgan2/w_2021_06_qaplots/HSC-I/tract-9813/visit-1228/compareVisit-v1228-diff_base_CircularApertureFlux_12_0-sky-gals.png

https://lsst.ncsa.illinois.edu/~emorgan2/w_2021_06_qaplots/HSC-I/tract-9813/compare-t9813-HSC-I-diff_modelfit_CModel-sky-gals.png

https://lsst.ncsa.illinois.edu/~emorgan2/w_2021_02_qaplots/HSC-I/tract-9813/plot-t9813-HSC-I-skyObjects-hist.png

https://lsst.ncsa.illinois.edu/~emorgan2/w_2021_06_qaplots/HSC-I/tract-9813/plot-t9813-HSC-I-skyObjects-hist.png

https://lsst.ncsa.illinois.edu/~emorgan2/w_2021_06_qaplots/HSC-I/tract-9813/compare-t9813-HSC-I-trace_calib_psf_used-sky-stars.png

Action Items