Data Management
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata


Mondays 12pm - 12:50pm EST

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees:


Agenda:

  • Review Action items from last week
  • Review w_2021_06  
    • What was noteworthy in this rerun?
      • It was a rough one. Filter problems with multiband. Memory problems with multiband which needed a config parameter to change in order to get it through. More filterName problems in pipe_analysis. Took 3 weeks instead of 1 week. 
      • Slurm issue appears to be there was an update in slurm versions in the most recent maintenance window. 
      • Features:  (scarlet and filter names)
      • Beware: 9813 uses a different config than 9697 and 9615. setSpectral=False.
    • What changed? Annotate
    • What do we expect from w_2021_10? 
      • New features:
        • Beware of: ticket that redfines isPrimary which might change metrics next rerun. maybe next
        • Native Gen3 rerun. 
          • For hopes of parity (from upstream to downstream) we need to get the following in by Wednesday:
            • Read Calibs the same way in ISR between Gen2 and Gen3:   DM-28985 - Getting issue details... STATUS
            • PsfWcsSelectImagesTask: DM-20695 (Reviewed. Will merge today) 
            • Apply jointcal/FGCM during makeWarp: DM-17062 or DM-27613 (working today. Need quick review tomorrow)
            • ForcedCcd ForcedCoadd:  DM-19942 
            • TransformObjectTable: DM-22277 Is anything downstream of this ready?
      • What new metric can we expect next time
  • Review w04 DC2
  • AOB
    • The result of ci_hsc_gen2 TE1 didn't change when scarlet was added on 1/27. Because Scarlet is not on in processCcd. processCcd still uses the SDSS Deblender. validation_data_hsc/validate_drp only runs validate_drp.
  • some plots to consider: https://lsst.ncsa.illinois.edu/~emorgan2/w_2021_06_qaplots/HSC-I/tract-9813/visit-1228/compareVisit-v1228-diff_base_CircularApertureFlux_12_0-sky-gals.png

https://lsst.ncsa.illinois.edu/~emorgan2/w_2021_06_qaplots/HSC-I/tract-9813/compare-t9813-HSC-I-diff_modelfit_CModel-sky-gals.png

https://lsst.ncsa.illinois.edu/~emorgan2/w_2021_02_qaplots/HSC-I/tract-9813/plot-t9813-HSC-I-skyObjects-hist.png

https://lsst.ncsa.illinois.edu/~emorgan2/w_2021_06_qaplots/HSC-I/tract-9813/plot-t9813-HSC-I-skyObjects-hist.png

https://lsst.ncsa.illinois.edu/~emorgan2/w_2021_06_qaplots/HSC-I/tract-9813/compare-t9813-HSC-I-trace_calib_psf_used-sky-stars.png


Action Items

DescriptionDue dateAssigneeTask appears on
  • Add a plot with fakes stats to the dashboard. Sophie Reed 
04 Sep 2020Sophie ReedDRP Metrics Monitoring 2020-08-07
  • Jeffrey Carlin  Make faro not attempt running on narrow-bands.  
16 Mar 2022Jeffrey CarlinDRP Metrics Monitoring 2022-02-28
  • turn catchFailures on in calibrate. Add flag to indicate that deblender failed because PSF is bad. 
 DRP Metrics Monitoring 2022-10-31
  • Yusra AlSayyad Eric's account was deleted; we need to make sure he has all his logs. 
 Yusra AlSayyadDRP Metrics Monitoring 2021-06-14
  • Arun Kannawadi Modify rho stats in pipe_analysis  to use debiased moments (see  DM-30751 - Getting issue details... STATUS ). 
 Arun KannawadiDRP Metrics Monitoring 2021-04-19
Arun KannawadiDRP Metrics Monitoring 2021-03-01
  • Yusra AlSayyad Do a rerun with w50 PS1 refcat and one with shrunk refcat errors. 
 Yusra AlSayyadDRP Metrics Monitoring 2021-01-04
  • Jeffrey Carlin Add an absolute astrometry match-to-refcat metric to dashboard  DM-34153 - Getting issue details... STATUS
Jeffrey CarlinDRP Metrics Monitoring 2021-01-04