New time! Monday's 12pm - 12:50pm EDT

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees:

Regrets: Keith

Agenda:

  • Review Action items from last week
  • Review w_2020_42
    • What was noteworthy in this rerun? 
      • How do we tell Eric to change contents of the rerun? (like this time: missing coaddAnalysis metrics)
        • Eric says: Alert him on  hsc-reprocessing
        • Add it to the ticket in advance
      • Json files are missing: grab list from Simon. 
      • Remaining issues with fakes processing. Missing FGCM, was run with jointcal. Need to update configs to be consistent with the main rerun. 
      • This is officially the most recent good rerun. 
    • What changed? Annotate.
    • New Rho/Rowe stats. 1-2 orders of magnitude higher than we'd like.  
      • New metric to probe correlation of rho/rowe with magnitude? 
      • Arun + Morgan : look into mag dep metrics
      • We don’t have to debug live, if we spend more than 3 minutes on a question we’ll assign someone or group to look into it for next time.
    • What do we expect from w_2020_46? 
      • coaddAnalysis metrics.
      • release candidate for v21 might beat it. 
    • What new metric can we expect next time
  • AOB


Photometric repeatabiliy metric details:

  • Proposals:
    • Done: Deterministic,  configurable.
    • New:
      • Higher SNR.  Either take into account errors or use SNR > 200. Set multiple SNRs. 
      • Filter for isolated stars. 
    • Send Jeff flag cuts for photometric repeatability metric computation. 
    • Ticket (with some discussion) related to previous two bullets:  DM-9546 - Getting issue details... STATUS
    • Plot showing issue of non-isolated stars used in the PSF modelling.  This is what they look like on the coadd: https://lsst.ncsa.illinois.edu/~emorgan2/w_2020_42_qaplots/HSC-I/tract-9813/plot-t9813-HSC-I-mag_base_CircularApertureFlux_12_0_forced_calib_psf_used-psfMagHist.png
    • Separate metric that subtracts off the photometric errors in quadrature. 
    • Chi2/dof for sky sources? (sky sources aren't in the same position) <—technically unimplementable now, unless we use forced phot for sky OBJECTS which we don't compute metrics on now. 
    • Chi2/dof for stars?

Plots from pipe_analysis where changes were expected due to FGCM fixes  DM-26862 - Getting issue details... STATUS :

Comparing a matched catalog between w_2020_42 & w_2020_38 runs:

*comparisons scripts were run against w30. 

https://lsst.ncsa.illinois.edu/~emorgan2/w_2020_42_qaplots/HSC-I/tract-9813/visit-1228/compareVisit-v1228-diff_base_CircularApertureFlux_12_0-sky-gals.png

https://lsst.ncsa.illinois.edu/~emorgan2/w_2020_42_qaplots/HSC-I/tract-9813/compare-t9813-HSC-I-diff_base_CircularApertureFlux_12_0-sky-gals.png

Principal colour analysis:

https://lsst.ncsa.illinois.edu/~emorgan2/w_2020_38_qaplots/color/tract-9813/plot-t9813-griPSF-wFit-fit.png

(width = 13.35 mmag)

vs.

https://lsst.ncsa.illinois.edu/~emorgan2/w_2020_42_qaplots/color/tract-9813/plot-t9813-griPSF-wFit-fit.png

(width = 13.15 mmag, so indeed a slight improvement)

RC2 and RC2fakes Processed data available on lsst-login machines at:

/datasets/hsc/repo/rerun/RC/w_2020_42

/datasets/hsc/repo/rerun/RCfakes/w_2020_42

Action Items

DescriptionDue dateAssigneeTask appears on
  • Add a plot with fakes stats to the dashboard. Sophie Reed 
04 Sep 2020Sophie ReedDRP Metrics Monitoring 2020-08-07
  • Sophie to add field in metric definition to hold thresholds. DM-43364 - Getting issue details... STATUS : We need to talk about this when Sophie is back!
 DRP Metrics Monitoring 2024-04-22
  • Sophie to add field in metric definition to hold thresholds. DM-43364 - Getting issue details... STATUS
DRP Metrics Monitoring 2024-03-18
  • Clare: add analyzeMatchedVisitsCore to drp_pipe step8
 DRP Metrics Monitoring 2023-06-26
  • Sophie: make a new list for outstanding analysis_drp plots that require moving, send to Jim
 DRP Metrics Monitoring 2023-06-26
  • turn catchFailures on in calibrate. Add flag to indicate that deblender failed because PSF is bad. 
 DRP Metrics Monitoring 2022-10-31
  • Yusra AlSayyad Eric's account was deleted; we need to make sure he has all his logs. 
 Yusra AlSayyadDRP Metrics Monitoring 2021-06-14
  • Arun Kannawadi Modify rho stats in pipe_analysis  to use debiased moments (see  DM-30751 - Getting issue details... STATUS ). 
 Arun KannawadiDRP Metrics Monitoring 2021-04-19
Arun KannawadiDRP Metrics Monitoring 2021-03-01
  • Yusra AlSayyad Do a rerun with w50 PS1 refcat and one with shrunk refcat errors. 
 Yusra AlSayyadDRP Metrics Monitoring 2021-01-04
  • Jeffrey Carlin Add an absolute astrometry match-to-refcat metric to dashboard  DM-34153 - Getting issue details... STATUS
Jeffrey CarlinDRP Metrics Monitoring 2021-01-04