DRP Metrics Monitoring 2022-06-27

Mondays 12pm - 12:50pm ET

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees: 

Yusra AlSayyad Clare Saunders , Arun Kannawadi , Eli Rykoff , Jim Bosch , Unknown User (brendal4) , Dan Taranu , Lauren MacArthur , Christopher Waters , Sophie Reed , Jeffrey Carlin , Lee Kelvin , Fred Moolekamp 

Agenda:

• Meeting recorder - Clare Saunders  (last 6 meetings were: Arun, Leanne, Sophie, Lee, Colin, Fred) 
• Announcements
• Review Action items from last month
• Processing Status
• Review w_2022_22  and w_2022_24 reruns RC2 reruns
  • w_2022_22/DM-34983 
    • some jobs producing NaNs in calibrate, new as of weekly 22
    • no matches for photo-calib is a problem that has been going on for a few weeks
  • w_2022_24/DM-35231
    • lots of held jobs in assembleCoadd, but Fred has a fix in for it.
    • "cannot compute coaddPSF" error - fix has been worked on, but got held up in dm-naming-things - DM-32756 - Arun will take care of finishing this.
    • A lot of the same issues as in w22
    • One job in forcedSrcTable_tract that still takes more memory than it can get.
  • Recall history:
    
    • w12: psfex
    • w16: piff  (bad size residuals)
    • w20: finalizeCharacterize (bad apcorr configs → bad stellar locus)
    • w22: lanczos11 + apcorr configs (better stellar locus and size residuals approx equivalent to psfEx!
  • Chronograf, plot-navigator 
    • Fresh analysis_drp run this morning:
    • u/yusra/coadd/RC2/w_2022_12-v26
    • u/yusra/coadd/RC2/w_2022_16-v26
    • u/yusra/coadd/RC2/w_2022_20-v26
    • u/yusra/coadd/RC2/w_2022_22-v26
    • u/yusra/coadd/RC2/w_2022_24-v26
  • Metrics after lanczos11 + apcorr changes:
    • astrometry unchanged, photometry improved, stellar locus is much improved
    • sky objects are not showing up - not clear why
  • Plots:
    • PSF size fractional difference – offset is smaller by an order of magnitude with a sign change to -5e-6.
    • New plots showing the scatter between the reference catalog and objects in the coadds.
    • Rho plots - why don't we believe them? The plots imply that the ellipticity errors are 10^-1, which is implausible. On the scatter plot, they look more like 10^-3.
    • Stellar locus - sigma MAD was problematic in w20, it is down to 0.021 in w22 and w24.
    • Nate is working on plots for bands other than i.
• New plots and metrics to expect?
• AOB
  • Surhud's plots from HSC Gen3 test run of 100 deg^2
    • They are cutting ccds with maxEllipResidual and maxScaledSizeScatter above certain levels, but these are all very low seeing CCDs and are concentrated in one area of the sky, so not sure if this is the best approach.
      • Yusra recommends looking at the effect on lensing.
      • Josh recommends looking at second moment residuals
      • Jim recommends looking at the pixel phase (?) 

Action Items