DRP Metrics Monitoring 2022-08-01

Mondays 12pm - 12:50pm ET

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees: 

Yusra, Lee, Jeff, Sophie, Hsin-Fang, Nate, Brock, Clare, Eli, Dan, Colin, Lauren, Chris

Agenda:

• Meeting recorder - Jeffrey Carlin (last 6 meetings were:  Clare, Arun, Leanne, Sophie, Lee, Colin) 
• Announcements
• Is this Brock's last RC2/DC2? What do we do about w_2022_30 (DC2) and w_2022_32 (RC2)? 
  • Everything must be done at SLAC as of ~Aug. 3-4. w_2022_28 (RC2) will be last run at NCSA. Brock will focus on training up Orion.
  • Update from Hsin-Fang: w_2022_28 (using panda) at SLAC running into some issues (e.g., can't run high-memory jobs, DM-35752 - Getting issue details... STATUS  is the SkyCorrection error that we only see at SLAC but cannot be reproduced at NCSA), but getting worked out. → Good to have at least one overlapping processing run.
  • Nikolay has been running DC2 at IDF. Repo not available at SLAC yet.
• Review Action items from last month
• Processing Status
• Review w_2022_26 DC2 rerun: DM-35344
  • Had to use w_27 instead of 26 to pick up a bug fix that caused rampant failures. A lot of failures in calibrate (mostly due to memory issues) - reran fine. 
  • bps_report doesn't work with panda - add instructions for panda reporting to gen3 cookbook (would like bps_report to work with panda!)
  • chronograf: no major changes between w_22 and 2_26. Everybody is silent, which must mean "good".
  • plots:
    • analysis_drp 

      • u/yusra/coadd/DC2/w_2022_18-w31 

      • u/yusra/coadd/DC2/w_2022_26-w31

      • Astrom residuals look good. Redgal color resids scatter - no comments. cmodel-ref mag sigma_mad ~4 mmag for S/N>100.
    • analysis_tools:

      • u/yusra/coadd/DC2/tools/w_2022_18-w31

      • u/yusra/coadd/DC2/tools/w_2022_26-w31
      • Now have ellipticity residual plots in all 6 bands! g-band has a positive offset in size fractional residuals (but actually the plot's zoomed in – medians show residuals larger by factor of ~3 i-band). Are the wonky points at very bright mags a concern? Q from Eli: do PSFs in imsim have chromaticity?  
• Review w_2022_28 RC2 rerun: DM-35609
  
  • Recall history:
    
    • w12: psfex
    • w16: piff  (bad size residuals)
    • w20: finalizeCharacterize (bad apcorr configs → bad stellar locus)
    • w22: lanczos11 + apcorr configs (better stellar locus and size residuals approx equivalent to psfEx!
    • w24: PIFF kernelSize to 25. new scarlet lite storage.
    • w28: attempt at fixing measure failures:  DM-35722
  • Chronograf, plot-navigator 
    • DC2 truth-match (Gen3) dashboard. Comments encouraged at DM-35111 - Getting issue details... STATUS . Currently requires match in >= 3 bands, ticket open to allow for cases where all cmodel fluxes are NaN. Can (in theory) toggle between resolved/unresolved for metrics, but SQuaSH has a 64-char limit for metric names, which is exceeded by resolved src metrics, so they don't currently get ingested.
  • Plots:
    • main analysis_drp 

      • u/yusra/coadd/RC2/w_2022_20-w31

      • u/yusra/coadd/RC2/w_2022_22-w31

      • u/yusra/coadd/RC2/w_2022_24-w31

      • u/yusra/coadd/RC2/w_2022_28-w31 

    • tickets/DM-35336 analysis_drp
      • u/lauren/w_2022_24_plots
      • u/lauren/w_2022_28_plots
      • Lauren notes that some fixes to stellar locus plots are on the ticket branch where she produced these. 
    • analysis_tools
• New plots and metrics to expect?
• AOB

DM-35752 - Getting issue details... STATUS

Action Items