DRP Metrics Monitoring 2021-01-04

Mondays 12pm - 12:50pm EST

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees:

Yusra, Leanne, Eli, Sophie, Lauren, Matthias, Christopher Waters, Keith, Dan, Morgan, Arun, Simon, Josh, Hsin-Fang, Ian, Fred, Robert, Jim, Zeljko, Eric, Lee, Jeff, Andres

Agenda:

• Review Action items from last week
• Review w_2020_50  
  • What was noteworthy in this rerun?
    • Switched from PS1 refcat to GAIA refcat
    • commandlineTasks' --clobber-output blows away the entire output repo.
    • visitAnalysis fails on RC2fakes. 
  • What changed? Annotate.
    • stellar locus regression (see below)
    • relative and absolute astrometry are independent
      
      • Jeffrey Carlin Add an absolute astrometry match-to-refcat metric to dashboard  DM-34153 - Getting issue details... STATUS
      • Yusra AlSayyad Do a rerun with w50 PS1 refcat and one with shrunk refcat errors. 
      • Lauren MacArthur Rerun visitAnalysis with refcat that was used in processing (Gaia) : plots from a current run on the  DM-28214 - Getting issue details... STATUS  ticket branches can be found here: https://lsst.ncsa.illinois.edu/~lauren/lauren/DM-28214/w06/gaia/plots/
• What do we expect from w_2021_02? 
  • Possibly Scarlet. See RFC-745.
  • What new metric can we expect next time
• Review  w48 DC2?
• AOB

Two glitches with pipe_analysis after switching to reading in the parquet tables diagnosed & fixed on  DM-28168 - Getting issue details... STATUS

Some significant "issues" to discuss.  Refer to  DM-28173 - Getting issue details... STATUS

Minutes:

Action items from last time:

Add injection metrics to dashboard.

Regularly running matchedVisitMetrics on DC2. Currently getting errors of type "can't find data". Eric will look into next week.

w_2020_50 FGCM outputs. Now in logs directory.

Why did FGCM solution change? Eli started looking but not too deeply yet. Overall model is the same. Runs on different sample of stars each time. Expect some amount of noise, however the current differences are larger than expected.

w_2020_50 was last re-run. One huge change is switching from Pan-STARRS refcat to Gaia refcat. Accidentally deleted entire output directory. Clobber behavior was not as expected. Anything with clobber-output will clobber entire output repo. Option to be handled with care! In general, we should avoid using clobber in a production setting. Clobber options should probably only be used when experimenting. Suggest not to ever use --clobber-output at this time. Will go away with Gen3.

Problem with coadd analysis, a few runs didn't work, working with Lauren on that.

RC2 fakes has problem with visitAnalysis, just found this morning. Posted issue on JIRA.

Astrometry. Single-frame repeatability improves slightly. There was a small change with the match radius.

Large stellar locus width regression. Lauren is investigating:  DM-28173 - Getting issue details... STATUS Match to reference catalog. Coadd positions determined from Gaia reference and matched to Pan-STARRS. Validate_DRP says photometric repeatability. Doing the match per patch. Fitting also done per patch. What is happening when JointCal WCS is applied. Density of Gaia sources is much lower. New jointCal with Gaia is applying much larger astrometric shifts. Both he visit-level and coadd are shifted. Suspect that image to image registration has gone wrong. This would mess up coadd stellar locus. PS1 refcat has a lot of galaxies it in, more galaxies than stars in astrometric fit. Check whether astrometry is only using stars. Have we done the check that running with PS1 goes back to the expected result? Would need to run jointcal again on everything. Plan to do that test this week. Note there was a bug where region loaded by refcat was shifted. Demonstrate that held-back Gaia stars are consistent with Gaia system. Coherent structures observed across focal plane. The regression is observed in COSMOS field (smaller dither, longer exposures?). AM1 and AM2 look OK. Atmospheric terms should be a level tens mas, where we are seeing shifts in ultra-deep at order hundred mas. Look at camera distortion field from wide and ultra-deep. No dithering in deep field to solve lower order terms of distortion field.

Re-runs: test with PS1 refcat.

PSF size residuals (coadd) have increased slightly.

Only the matchedVisit metrics are visit level on the dashboard currently.

Sky Objects Metrics are now by tract.

w_2021_01 will be tagged in next couple days. w_2021_02 will be tagged next Wednesday (~10 days from now). Possibly scarlet. RFC-745.

w_2021_01 is out already, so I guess _02 is this week. (w_2021_01 went out on Dec 31, 2020)

Was there a w_2020_48 DC2 re-run? No matchedVisit metrics to look at.

Action Items