Mondays 12PT (3 - 3:50pm ET)

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees:

Regrets:

Agenda:

  • Meeting recorder - Clare (last 6 meetings were:  Dan, Erfan, Lee, Robert, Arun, Clare) 
  • Announcements
    • Next weekly (39) will be tagged in a week and a half.
    • Current plan is for 32 to become the release candidate and to backport as needed.
  • Action items from last time:
    • Sophie: make a new list for outstanding analysis_drp plots that require moving, send to Jim (Sophie sent it to Colin instead, and will tell us where it is soon) 
  • Processing Status 
    • Processing of w_2023_35 RC2 (Orion)
    • Pretty straightforward, same errors as before from subtract images
    • Interesting plots: astrometry looks totally the same, stellar locus not significantly changed (~mmag order variations)
    • PA1 metrics are not on chronograph yet, but can be accessed through the plot navigator - all bands around ~7 for 9813. Jeff will investigate why PA1 is not showing up.
    • What is the source of small trends in the shapeSizeFractionalDiff? Possibly brighter-fatter, but we probably don't want to jump to conclusions on this. Orion would like guidance on how to decide if this is a trend that needs looking into.
    • Ellipticity - all over the place within small ranges. Nothing except may be 9615 g-band looks significantly worse.
    • Sky objects - the metrics look the same, but the plots show that the aperture and PSF S/N plots are actually broader than they should be (sigma_mad > 1). This has been true since at least w_2023_23
      • Lee: I look at the aperture flux the most. There could be some contamination because sky objects are placed so there is nothing within 8 pixels, but we are looking at 9 pixel aperture fluxes - fixing this is a longstanding issue. There is going to be some further contamination by undetected faint sources, which could cause this broadening. This a long-term issue, not new.
      • Jim: This could be related to longstanding issue of flux underestimates. 
      • Eli: This looks the same with PSF fluxes which have the same issue. I agree that it's something to do with the variance plane.
    • Tour of matchedVisitCore plots:
      • input: sourceTableVisit
      • objects with measurements from multiple visits
      • do statistics on same objects over multiple visits; this is how we get AM1, PA1 etc.
      • AM1/stellarAstrometricRepeatability1: deviation on the distance between stars measured multiple times. we need to put AM1 value, design goal on the plots
      • Many sky plots of residuals binned over the focal plane to look for camera-dependent effects. We can see some small coherent effects on the photometric residuals
      • Histograms may not be as readable as we want; stay tuned for updates to histograms.
      • Jeff to place vertical line better in PR! (DM-40768)
    • Lauren notes that the hotfix to dynamic detection that was applied during the internal PDR2 reprocessing (to ignore the INTRP mask plane when putting down sky objects) was merged to main in w33.  We don't seem to be seeing any big changes, which is a good thing. 
  • Recall what we expected: Recall: expected changes include QBB, matchedVisitMetrics run during step8, rubin-env7. Review the w_2023_35 rerun:Recall 2023 history:
    • w03: Issues with using finalize characterize PSFs downstream. 
    • w07: ip_diffim. Cannot find plugin (fixed on DM-38209) run with ticket branch. Issues with finalizeCharacterize downstream gone.
    • w11: GBDES on.  detectAndMeasure segfaults (fixed with hours to spare for w15) 
    • w15: quasi-random skyObject placement (DM-23781), source selectors use isPrimary (DM-39141)
    • w19: Parallax and PM on (DM-37943), astrometric match improved (DM-38808), cleared mask plan of the template. (DM-38901)
    • w23: RA/Dec columns renamed
    • w27: No major pipeline changes
    • w32: Use GAIA DR3 refcat in GBDES
      • w35:
  • What pipeline changes do we expect for w_2023_39? (to be tagged Step 27)
    • MAYBE AA and AB
    • Eli may change the scale size scatter computation and cut.
    • scarlet lite as a standalone package AND the fix to conditionally skip sources that don't have enough bands.
    • Updating the FGCM lookup table which we expect to change the photometric solution
    • rho stats plots from analysis_tools are looking very similar to analysis_drp, so the latter is probably ready to be deleted.
  • Show and tell:
  • AOB:
    • Given w39 tagged the 27th, a week and a half processing would mean meeting next Oct 9. Is this possible?
      • Not sure. Some things not working smoothly with CM tools. Hard to react quickly with CM visualization stuff also broken.




Action Items

DescriptionDue dateAssigneeTask appears on
  • Add a plot with fakes stats to the dashboard. Sophie Reed 
04 Sep 2020Sophie ReedDRP Metrics Monitoring 2020-08-07
  • Clare: add analyzeMatchedVisitsCore to drp_pipe step8
DRP Metrics Monitoring 2023-06-26
  • Sophie: make a new list for outstanding analysis_drp plots that require moving, send to Jim
DRP Metrics Monitoring 2023-06-26
  • turn catchFailures on in calibrate. Add flag to indicate that deblender failed because PSF is bad. 
DRP Metrics Monitoring 2022-10-31
  • Yusra AlSayyad Eric's account was deleted; we need to make sure he has all his logs. 
Yusra AlSayyadDRP Metrics Monitoring 2021-06-14
  • Arun Kannawadi Modify rho stats in pipe_analysis  to use debiased moments (see  DM-30751 - Getting issue details... STATUS ). 
Arun KannawadiDRP Metrics Monitoring 2021-04-19
Arun KannawadiDRP Metrics Monitoring 2021-03-01
  • Yusra AlSayyad Do a rerun with w50 PS1 refcat and one with shrunk refcat errors. 
Yusra AlSayyadDRP Metrics Monitoring 2021-01-04
  • Jeffrey Carlin Add an absolute astrometry match-to-refcat metric to dashboard  DM-34153 - Getting issue details... STATUS
Jeffrey CarlinDRP Metrics Monitoring 2021-01-04