DRP Metrics Monitoring 2024-02-26

Mondays 12PT (3 - 3:50pm ET)

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees:

• Jeffrey Carlin
• Yusra AlSayyad
• Lauren MacArthur
• Dan Taranu
• Clare Saunders
• Sophie Reed
• Arun Kannawadi
• Keith Bechtol
• Orion Eiger
• Eli Rykoff
• Jim Bosch
• Erfan Nourbakhsh
• Joshua Meyers 
• Trys McCann 
• Fred Moolekamp

Regrets:

Agenda:

• Meeting recorder - Jeff; last 5 meetings were: Lee Keith, Orion, Jim, Clare

• Announcements
• Action items from last time:
  
  • CModel - PSF plots should get products DONE
  • Maybe number of sky plots should be reduced, taken out of core pipeline, replace with scatter plot: Not Done (done for one, but not systematically for all)
  • Figure out why survey property maps not appearing in plot navigator: What's the hold up on this PR?? DONE
  • we don't have a good sense right now if the plots we're looking at are showing data near the "danger zone" or not;
    • We don't know if we're approaching "danger" with the current metrics, as we're not comparing vs. thresholds.
    • JB: should have a policy that metrics are born with those thresholds (and grandfather that in for existing metrics)
  • Yusra: Generate resource usage plots for w_2024_02 (DONE, posted in this thread: https://lsstc.slack.com/archives/C4JQP6FRS/p1705962681642319)
  • Trys: Deep dive into generation of synthetic input catalog magnitude distributions; DONE(?) – new tool that generates uniform distributed mags in generate_injection_catalog.

• Processing Status
  • Processing of w_2024_04 DC2 and  w_2024_06 RC2 (Jen/Orion)
• Review the w_2024_06 rerun DM-42797 - Getting issue details... STATUS :
  • https://usdf-rsp.slac.stanford.edu/plot-navigator/dashboard_gen3
  • https://usdf-rsp-dev.slac.stanford.edu/chronograf/
• Recall what we expected for w06:
  • From before: new analysis tools potentially (w48 with followup over the weekend. photometric ref cat matches)
  • From before: Healsparse property maps pngs made by analysis_tools this PR
  • Clare predicts that AB and AA will get into metrics; AA is in, but not AB
  • GBDES should be updated (we don't remember what this referred to?)
  • wPerp ticket should be merged by w_2024_04 (DM-39081 made it in for w_2024_05)
    
    • Recall 2023 history:
      • w03: Issues with using finalize characterize PSFs downstream. 
      • w07: ip_diffim. Cannot find plugin (fixed on DM-38209) run with ticket branch. Issues with finalizeCharacterize downstream gone.
      • w11: GBDES on.  detectAndMeasure segfaults (fixed with hours to spare for w15) 
      • w15: quasi-random skyObject placement (DM-23781), source selectors use isPrimary (DM-39141)
      • w19: Parallax and PM on (DM-37943), astrometric match improved (DM-38808), cleared mask plan of the template. (DM-38901)
      • w23: RA/Dec columns renamed
      • w27: No major pipeline changes
      • w32: Use GAIA DR3 refcat in GBDES
      • w35: 
      • w40: scarlet_lite as a standalone package. 
      • w47: shapeHSM wraps Galsim
      • w48: shapeHSM wraps Galsim without slow down. Scarlet_lite standalone package bug fix for suspected cause of wPerp/PSF-metric regression. 
      • w50:
      • w02: 

• • Notes from look at dashboard(s):
    • No noticeable change in most metrics (in either RC2 or DC2). Number of skipped blends increased slightly. (fewer footprints that are too large – maybe a change streak masking or something else that causes "too large" footprints)
  • New/updated plots: 
    • wPerp added a new panel with residuals vs. (g-r) color
    • Photometric residuals vs refcat
    • Healsparse maps! (ER notes that these will look a lot different in time of cell-based coadds; will be a checkerboard of uniform size)
    • cell-based coadds for all coadd measurements, (missed the context for what was being said here...)

• What pipeline changes do we expect for the w_2024_08 (DC2) or w_2024_10 (RC2)?
  • DM-42837 - Getting issue details... STATUS (Hopefully this changes nothing. Would only affect DC2. )
  • DM-41876 - Getting issue details... STATUS  should reduce the number of diaSources. 
  • DM-42663 - Getting issue details... STATUS moments-based star/galaxy classifier. To be tested on ComCamSim1 before March 6. 
• What additional plots or metrics do we expect for the w_2024_08 (DC2) or w_2024_10 (RC2)?
  • Per-tract AA1 – need to make a dashboard for this
  • Sophie's working on all-sky plots; not sure if these will be ready for w_10

• ComCamSim Run DM-42988 - Getting issue details... STATUS

  • analyzeMatchedVisitCore, analyzeObjectTableCore are up on the plot navigator
    • If you find that you are missing your analysis_tools tasks, add them to step3 (or step7) of OR3 pipeline
    • Collection for initial OR3 run: u/homer/htc-test1
    • No pattern visible in photRep with focal plane position. Maybe because it's only the central raft, so you don't get to the wonky outer parts.
    • PSF-cModel, shapeSizeFractionalDiff (and other quantities) shows trends because BF is simulated, but correction not included in processing
    • skyObjectFlux for ap09 is much larger than typical HSC values
    • wPerp has a super-tight distribution (as expected from sims); (e.g., sigma_MAD = 10 in both DC2 and OR3)
    • rho stats running now; should show up by end of day
  • Any other tract level analysis tasks we want to run? We must add them to https://github.com/lsst/drp_pipe/blob/main/pipelines/LSSTComCamSim/DRP-ops-rehearsal-3.yaml
  • AK: should switch to the cell-based skymap. That will change the metrics, 'cause patches won't be the same. When should we do the switch? (JB: want to make sure the ComCam ones start with the cell-based skymap.)

• Show and tell?
• Actions
• AOB:

Action Items