DRP Metrics Monitoring 2023-01-30

Mondays 12PT (3 - 3:50pm ET)

Yusra's Zoom: https://princeton.zoom.us/my/yusra

1.1.1. Attendees: 

• Dan Taranu 
• Orion Eiger 
• Hsin-Fang Chiang 
• Yusra AlSayyad 
• Joshua Meyers 
• Robert Lupton 
• Fred Moolekamp 
• Clare Saunders 
• Sophie Reed 
• Sierra Villareal 
• Keith Bechtol 
• Agnès Ferté 
• Jeffrey Carlin 
• Christopher Waters 
• Eli Rykoff 
• Arun Kannawadi 
• Lee Kelvin 
• Erfan Nourbakhsh 

Regrets: Lauren MacArthur (dr.’s appointment)

1.1.2. Agenda:

• Meeting recorder - Colin (last 6 meetings were:  Jim, Keith, Fred, Nate, Jeff, Clare) 
• Announcements
• Review Action items from last month
  • Lauren MacArthur :  DM-37411 - Getting issue details... STATUS  merged, so should no longer see failures in calibrate.deblender along the lines: Execution of task 'calibrate' on quantum {instrument: 'HSC', detector: 101, visit: 11706, ...} failed. Exception ValueError: PSF at (1908.3, 3245.7) has an invalid FWHM value of nan
  • Jim added PSF-based selection of images for coadds, but not in time w01 which is this week's rerun.
  • Wanted to start CharImage using PSFEx asap for NAOJ, give more time for other Piff Robustification work. Will find out how many non-robustnesses remain with w03. Were still some images that didn't have an obvious excuse for failing, will see how they turn out in w03.
  • Still outstanding why 9697 patch 7 succeeded in w48.
    • Double check that it was actually w48 that worked? Orion Eiger  double checked, but can triple check. → Slack.
  • analysis_tools failure in 48 was a mystery, but has been solved. Hsin-Fang Chiang  : see this comment on the w48 ticket for the longer explanation. For this week (w_2023_01), had to set the instrument to be HSC, then runs fine.
    • For other yamls, the instrument was built into the yaml, didn't need to add --instrument. Why is this different? Hsin-Fang Chiang : we don't have a yaml in drp_pipe for this, should we add one? Yes  (new ticket --> DM-37784)
    • Policy proposal: "never run pipelines from a yaml that's not in drp_pipe"? Maybe not never, but anything that campaign management runs should be in drp_pipe.
  • Add sky coverage plots: Lauren wanted feedback on what ones we wanted, but the capability is there. Sophie has bugfixes on the todo list.
• Processing Status
• Review w_2023_01 rerun:
  • Same error messages as before, details are on the Jira ticket.  DM-37483.
  • Recall history:
    
    • w12: psfex
    • w16: piff  (bad size residuals)
    • w20: finalizeCharacterize (bad apcorr configs → bad stellar locus)
    • w22: lanczos11 + apcorr configs (better stellar locus and size residuals approx equivalent to psfEx!
    • w24: PIFF kernelSize to 25. new scarlet lite storage.
    • w28: attempt at fixing measure failures:  DM-35722 - Getting issue details... STATUS
    • w32: First RC2 at SLAC. subtractImages compatibility mode on
    • w36: subtractImages Compatibility mode off
    • w40: 
    • w44:
    • w48: 9697/7 succeeds, extra subtractImage failures gone, 
  • Chronograf, plot-navigator review of _01
  • New Plots:
    • skyObjectHistPlot, added a one-arc-second GaaP plots. .
      • Conclusion: Both gaap and aperture underestimate the noise. S/N is per-tract but not per band? Plot title is wrong, plot is single-band.
      • Eli Rykoff: Normalize left plot as Flux per square arcsecond? Because aperture is changing between algorithms. Noise-effective area would be even better?
    • New Rho stats plot, combines three tracts. Was hoping that a bias was coming from only having small areas, but even after combining the result was still biased.
      • How did you combine the tracts? Runs on all tracts specified in the data query.
  • Interlude while waiting for plots:
    • Spoiler alert. More details on Wednesday
      
      • Looking at source injection, injecting quasars into HSC. Seeing z and y offsets.
  • Computational performance:
    • Jim wrote build-gather-resource-usage-qg. One row for every data ID, gives memory and runtime for every task.
    • Orion Eiger  will run after RC2/DC2, so we can make a chronograph dashboard. Monitor when things change or get worse. Hsin-Fang Chiang ran on w01. Wouldn't work in bps (have to find how to tell bps to use an existing qgraph), but worked with pipetask.
  • Next run: Sophie expects photometry vs ref cat plots.
• AOB:
  • DC2 will be run on the version tagged this upcoming wednesday.
  • Next meeting after w03 is done.

1.1.3. Action Items