DRP Metrics Monitoring 2021-05-17

Mondays 12pm - 12:50pm EST

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees: Arun, Chris (Waters), Dan, Fred, Hsin-Fang, Jeff, Jim (Bosch), Josh, Lauren, Leanne, Lee, Morgan, Nate, Robert, Sophie, Yusra

Agenda:

• Review Action items from last week
  
• Review w_2021_18
  • What was noteworthy in this rerun?
    • Gen2:  Missing matchedVisitMetrics, objectTable_tract.   https://lsst.ncsa.illinois.edu/~yusra/RC_QA/w_2021_18/
    • Gen3: Yay!  First version to complete. Includes diffim.
      • (does not include 9813 narrow-band) DM-29944
  • What changed? Annotate
• What do we expect from w_2021_22? 
  • New features:
    • debiased HSM moments
    • parquet ForcedPhotTable
  • Changes toward Gen2-Gen3 parity:
    • DRP Gen2 to Gen3 Pipeline Migration Coordination
    • a few lingering SFM differences revealed in w14 RC2 runs: see DM-29881 - Getting issue details... STATUS &  DM-30030 - Getting issue details... STATUS
      • LM: How do account for padding in raw vs jointcal wcs.  HSC default is 250. default default is 0. There are a bunch of places where padding is definded "pixelMargin" should be < 250 padding default. Proposal: set all the defaults to 250. Not a big deal for DC2. Might be for DECam.  
      • Want to be able to run faro on both gen2 and gen3 reruns of w_2021_18. Monika ran faro on the Gen3 repo. Haven't run the the gen2-gen3 converted repo yet.

      • Clare noticed a satellite trail is missing in the gen3 version that's still in the gen2 version. Get patch number from Clare: 9813, i-band, patch, '4,8'=76 

        • Generate a sky frames that are more similar to the ones that NAOJ are using (created DM-31401 - Getting issue details... STATUS ). Lauren MacArthur
        • Copy Lauren's Gen2 SkyFrames to the Gen3 repo (awaits decisions to be made on and execution of DM-31401 - Getting issue details... STATUS  which may moot this task if we opt to wait for post-Gen2) Jim Bosch
  • What new metric can we expect next time
• Review DC2
  • w_2021_16:
    • DT: Surprising OK. We were missing configs so we definitely won't have parity between gen2-gen3 w16.  w20 SHOULD have the same parity that HSC does. 
      • Ran faro the gen3 w16 and started the gen2 w16. Noticed some patches are missing. Need to check if there's anything missing. 
      • There's a script that Simon wrote: "makeJobDoc" which will make the json. 
      • DT: how are we going to Manage all the dashboards. 
    • LM: If you see differences between gen2 and gen3 is probably configs. 
      • Go through dashboards and see if we can prune any or mark them legacy
  • Gen3 Pipeline for DP0.2
• AOB

Action Items