DRP Metrics Monitoring 2021-06-14

Mondays 12pm - 12:50pm EST

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees:

Yusra, Arun, Chris (Waters), Clare, Dan, Eli, Fred, Hsin-Fang, Ian, Jim (Bosch), Josh, Lauren, Leanne, Lee, Monika, Merlin, Morgan, Nate, Robert, Sophie

Agenda:

• Review Action items from last week
  
  • New refcat loader that will make it easier for faro to load in refcats. DM-30054
• Review w_2021_22
  • What was noteworthy in this rerun?
    • Gen2: Issues with singleFrameDriver related to out-of-bounds psf ticket. Lost 9813 during mulitband driver.  SkyCorrection required more time than last rerun. 9615 HSC-G.  see if transient.  faro on the gen2-gen3 conversion pending Eli's afterburner converter. 
    • Gen3: Failures for HSM PSF. and a missing skymap. Some failures with the narrowband filter for filter. 
      • Set up appointment for Simon to teach us how to link faro to squash. Yusra AlSayyad
  • What changed? Annotate
• What do we expect from w_2021_26? 
  • New features:
    • No HSM crashing!  DM-30426 
      • ER: This is a new feature. Before this would have corrupted memory. 
    • New NoWorkFound "expected failures" DM-30649
    • Fix for singleFrame astrometric matching for DC2 DM-30490
    • analysis_drp plots. 
  • Changes toward Gen2-Gen3 parity:
    • DRP Gen2 to Gen3 Pipeline Migration Coordination
  • What new metric can we expect next time
• Review DC2
  • w_2021_20:
    • Gen2: Eric
      • Yusra AlSayyad Eric's account was deleted; we need to make sure he has all his logs. 
      •  
    • Gen3:  
  • w_2021_24:
    • Gen3: Done. Some errors. ShapeHSM error. Two WriteSourceTable jobs failed with no space on disk. Some failures for image difference DM-30649 on its way. Takes only 2 days if there are no code problems.  
      • Monika will rerun two writeSourceTable jobs just to see if they're transient. JB:  BPS will do automated retries which will fix but also hide these.  RL: When? JB: Weren't expecting on having to do retries at these smaller scales reruns. We can tell HTCondor to do this but I'm not sure the details. 
    • Gen2: Completely blocked by brighter fatter correction. CZW: Converting the existing kernels to be read in should not be too hard and I'll file a ticket today. DM-30738
      
      • DT: Then why does the bright star photometry look OK for the older gen2 w_2020_48? LM: we were not doing BF correction in Gen3, we just turned it on. Turning it on broke the Gen2 rerun.
        • Compare w20 and w24 to confirm that BF is turned on and functioning. This is now confirmed for gen3, see  DM-30747 - Getting issue details... STATUS  for details.
  • Gen3 Pipeline for DP0.2
• AOB
  • DT: I'm interested in seeing what the new debiased moments tell us about PSF modeling.
  • SR: The blendedness column in the objectTable is entirely nans. 
    • Sophie Reed will file a ticket 

Meet again in 2 weeks:

By then we need:

• faro run on gen2-gen3 converted repos, with Eli's afterburner. 

Action Items