Mondays 12pm - 12:50pm ET

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Postponed from Feb 7.

Attendees:

Yusra AlSayyad , Eli Rykoff , Fred Moolekamp , Nate Lust , Jim Bosch , Angelo Fausti , Unknown User (brendal4) , Sophie Reed , Lauren MacArthur , Dan Taranu , Christopher Waters , Colin Slater , Jeffrey Carlin , Clare Saunders , Arun Kannawadi , Joshua Meyers , Lee Kelvin , Leanne Guy 

Agenda:

  • Announcements:
    • We can review w08 too! 
  • Review Action items from last week
  • Processing Status
  • Review HSC RC2 rerun w_2022_04
    • What was noteworthy in this rerun? How did processing go?
      • w04: DM-33402
        • BB: How to tell faro not to run on the narrow-bands?
          • Jeffrey Carlin  Make faro not attempt running on narrow-bands.  
        • slow because of filesystem overload and issues.
        • wPerp needed to be run on lsst-devl
      • w08: DM-33741
        • tracts 9697/9615 wPerp succeeded on batch nodes!
        • step5 has tasks that take a lot of memory.
        • New image difference error
      • Expected changes
        • matchedCatalogsTract changed, skyCorr run as of w50, new skyFrames 
        • scarlet lite was turned on for w08!!!
          • There is a config option to switch back.
    • What changed in the metrics?
      • Big change in repeatabilty metrics between w04 and w08. What do we do?
        • Run w04 faro on w08 outputs?  Unknown User (brendal4) Jeffrey Carlin 
        • We expected scarlet lite to reduce runtime and memory. File ticket for adapting what ap_pipe uses. Label PairCoding. 
        • Disentangling effect of sky frames and scarlet lite on w08 Fred Moolekamp 
      • YA: see DM-33863 for pair coding ticket
    • What do we expect from w_2022_08? (Tagged Feb 16)
      • What new metric or plot can we expect next time?
      • Option 2b
  • Review DC2
    • w_2022_03
  • AOB
    • SR: The 70th mag stars are gone! 
    • DT: Main diff in scarlet lite in DC2 is the bright and saturated (just below triggering the flag) stars used to be classified as extended and now they're not.  The correct thing to do is to reconcile the stars used in the metrics before and after the matching change. 
    • LM: Gen3 pipelines are now scientifically equivalent to Gen2!!!!!! Yay!!!!!!
    • JFB: which means that we can stop the gen2 reruns. 
    • We need to port over the quick checks at the end of ci_hsc_gen2 to ci_hsc_gen3


Action Items

DescriptionDue dateAssigneeTask appears on
  • Add a plot with fakes stats to the dashboard. Sophie Reed 
04 Sep 2020Sophie ReedDRP Metrics Monitoring 2020-08-07
  • Sophie to add field in metric definition to hold thresholds. DM-43364 - Getting issue details... STATUS
DRP Metrics Monitoring 2024-03-18
  • Sophie: make a new list for outstanding analysis_drp plots that require moving, send to Jim
 DRP Metrics Monitoring 2023-06-26
  • Clare: add analyzeMatchedVisitsCore to drp_pipe step8
 DRP Metrics Monitoring 2023-06-26
  • turn catchFailures on in calibrate. Add flag to indicate that deblender failed because PSF is bad. 
 DRP Metrics Monitoring 2022-10-31
  • Yusra AlSayyad Eric's account was deleted; we need to make sure he has all his logs. 
 Yusra AlSayyadDRP Metrics Monitoring 2021-06-14
  • Arun Kannawadi Modify rho stats in pipe_analysis  to use debiased moments (see  DM-30751 - Getting issue details... STATUS ). 
 Arun KannawadiDRP Metrics Monitoring 2021-04-19
Arun KannawadiDRP Metrics Monitoring 2021-03-01
  • Yusra AlSayyad Do a rerun with w50 PS1 refcat and one with shrunk refcat errors. 
 Yusra AlSayyadDRP Metrics Monitoring 2021-01-04
  • Jeffrey Carlin Add an absolute astrometry match-to-refcat metric to dashboard  DM-34153 - Getting issue details... STATUS
Jeffrey CarlinDRP Metrics Monitoring 2021-01-04


Sky Frames w04 vs w08

Comparing 'HSC/runs/RC2/w_2022_04/DM-33402-b' against 'HSC/runs/RC2/w_2022_08/DM-33741':

deepCoadd_calexp:

deepCoadd: