Data Management
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata


Mondays 12pm - 12:50pm EST

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees: 

Yusra, Andrés, Arun, Chris (Waters), Clare, Colin, Dan, Eli, Hsin-Fang, Ian, Jim, Josh, Lauren, Leanne, Lee, Merlin, Morgan, Nate, Robert, Simon, Sophie, Tim (Morton)

Agenda:

  • Review Action items from last week
  • Review w_2021_14 
    • What was noteworthy in this rerun?
      • Gen2: Done and converted to Gen3 (last few gen2 reruns are in /repos/main
      • Gen3: Lots of little issues revealed, most fixes in by 16.  9813-only of w16 to see if we can do it in 2 QGs. Not knowing how much memory takes is the big question and just testing that it all works after w16. 
      • Likewise, Dan is running the DC2 dataset through the Gen3 pipeline and is working through configs. 
      • LM: Have parity up to the end of single processing (calexps), and am moving on to jointcal/FGCM.
      • Still need to run and compare faro on both the gen2 and gen3 reruns. 
        • DM-29729 → pair-coding
        • Ensure that isPrimary column is set/consistent in fields where we don't have full band coverage and scarlet doesn't run (see DM-29672 - Getting issue details... STATUS  for some plots and details). Fred Moolekamp 
    • What changed? Annotate
      • JM: There’s a new algorithm for measuring moments available to remove a bias that scales with signal to noise. Basically by adding noise to the high-SNR to look like low SNR.
        • Joshua MeyersTurn on debiased HSM moments in obs_subaru, obs_decam, and obs_lsst. 
        • Arun Kannawadi Modify rho stats in pipe_analysis  to use debiased moments (see  DM-30751 - Getting issue details... STATUS ). 
    • What do we expect from w_2021_18? 
      • New features:
        • Debiased HSM moments.
        • High-level pipeline files to new packages. See draft RFC out later today. 
        • analysis_drp? Probably not. 
      • Changes toward Gen2-Gen3 parity:
      • What new metric can we expect next time
  • Review DC2
    • w_2021_12:
      • Jim will convert DC2 gen2 rerun w_2021_12 → /repo/dc2. Someone from the faro team will run faro. 
    • Gen3 Pipeline for DP0.2
  • AOB


Action Items

DescriptionDue dateAssigneeTask appears on
  • Add a plot with fakes stats to the dashboard. Sophie Reed 
04 Sep 2020Sophie ReedDRP Metrics Monitoring 2020-08-07
  • Sophie: make a new list for outstanding analysis_drp plots that require moving, send to Jim
 DRP Metrics Monitoring 2023-06-26
  • Clare: add analyzeMatchedVisitsCore to drp_pipe step8
 DRP Metrics Monitoring 2023-06-26
  • turn catchFailures on in calibrate. Add flag to indicate that deblender failed because PSF is bad. 
 DRP Metrics Monitoring 2022-10-31
  • Yusra AlSayyad Eric's account was deleted; we need to make sure he has all his logs. 
 Yusra AlSayyadDRP Metrics Monitoring 2021-06-14
  • Arun Kannawadi Modify rho stats in pipe_analysis  to use debiased moments (see  DM-30751 - Getting issue details... STATUS ). 
 Arun KannawadiDRP Metrics Monitoring 2021-04-19
Arun KannawadiDRP Metrics Monitoring 2021-03-01
  • Yusra AlSayyad Do a rerun with w50 PS1 refcat and one with shrunk refcat errors. 
 Yusra AlSayyadDRP Metrics Monitoring 2021-01-04
  • Jeffrey Carlin Add an absolute astrometry match-to-refcat metric to dashboard  DM-34153 - Getting issue details... STATUS
Jeffrey CarlinDRP Metrics Monitoring 2021-01-04