Mondays 12PT (3 - 3:50pm ET)

Yusra's Zoom:




  • Meeting recorder: Jim (last 6 meetings were:  Clare, Dan, Erfan, Lee, Robert, Arun) 
  • Announcements
  • Action items from last time:
    • Sophie sent list of outstanding analysis_drp plots that require moving...or did she?!  Jim and Colin do not remember, but Sophie says she has the receipts.
    • New action: put it somewhere public (didn't catch where Yusra suggested).
  • Processing Status
    • Processing of w_2023_39 RC2 (Orion)
  • Review the w_2023_39 rerun:
    • Recall what we expected:
      • MAYBE AA and AB
      • Eli may change the scale size scatter computation and cut. ( DM-40668 - Getting issue details... STATUS )
        • Did not make it for _39 (still needs to go into review).
      • scarlet lite as a standalone package AND the fix to conditionally skip sources that don't have enough bands.
        • Latter made it for _39, former did not (but is on main).
      • Updating the FGCM lookup table which we expect to change the photometric solution
        • Did not change the lookup table on main; we changed the logic instead.  Merged to main but did not make _39.
      • rho stats plots from analysis_tools are looking very similar to analysis_drp, so the latter is probably ready to be deleted.
    • How it went:
      • PanDA just got its own setup at USDF.  Many problems.
      • DM-41008 made things very slow and exacerbated PanDA issues.
        • Still trying to finish up step4.
        • Might need help coming up with workarounds/ticket branches needed to finish this off.
      • No other new pipeline errors.
      • Ran what analysis tasks we could with the state the processing got to.
        • No matched-visit metrics.
        • No obvious changes to coadd-based metrics.
      • Last DC2 run:
        • Got metrics from a few irrelevant tracts due to recent QG generation changes; need to filter those out.
        • No interesting changes.
    • Recall 2023 history:
      • w03: Issues with using finalize characterize PSFs downstream. 
      • w07: ip_diffim. Cannot find plugin (fixed on DM-38209) run with ticket branch. Issues with finalizeCharacterize downstream gone.
      • w11: GBDES on.  detectAndMeasure segfaults (fixed with hours to spare for w15) 
      • w15: quasi-random skyObject placement (DM-23781), source selectors use isPrimary (DM-39141)
      • w19: Parallax and PM on (DM-37943), astrometric match improved (DM-38808), cleared mask plan of the template. (DM-38901)
      • w23: RA/Dec columns renamed
      • w27: No major pipeline changes
      • w32: Use GAIA DR3 refcat in GBDES
      • w35: 
  • Nightly rc2_subset runs.
    • Lee reports yPerp has been fluctuating more lately (since ~w39). Could not get Chronograph to show this live; something seems awry with the dashboard (link, screenshot below).
  • What pipeline changes do we expect for w_2023_43?
    • Scaled-size scatter cuts
    • Reference-catalog matching plots need to get closed out.
    • Manifest checker: counts of quanta that failed and succeeded
    • scarlet lite as a standalone package 
    • FGCM logic change not lookup table
  • PDR2 Tanaka-san plots (Sophie Reed )
  • Show and tell:
  • AOB:

Action Items

DescriptionDue dateAssigneeTask appears on
  • Add a plot with fakes stats to the dashboard. Sophie Reed 
04 Sep 2020Sophie ReedDRP Metrics Monitoring 2020-08-07
  • Sophie: make a new list for outstanding analysis_drp plots that require moving, send to Jim
DRP Metrics Monitoring 2023-06-26
  • Clare: add analyzeMatchedVisitsCore to drp_pipe step8
DRP Metrics Monitoring 2023-06-26
  • turn catchFailures on in calibrate. Add flag to indicate that deblender failed because PSF is bad. 
DRP Metrics Monitoring 2022-10-31
  • Yusra AlSayyad Eric's account was deleted; we need to make sure he has all his logs. 
Yusra AlSayyadDRP Metrics Monitoring 2021-06-14
  • Arun Kannawadi Modify rho stats in pipe_analysis  to use debiased moments (see  DM-30751 - Getting issue details... STATUS ). 
Arun KannawadiDRP Metrics Monitoring 2021-04-19
Arun KannawadiDRP Metrics Monitoring 2021-03-01
  • Yusra AlSayyad Do a rerun with w50 PS1 refcat and one with shrunk refcat errors. 
Yusra AlSayyadDRP Metrics Monitoring 2021-01-04
  • Jeffrey Carlin Add an absolute astrometry match-to-refcat metric to dashboard  DM-34153 - Getting issue details... STATUS
Jeffrey CarlinDRP Metrics Monitoring 2021-01-04