Mondays 12PT (3 - 3:50pm ET)

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees:

Regrets:

Agenda:

  • Meeting recorder - Erfan (last 6 meetings were: Lee, Robert, Arun, Clare) 
  • Announcements
  • Action items from last time:
    • Sophie: make a new list for outstanding analysis_drp plots that require moving, send to Jim (Sophie sent it to Collin instead - it's now on a computer) 
      • LAM: looks like histPlot[skyPlot]_skyObject_i can be removed (replaced by objectTableCore_i_skObjectFlux[SKy]_Hist[Sky]Plot), e.g. (new on left, old on right).
      • Q: was the decision to leave off the patch outlines deliberate? Sophie Reed: No
      • In fact, most (all?? ...the plot browser keeps freezing up on me, so I couldn't check them all) of the scatterTwoHistPlot_{etc}, skyPlot_{etc}, and rho{N}_deltaCoords plots have been superseded? E.g.
      • Q: why include the "All" histogram on the new versions since it's clearly just the stars? Sophie Reed: It is a bug/feature.
      • We should delete everything from analysis_drp plot that are being made in analysis_tools (Yusra: Let's make a ticket to do this!)
      • These 2 are two examples of many where new is almost as good as old. 
      • File a ticket to add patch lines to sky plots. (Sophie: it's not worth having it!)
    • add analyzeMatchedVisitsCore still not in step8
  • Processing Status
    • Processing of w_2023_27 RC2 (Orion) DM-39959 - Getting issue details... STATUS
      • expected changes: 
        • Eli: three detectors in RC2 that were formerly crashing now fixed - merged!
        • Orion: cm_tools changes:
          • when one step is complete, the next will now auto-begin (DM-39610)
        • RA/Dec columns were renamed
      • report: Orion: We ran into the same issues we had before with subtractImage, warnings from aperture corrections etc. but everything ran smoothly eventually. 
      • Jim: Do we have any tickets to fix those image differencing errors (they are not real problems - maybe we should make them non-errors)? Yusra: I don't think we have any tickets. Jim:  DM-40171 - Getting issue details... STATUS .
      • chronograf report: Stellar locus is either down or the same. Since we changed the name of the metric, one color is the old metric, and the other one is the new metric. Fractional size residual looks unchanged; same for e1 and e2 residual - except the r-band is getting us nothing. We are getting no r-band metrics!
      • memory / runtimes (w23 → w27)
  • Review the w_2023_27 rerun:
    • Recall what we expected:
      • Clare:  repeatability metrics (both astrom/photom) on sasquatch) reinterpreted AFx. So we expect the baseline AF values to the higher than we were used to before.
      • Those 3 errors in step 1 gone! (They just moved to step 4)
      • Lee: Some of the failures (in detectAndMeasureDiaSources) we are seeing here with w_2023_27 are not seen in w_2023_25 on Tiger (DM-39770, step 4 errors).
        • Tiger collection: HSC/runs/RC2/w_2023_25/DM-39770 
        • Orion has double checked and this is in all their logs
        • Eli: DM-39626 was the fix.
        • Mystery is that 1 of those 3 dataIDs didn't fail: visit=36180, detector=34.
    • Recall 2023 history:
      • w03:  Issues with using finalize characterize PSFs downstream. 
      • w07: ip_diffim. Cannot find plugin (fixed on DM-38209) run with ticket branch. Issues with finalizeCharacterize downstream gone.
      • w11: GBDES on.  detectAndMeasure segfaults (fixed with hours to spare for w15) 
      • w15: pseudo- quasi-random skyObject placement (DM-23781), source selectors use isPrimary (DM-39141)
      • w19: Parallax and PM on (DM-37943), astrometric match improved (DM-38808), cleared mask plan of the template. (DM-38901)
      • w23: RA/Dec columns renamed
      • w27: 
  • What do we expect for w_2023_31? 
  • AOB:
    • stellarAstrometricRepeatability3 plots are empty,
    • YA: We don't expect (inter-tract scale astrom repeatability metrics) AM3/AD3/AF3 for RC2s because we don't have neighboring tracts.
    • Clare is going to add the proper motion correction to the metrics.
    • Lee: there is a lot to talk about the sky sources in the future.




Action Items

DescriptionDue dateAssigneeTask appears on
  • Add a plot with fakes stats to the dashboard. Sophie Reed 
04 Sep 2020Sophie ReedDRP Metrics Monitoring 2020-08-07
  • Clare: add analyzeMatchedVisitsCore to drp_pipe step8
DRP Metrics Monitoring 2023-06-26
  • Sophie: make a new list for outstanding analysis_drp plots that require moving, send to Jim
DRP Metrics Monitoring 2023-06-26
  • turn catchFailures on in calibrate. Add flag to indicate that deblender failed because PSF is bad. 
DRP Metrics Monitoring 2022-10-31
  • Yusra AlSayyad Eric's account was deleted; we need to make sure he has all his logs. 
Yusra AlSayyadDRP Metrics Monitoring 2021-06-14
  • Arun Kannawadi Modify rho stats in pipe_analysis  to use debiased moments (see  DM-30751 - Getting issue details... STATUS ). 
Arun KannawadiDRP Metrics Monitoring 2021-04-19
Arun KannawadiDRP Metrics Monitoring 2021-03-01
  • Yusra AlSayyad Do a rerun with w50 PS1 refcat and one with shrunk refcat errors. 
Yusra AlSayyadDRP Metrics Monitoring 2021-01-04
  • Jeffrey Carlin Add an absolute astrometry match-to-refcat metric to dashboard  DM-34153 - Getting issue details... STATUS
Jeffrey CarlinDRP Metrics Monitoring 2021-01-04