2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020



Topics for discussion:

  • Project updates (Eric Bellm , Ian Sullivan ):
    • Upcoming meeting free weeks: weeks of December 25 and January 1
  • Auxtel processing status (all ):
    • successful single-frame processing again, some more emergent bugs
  • Pipelines Infrastructure (John Parejko, Krzysztof Findeisen ):
    • John: DM-37386 - Getting issue details... STATUS centerAll flag in review, waiting on Jenkins to merge.  Negligible processing time change.
      • (we had flag for "any pixel in footprint is flagged" and "any pixel in 3x3 central is flagged", adds "all 9 pixels in 3x3 central region is flagged")
      • tests show only gets set on a handful of sources in single frame, but 10 (DC2)-50 (HSC)% of diffim sources get this set in small ap_verify tests
      • next step, slightly longer run, examine some cutouts
    • can we remove some of the obs_ override hacks in ap?  John thinks a day or two of effort.
  • Precursor processing & Campaign Management ( Kenneth Herner , Erin Howard):
    • Erin: trying to process full DECam HITS dataset and hitting a new error–isn't able to find refcat for one visit
    • Ken: trying to get meeting set up to discuss glide-in service
  • Satellites (Meredith Rawls ):
    • out today
  • Image differencing algorithms, DCR  ( Bruno Sanchez , Ian Sullivan ):
    • Ian: focus on duplicate DIAObject
    • Should we set a maximum footprint size?
      • formally we expected to send full footprint
      • but we're seeing some enormous alerts
      • DMTN-199 imposes a maximum trail length, suggesting we should be worried about footprints larger than that size
      • we should understand the cause of these enormous footprints
      • we probably need the safety net in addition though
      • we do already have an argument from Chris Morrison long ago but it's not set by default? 
        • nope, it's not in packageAlerts.py
        • maybe Eric was thinking of the maxSize in transformDiaSourceCatalog.py
      • Ian will draft a ticket
  • alert distribution (Brianna Smart , Eric Bellm ): 
    • Bri: don't have a full solution yet for secret handling on USDF, but it's at least out of the files.  Can produce alerts from ap_verify runs to Kafka.
  • ML (Nima Sedaghat Harshit Rai , Eric Bellm ):
    • Nima out
    • Harshit worked on excluding sky sources from RB classification, implementing spuriousness metrics
  • Solar-system processing (Ari Heinze, Kurlander Jake , Mario Juric ):
    • Ari: reached hypothesis saturation on NEOs interior to Earth's orbit by increasing clustering radius.  Now checking effect on MBA, but filtering should be possible if needed–sort of the inverse of the MPC digest score.
  • Review CI (https://usdf-rsp-dev.slac.stanford.edu/chronograf/):
    • modest changes to DIAObject counts.  We should just label all of these sources and do real truth metrics.
    • we should also put in astrometry (and photometry) metrics.  We're not sure if DRP has any single-frame appropriate metrics since theirs are all downstream
  • Review outstanding action items
  • QA meeting December 4:
    • streak masking (Meredith–TBC, ill)
    • status of preloading & Solar System Association?  (Jake)
    • duplicate DIAObject status? (Ian & Erin)
  • AOB
    • watch for sprint planning schedule

Action Items

DescriptionDue dateAssigneeTask appears on
  • Eric Bellm write up retention policy and sizing for APDB  
17 Jul 2023Eric BellmAP Pipeline Meeting, 2023-06-26
  • Eric Bellm dig into alert serialization test in ap_association and ticket ways to improve it  
11 Sep 2023Eric BellmAP Pipeline Meeting, 2023-08-21
  • Eric Bellm review broker communication channels  
25 Sep 2023Eric BellmAP Pipeline Meeting, 2023-08-14