2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

regrets

Topics for discussion:

  • Project updates (Eric Bellm , Ian Sullivan ):
  • Outbrief from SC Chairs + PST DIA talk (Eric Bellm ):
    • thanks to everyone who contributed materials
    • talk & discussion were cut to 30 minutes due to schedule discussion, so little chance for detailed Q&A.  No substantive discussion.  Post-talk feedback was positive.
  • Auxtel processing status (all):
    • no new data since last meeting–AuxTel still cooling down
    • last two runs didn't get data, and we're moving on to new fields which will need new templates–somewhat of a dry spell.  Does test PP through SFM
    • Meredith: DRP has reprocessed data with some deeper templates, including DIA–so one place to look for AuxTel DIA
    • mainly DRP folks sprinting this week on LATISS config improvements
  • Pipelines Infrastructure (Krzysztof Findeisen , John Parejko ):
    • both out
  • Precursor processing & Campaign Management ( Kenneth Herner , Erin Howard):
    • Ken: what happened with flags?  Eric: RFC pending, comment there.
    • Erin: had reprocessing run with alerts scheduled–how to retry?
      • Eric: let's do whole run without alerts, and then afterburn alerts from a few visits?
  • Satellites (Meredith Rawls ):
    • streak masking very close to merge–Jenkins hitting errors on schemas
    • lots of AAS sessions on satellites planned
  • Image differencing algorithms, DCR  ( Bruno Sanchez , Ian Sullivan ):
    • Bruno:
      • merged ticket to propagate injection masks into diffim
      • made some progress replacing old injection catalog with source_injection–doesn't think there will be any big changes
      • now wrestling with migration of injection tasks.
      • and reviewing Lee's documentation ticket for source_injection
      • will chat with Lee about all of the above
    • Ian
      • pings Eric for review of preconvolution
      • new tickets: following up DRP DC2 processing to understand handful of diffim failures which are new since the diffim refactor.  Identified a couple of bugs due to no longer rerunning detection in diffim–means that code doesn't have access to template flags
      • thinks passing more limited set of kernel candidates will speed diffim
      • with CalibrateImageTask, we see very slightly worse astrometry (0.03" scatter vs 0.005", spec met by both)–but number of dipoles blows up with worse astrometry.   Digging into it with John.  Some weirdness is DC2-related (PSFs had strange unphysical assymetries), but maybe not all.
        • Eric: would be good to develop an analytic way to estimate dipole prevalence
  • alert distribution (Brianna Smart , Eric Bellm ): 
    • Bri: still awaiting RFC on confluent Kafka; trying some local tests in the meantime but mainly was working on streak tickets last week.
  • ML (Nima Sedaghat Harshit Rai , Eric Bellm ):
    • out
  • Solar-system processing (Ari Heinze, Kurlander Jake  ):
    • Ari: running modernized version of heliolincc, working on updated documentation
    • Jake: started planning structure of association pipeline, seeking feedback on design:
      • pre-compute intermediates at the start of the night
      • take next_visit prediction for time/position
        • (discussion of expected precision: we'll follow up in #dm-ssp)
        • need to figure out buffer zone and polynomial degree
        • best boresight/center comes attached to calibrated exposure, when time to run association
          • Eric thinks it's fine to trigger a recompute if the actual next visit is way is out of bounds, we should be robust
  • Review CI (https://usdf-rsp-dev.slac.stanford.edu/chronograf/):
    • we see a few-DIASource jump due probably to DM-39970.  This is weird because it's a DRP-config, but it changed the defaults everywhere and perhaps affected some plugins that maybe weren't inheriting the configs we set at a higher level?  This is speculative, feels like spooky action at a distance.
  • Review outstanding action items
  • QA meeting November 6:
    • metrics sprint report
    • streak masking (though Meredith may not be available to present due to ADASS)
    • transition to CalibrateImageTask and astrometry/dipole concern
  • AOB
    • sprint planning, ops sprint planning
    • no live standup tomorrow


Action Items

DescriptionDue dateAssigneeTask appears on
  • Eric Bellm write up retention policy and sizing for APDB  
17 Jul 2023Eric BellmAP Pipeline Meeting, 2023-06-26
  • Eric Bellm dig into alert serialization test in ap_association and ticket ways to improve it  
11 Sep 2023Eric BellmAP Pipeline Meeting, 2023-08-21