2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

regrets:


Topics for discussion:

  • Announcements
    • Welcome in-kind contributors working in transient and variable science unit
  • AuxTel Processing (all):
    • Prompt Processing with AuxTel status (Hsin-Fang Chiang , Eric Bellm , Neven Caplar )
      • generally smooth processing, team continuing to improve process and find issues when images are
      • outputs in /repo/embargo and APDB
      • Neven looking at outputs scientifically--
    • 10am processing status (Colin Slater , Meredith Rawls ):
      • Colin at SCOC, Meredith just back from vacation
    • some discussion that AuxTel observing will go from 3 nights every 2 weeks to 5 nights per week, albeit with less nighttime support. Need to think about how 10am processing adjusts.
  • Precursor processing (AP team):
    • CalibrateImageTask (John Parejko ):
      • prompted by DMTN-172 plans for refactoring calibration
      • AP & DRP share first stage, so John just wrote the replacement for characterize and calibrate–also obviates a number of tickets to clean up wasteful processing in predecessor tasks
      • calibrateImage now implemented and running in AP, not yet in nightly CI
      • dataset type changes include: q
        • initial_pvi instead of calexp
        • initial_stars_footprints instead of src catalog
      • previously took 40-45 seconds; now takes 20-25 seconds (laptop testing)
      • still some configuration tweaks and A/B comparisons needed
      • anticipate future RFCs to make the default for AP processing; then will hand off to DRP from there
      • MWV: what are the implications for early in commissioning, when we might not have all of our calibration products?  JP: input is "the monster": so precursor surveys are "enough".  Michael: what about debugging when a match fails?  you lose the intermediates.  JP: some intermediates are persisted, and control flow is simpler to follow.  MWV: what about out of focus stuff?  KB: AOS group should be able to get us close, within a few nights
    • run comparison tool (Kenneth Herner ):
      • compare_sources in analysis_ap
      • give it two butlers and two APDBs
      • returns dataframes and optionally cutouts for AND and XOR of DIASources in the two runs
      • can reuse cutouts if they already exists
        • John: just merged a fix that makes cutout generation much faster
      • AP-specific for now, but could be extended to work with DRP products
      • merge to main coming later this week
  • SITCOM team updates (Andrew Connolly, Mario Juric , Bryce Kalmbach , Krzysztof Suberlak , Neven Caplar  ):
    • Bryce Kalmbach : transitioning from phosim to imsim, now getting closed loop to converge.  in ts_imsim.  next to look at deblending and rotation
    • Krzysztof Suberlak : wrote a technote comparing different approaches for (Eric missed this)
    • will make sure new CalibrateImageTask doesn't cause breakage
    • Keith Bechtol : has been going through system-level performance requirements, will start meeting with individual science units soon
  • DIA Transients & Variables Science Unit update (Eric Bellm ; #rubinobs-sciunit-transient-variable):
  • DIA Solar System Science Unit update (Mario Juric ; #sitcom-solsys):
    • Mario is out
    • ongoing discussion in AP about trailed source representation
  • PreOps V&V (Colin Slater ): 
    • Colin is at SCOC
  • Review CI (https://usdf-rsp-dev.slac.stanford.edu/chronograf/):
    • More metrics for timing have been added
  • Next meetings (all at 2pm Pacific):
    • Oct. 2
    • Nov. 6
  • AOB


Action Items

DescriptionDue dateAssigneeTask appears on
  • Eric Bellm write up retention policy and sizing for APDB  
17 Jul 2023Eric BellmAP Pipeline Meeting, 2023-06-26
  • Eric Bellm dig into alert serialization test in ap_association and ticket ways to improve it  
11 Sep 2023Eric BellmAP Pipeline Meeting, 2023-08-21
  • Eric Bellm review broker communication channels  
25 Sep 2023Eric BellmAP Pipeline Meeting, 2023-08-14