AP Pipeline Meeting, 2023-12-18

2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

• Eric Bellm
• Brianna Smart 
• Erin Howard 
• Ari Heinze 
• Krzysztof Findeisen 
• John Parejko 
• Nima Sedaghat 
• Hsin-Fang Chiang 
• Bruno Sanchez 

regrets

• Ian Sullivan (vacation)

Topics for discussion:

• Project updates (Eric Bellm , Ian Sullivan ):
  
  • Upcoming meeting free weeks: weeks of December 25 and January 1.  Next meeting Jan. 8
  • new JTM block
• Auxtel processing status (all ):
  • a little weirdness with Friday's night's processing due to USDF butler overload
    • there was a butler bug that led to many more transactions, which is now fixed.
    • would be helpful to have more isolation of /repo/embargo from /repo/main
  • Auxtel will run until the end of the week and then shut down
• Pipelines Infrastructure (John Parejko, Krzysztof Findeisen ):
  • no reports.
• Precursor processing & Campaign Management ( Kenneth Herner , Erin Howard):
  • Erin: working w/ Orion on pretty print on pipetask reports
• Satellites (Meredith Rawls ):
  • met with Claire last week at pair coding to explore why streak masking is over-enthusiastic.
• Image differencing algorithms, DCR  ( Bruno Sanchez , Ian Sullivan ):
  • Bruno: ill last week, but attended the LSST France meeting remotely.  working on a review for Ian
  • Ian & Eric figured out why where were getting ~0% fakes completeness with preconvolution: the FAKES mask plane was being interpolated over when constructing the score image.  Fixed on DM-42121 - Getting issue details... STATUS
• alert distribution (Brianna Smart , Eric Bellm ): 
  
  • Bri:
    • was at conference with brokers last week.  Brokers eager to get going on testing in the new year
    • Was interested in a talk about Alerce using Fourier Transforms to find glints in ZTF postage stamps, have messaged for more info
  • Eric:
    • will chat with Bri about schema update ticket
• ML (Nima Sedaghat Harshit Rai , Eric Bellm ):
  • Nima: model packages butler mode is done, now trying to understanding how to make use in ap_verify
• Solar-system processing (Ari Heinze, Kurlander Jake , Mario Juric ):
  • Ari: sprinted with Ian, Eric, Mario, Jake on getting pipe tasks working last week
    • have APDB query to generate outputs for input to butler
    • got makeTracklets working and bringing data in and out of the butler
    • will resume downstreak SSP tasks on return from vacation
  • had a scare about singletons resulting from link purification code, but bug was in the downstream analysis code rather than algorithmic code
  • adding some prescreening for planarity suggested by Jake to reduce needed orbit fits
• Review CI (https://usdf-rsp-dev.slac.stanford.edu/chronograf/):
  • nothing new
• Review outstanding action items
• QA meeting February 5 (TBC):
  • streak masking (Meredith)
• AOB
  • Meredith: talking about SFM metrics with Alex DW, couple of threads
    
    • JP: astrometry metric is just mean scatter on refcat fit, which is not the most useful–would want to use holdout sources to assess
    • JP: thinks those numbers are a bit of a grab bag

Action Items