AP Pipeline Meeting, 2023-01-30

2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

• Eric Bellm 
• Ian Sullivan 
• Ari Heinze 
• Krzysztof Findeisen 
• Brianna Smart 
• Erin Howard 
• Kenneth Herner 
• John Parejko 
• Nima Sedaghat 

regrets

Topics for discussion:

• Project updates (Eric Bellm , Ian Sullivan ):
  • JP: if you are traveling with a partner to JTM you can potentially purchase tickets yourself with permission
• Pipelines Infrastructure (Krzysztof Findeisen , John Parejko ): 
  • KF:
    • still working on getting Prompt Processing back up and running–COSMOS dataset didn't have enough template coverage for all the raws
    • updating all of the ap_verify datasets to use the latest repo format–pull if you have local copy
• Precursor processing ( Kenneth Herner , Erin Howard ):
  • Erin: still fighting Saha Bulge templates–output log file is 1 GB and has been running for 5 days
    • Ken wonders if there are jobs that are over the memory requirements, which would hold those jobs and any that depend on them.
  • Ken: ticket for HSM shape changes is now ready for review in ip_diffim
    • one question: slot_shape flag is the general one, SDSSShape and HSMShape have sub-flags.  sounds like we'll just propagate them up.
• Image differencing algorithms, DCR  ( Ian Sullivan , Bruno Sanchez ):
  • no updates.  Bruno hopefully joining ~next week
• transinet (Nima Sedaghat, Eric Bellm):
  • no update; Ian still owes a code review
• alert distribution (Brianna Smart , Eric Bellm ):
  • alert brokers running successfully for a week now. 
  • A few weird things–e.g., the archiver writer has died.  One broker shows access denied, not clear yet what is causing the error
  • Bri happy to invite the brokers to connect to the IDF ~tomorrow once she checks a couple things
  • Eric & Leanne talking about how to commission the brokers–various political minefields.  Sending DC2 alerts may be the most straightforward
• Solar-system processing (Ari Heinze ):
  • heliolincc can already digest detections from all of PS1, ATLAS, Catalina, & ZTF–could postprocess and discover new things
  • meeting with Mario tomorrow to talk pythonization of heliolincc
• Review CI (https://chronograf-demo.lsst.codes/):
  • DM-32756 was a DRP template bugfix, removed some unassociated DIAObjects from both HSC and DECam (good), slight improvement in fake completeness (good), but adds runtime (bad).  Ian Sullivan to ticket future performance optimization of this step.
  • one of the two HSC images takes twice as long to execute subtractImages as the other--why?
• Review outstanding action items
• QA meeting in February (Ian will lead):
  • HSM shape flag – Ken
  • HSC with convolution mode=auto  – Erin
    
    
  • potential future:
  • edge flags?
  • preconvolution
  • RB scoring?  still too early, want to show it one of the large runs
  • Discussion of AP QA needs in the future that Commissioning/LINCC might assist with
  • Eric on detection thresholds/fakes forced photometry? (still no, Eric expects to miss Feb. meeting)
• AOB
  • JP: created a Slack room related to travel for JTM #chile-jtm-planning

Action Items