AP Pipeline Meeting, 2022-10-10

2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

• Eric Bellm 
• Ian Sullivan 
• John Parejko 
• Erin Howard 
• Meredith Rawls 
• Kenneth Herner 
• Krzysztof Findeisen 
• Brianna Smart 

regrets

Topics for discussion:

• Project updates (Eric Bellm , Ian Sullivan ):

  • 
    

• USDF transition status:
  • Old DC2 runs chained from 2021 runs, so those are not available or useable yet. 
  • Postgres starting to be useable
  •   DM-36497 - Getting issue details... STATUS and DM-36505 - Getting issue details... STATUS
• Pipelines Infrastructure (Krzysztof Findeisen , John Parejko , Brianna Smart ): 
  • discussion of DM-27858 - Getting issue details... STATUS and RFC-697 - Getting issue details... STATUS
  • current AP defaults should fall back to PS1?
  • discuss DM-33039 - Getting issue details... STATUS (regenerating CI datasets)
• Precursor processing (Meredith Rawls, Kenneth Herner , Erin Howard ):
  • Erin Howard : on Friday our "full" ap_pipe with fakes ran to completion with bps/parsl and into postgres!
    • Eric Bellm was able to pull out trailed source lengths from postgres for a presentation
• Image differencing algorithms, DCR  ( Ian Sullivan ):
  • made some progress on template variance bug with Yusra and Jim, Yusra got excited and took it over
  • found a good resolution but will take some new options to add to afw.math to average
  • Ian has been taking
• transinet (Nima Sedaghat, Eric Bellm, John Parejko ):
  • Nima
• alert distribution (Eric Bellm , Brianna Smart ):
  • chatting later today
• Review CI (https://chronograf-demo.lsst.codes/):
  •  one weird blip in one chip of DECam Fraction of DIAObjects updated.  Ian thinks maybe a ticket that Nate had?  let's keep an eye on it
  • we should drop image different runtime plot since it no longer has any new data
• Review outstanding action items
• QA meeting in November:
  • show results from HiTS/COSMOS reprocessing
  • Eric on detection thresholds/fakes forced photometry?
  • Ian hopes to show some results from convolve science
• AOB
  • Meredith: two fun facts from ops-usdf: dc2 collection not easy to recover
  • questions about bps processing blew up into parsl vs. panda–we don't know what to do
    • Eric says the DMLT is now going to discuss it...  in the meantime if parsl is working for us let's hang with it.

Action Items