AP Pipeline Meeting, 2023-02-13

2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

• Eric Bellm 
• Ian Sullivan 
• Ari Heinze 
• Erin Howard 
• John Parejko 
• Brianna Smart 
• Nima Sedaghat 
• Krzysztof Findeisen 
• Bruno Sanchez 
• Kenneth Herner 

regrets

Topics for discussion:

• Project updates (Eric Bellm , Ian Sullivan ):
• Pipelines Infrastructure (Krzysztof Findeisen , John Parejko ): 
  • Krzysztof: nothing really to report, working on PP
  • John: looking into lmfit failures: dipole fitting failing with higher versions. 
    • nothing in the changelog to suggest a big change
    • replacing with scipy.least_squares would be difficult because it uses a lot of lmfit's constraints
• Precursor processing ( Kenneth Herner , Erin Howard ):
  • Erin:
    • reprocessing of HSC and DECam are going well.  going to compare to October because of issues with the November run
    • Saha Bulge still causing trouble.  monitoring causing some of the failures, as well as all of the debug output–pair coded with Michelle Gower on this.  Going to try removing the monitoring to see if the processing will succeed.
      • loglevel info is set everywhere we can, but monitoring injects debug somewhere, need to root that out
    • found and fixed a template bug with Ken.  how often to remake templates?  Eric says ideally we'd do it each month
    • side discussion of how we will be generating templates with our own template generation code vs. based on DRP to take advantage of better full-focal plane astrometry, background subtraction, etc.
      • Eric: in incremental template generation period we won't necessarily have a fully DRP
      • we should plan to try making template from HSC RC3 for cosmos to check difference
  • Ken: pair coded on crossmatching and fixed problem, will revise plot
• Image differencing algorithms, DCR  ( Ian Sullivan , Bruno Sanchez ):
  • Ian: preconvolution merged last week, so have scheduled some runs for Erin to test
    • need to improve handling of edge sources for convolution
  • Bruno: still getting onboarded at USDF
• transinet (Nima Sedaghat, Eric Bellm):
  • Nima: spent some time with Ian on technical discussions and now implementing model packages
  • planning to bring end-to-end transinet into the same repository
• alert distribution (Brianna Smart , Eric Bellm ):
  • Bri: no changes, no contact from the broker teams–probably they haven't tried to connect
    • strimzi has been going in and out due to square changes
  • Eric notified the broker teams of the IDF testbed.  Will write a technote on broker prep.
• Solar-system processing (Ari Heinze ):
  • doing experiments on ATLAS to probe how to optimally sample the hypotheses
  • has a vision for the pipeline wrapping, haven' started  implementing yet.
• Review CI (https://chronograf-demo.lsst.codes/):
  • several changes in both object counts and timing for HSC and DECam but not DC2 on 2/8 and 2/9
  • object counts increased due to wrapping the template PSF bugfix in try/except (DM-37884)–this isn't expected
  • maybe preconvolution? DM-34909  not clear why this would cause a change though
  • runtime decreased as expected for DM-37884
  • (looks like changelog and chronograph are in UTC)
• Review outstanding action items
• QA meeting in March:
  • edge flags?
  • preconvolution
  • RB scoring?  still too early, want to show it one of the large runs
  • Discussion of AP QA needs in the future that Commissioning/LINCC might assist with
  • Eric on detection thresholds/fakes forced photometry? (still no, Eric expects to miss Feb. meeting)
• AOB
  • Bri needs a volunteer for Science Pipelines team meeting Wednesday

Action Items