AP Pipeline Meeting, 2023-01-23

2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

• Eric Bellm 
• Ian Sullivan 
• Krzysztof Findeisen 
• Brianna Smart 
• Nima Sedaghat 
• Ari Heinze 
• Erin Howard 
• Kenneth Herner 
• John Parejko 

regrets

Topics for discussion:

• Project updates (Eric Bellm , Ian Sullivan ):
  • Chile JTM: get your TRs in to Nikolina
• Pipelines Infrastructure (Krzysztof Findeisen , John Parejko ): 
  • Krzysztof:
    • prompt prototype work held up by growing pains in the central repo. 
    • otherwise, fixing a bug in our metrics: our "wall clock time" metrics are actually measuring CPU time, which can be a factor of a few less than CPU time!
      • profiling sprint mostly didn't use the metrics, although the timing of the association metrics did
  • Eric talked some to Colin about AP analysis tooling:
    • afterburner on PP for rapid visit-level metrics
    • analysis_ap type query interface to PPDB for history/query analysis
    • maybe a small flask viz on alert packets
    • JP: talked to Andy about LINCC contributions to visualization: Andy said we should make a concrete proposal
• Precursor processing ( Kenneth Herner , Erin Howard ):
  • Erin sick, but thinks they have a lead on where the Saha Bulge templates went wrong.
    • HSC with convolution mode=auto went fine?  going to try the same with DECAM
    • had campaign management meeting Thursday
  • Ken showing test of HSM Shape for shape flag–little change to raw detections, as expected.
    • Eric would like to see cutouts with/without new shape flag
  • per Bri's ticket on unflagged CRs:
    • EDGE flag here seems to mean convolution with NO_DATA: Jim Bosch thinks this isn't supposed to be the meaning of EDGE?
      • nobody knows why this is happening like this in the diffim.  JP thinks this is question for the subaru folks, wonders if they're doing something strange in ISR
        • Eric Bellm  will make a small ticket to chase down the bookkeeping 30 Jan 2023 
• Image differencing algorithms, DCR  ( Ian Sullivan , Bruno Sanchez ):
  • Ian: option to add preconvolution languishing, but merged a blocking bugfix–should dramatically improve results when convolving science
    • Eric: did Erin's HSC run include the bugfix?  No, HSC run used w_29, just showed up in latest weekly.  We'll discuss next steps in sprint planning
  • Eric: when should we move to mode='auto' by default? 
    • we should look at results of the mode=auto runs Erin is doing first
    • one question of whether to do convolve science or preconvolution in mode=auto.  Needs some more study
    • this is increasing our need to nail down our fake and real catalogs in a useful way
• transinet (Nima Sedaghat, Eric Bellm):
  • Nima: lots of discussion of model storage in DM-34378, working towards a concrete implementation
• alert distribution (Brianna Smart , Eric Bellm ):
  • alert stream broker now working in the IDF!  some mislabeled configurations caused the confusion. 
  • has tested all the user credentials, preparing to allow them to connect
• Solar-system processing (Ari Heinze ):
  • learning about pybind11, getting close to think about python-wrap heliolincc
• Review CI (https://chronograf-demo.lsst.codes/):
  • no changes
• Review outstanding action items
  • none
• QA meeting in February:
  • HSM shape flag?
  • HSC with convolution mode=auto?
  • potential future:
  • edge flags?
  • preconvolution
  • RB scoring?  still too early, want to show it one of the large runs
  • Discussion of AP QA needs in the future that Commissioning/LINCC might assist with
  • Eric on detection thresholds/fakes forced photometry? (still no, Eric expects to miss Feb. meeting)
• AOB

Action Items