AP Pipeline Meeting, 2024-03-11

2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

• Eric Bellm
• Ian Sullivan 
• John Parejko 
• Ari Heinze 
• Krzysztof Findeisen 
• Brianna Smart 
• Erin Howard 
• Harshit Rai 
• Hsin-Fang Chiang 
• Meredith Rawls 
• Nima Sedaghat 
• Harshit Rai 
• Bruno Sanchez 

regrets

Topics for discussion:

• Project updates (Eric Bellm , Ian Sullivan ):
  • ops red team review this week
  • mirror is in the observatory! see #follow-comms-team
• Auxtel processing status (all):
  • EFD syncing is fixed, so we are running PP as usual
  • now on new weekly, so now running CalibrateImageTask.
    •  a couple of failures cropped up, discussed below
  • John Parejko deeper questions on DM-43247
    • no clean stars available for PSF determination
    • pixel scale on LATISS means that footprints grow a fair bit, creates lots of blends
    • what parameters should be loosened?  some discussion of the grow radius
      • grow radius defined in pixel space, may not be appropriate for LATISS and/or the specific task of finding PSF stars
      • Eric: wonder what our footprint radial profiles would like, suspect we're way down in the background oise
      • some sticky tuning questions
• Ops Rehearsal (all):
  • Pipelines for Ops Rehearsal #3
  • discussing some concerns about scope creep on metrics
    
    • still don't have auth for production sasquatch; we could use dev, or just push metrics to the central repo
• Priorities from now-ComCam (Eric Bellm ):
  • AP Ready for ComCam / AP Ready for LSSTCam
• Pipelines Infrastructure (John Parejko, Krzysztof Findeisen ):
  • see above
• Precursor processing & Campaign Management ( Kenneth Herner , Erin Howard):
  • Erin: CM wrapped up the sprint on ComCamSim
  • Erin: need to connect with Nima on precursor processing to get the model naming issue
• Satellites (Meredith Rawls ):
  • paper out on arxiv on glint detection with Tony
• Image differencing algorithms, DCR  ( Bruno Sanchez , Ian Sullivan ):
  • Ian: lots of tickets coming in. 
    • merged ticket renaming some diffim mask planes so we can distinguish science and template mask planes, which will allow us to remove science SATURATED without the template messing things up.  May need to add some pixel flags later.  John: that's easy
      • still need to make the APDB schema changes to break out flag columns
    • deblending of diffim is close to merging
  • Bruno: processing some of the ops rehearsal dataset, the subset with the variable sources and the truth table
• alert distribution (Brianna Smart , Eric Bellm ): 
  
  • Bri:
    • attempting to do a partial bps reprocessing run with alerts turned on, but right now can't start because it's unhappy with rbclassifier
    • once that runs, will talk to Krzysztof about adding credentials and configs to Prompt Processing
• ML (Nima Sedaghat Harshit Rai , Eric Bellm ):
  • Nima:
    • working with kubernetes on GPU tests
    • with Harshit, close to getting set of fields ready from Open Supernova catalog to make truth table
• Solar-system processing (Ari Heinze, Kurlander Jake , Mario Juric ):
  • Ari still trying to nail down Mario to work on pipelinetasks.  otherwise migrating python wrappers to latest C++.
• Review CI (https://usdf-rsp-dev.slac.stanford.edu/chronograf/):
  • reviewing changes due to switch to CalibrateImageTask–nothing alarming
• Review outstanding action items
• QA meeting (May):
  • Ops rehearsal debrief
• AOB
  • Meredith: https://lsstc.slack.com/archives/C2JPMCF5X/p1709770166968539
    • some thoughts from discussion:
      • we're not actually in a pure gaussian field
      • how many of these are dipoles?
      • RHL may have misinterpreted the plot??–the line is not a flux limit

Action Items