AP Pipeline QA Meeting, 2023-07-10

2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

• Eric Bellm
• Ian Sullivan 
• Erin Howard 
• Bruno Sanchez 
• Krzysztof Findeisen 
• Meredith Rawls 
• John Parejko 
• Colin Slater 
• Kenneth Herner 
• Jeffrey Carlin 
• Keith Bechtol 
• Leanne Guy 

regrets:

• Steve Ritz 
• Andrew Connolly 
• Bryce Kalmbach 
• Krzysztof Suberlak 
• Mario Juric 
• Brianna Smart 
• Nima Sedaghat 

Topics for discussion:

• Announcements
  • schedule workshop last week
• AuxTel Processing (all):
  • Prompt Processing with AuxTel status (Hsin-Fang Chiang , Eric Bellm ):
    • USDF came up in time for only one night of observing during last rung
    • PP service came up without human intervention, but the build was old and so didn't have all the
    • need to have a pre-run checklist
  • 10am processing status (Colin Slater , Meredith Rawls ):
    • Colin: Huan will try to run what data exists in the next few days–but there wasn't much data anyway due to wind closures etc.  Maybe there are 500 exposures–TBD.
  • AuxTel run planned for next week, but weather a bit iffy
• Precursor processing (AP team):
  • fake source injection improvements (Bruno Sanchez ):
    • was comparing default, preconvolution, and auto-mode diffim on 2021 diffim sprint dataset. 
    • very few differences between the techniques in fakes—suggests need a broader range of conditions in the dataset for testing.  (templates are better seeing than ~all of the science)
    • but had trouble looking at fakes completness:  increased the magnitude range of fakes (20-27 (from 26)) and a higher fakeDensity (2k→5k per square degree).  Still TBD if this is enough
    • running detection at lower SNR=3 to enable efficiency estimation.
    • small test over 6 CCDs
    • KF: have these changes been merged?  Bruno: not yet, has a PR, wanted to test once USDF comes up.  Aiming to merge this week.  KF: we'll be sure to flag this in Chronograf since the number of fakes will change.
    • new results seem rather low–some discussion.  May want to perform SNR-based injection with new source_injection framework.  Right now he's estimating the fake SNR.  Eric: there were some concerns we had with matching injected sources in the past, not clear it's totally been run to ground–we should look at spatial plots of hits and misses and see if there are bookkeeping errors
    • Colin: shape of SNR distribution near 5 sigma should be set by Poisson stats, both the distribution and fitting function look weird.  Bruno: bins are binomially distributed, with the parameter set by the SNR in that bin. 
  • off-chip centroids/maxDistToPeak (John Parejko ):
    • when making cutouts long ago, John found that some sources had centroids off the chip (hundreds or thousands of pixels outside of the detection footprint)
    • Bri dug in on DM-38285
    • due to an edge failure case in the centroiding code if there are background subtraction problems, etc.  Centroid alg. can run away.  There is a config parameter to control this: maxDistToPeak, which was defaulted to -1.  After some experiments, settled on a 5 pixel value as a default, which eliminates the catastrophic centroids problem
    • Colin: does it make sense to clip to 5 pixels, or to set it to the peak location?  John: maybe it does that, will check.  Eric: seems like a case for a flag.  John: there is a flag if this triggers. 
    • (update: does use the peak centroid if it hits the limit value, and sets a flag.  We don't think the flag propagates to the APDB, but it does also set the generic SDSS Centroid flag)
    • John: there were some old tickets capturing related behavior
• SITCOM team updates (Andrew Connolly, Mario Juric , Bryce Kalmbach , Krzysztof Suberlak  ):
  • conflicting meeting today: report by email:
     "An update from our side is that we expect to have transitioned from phosim to imsim/galsim by the PCW. Bryce has imsim data running through the pipeline and outputting zernikes and we are now testing that the closed loop converges."
• DIA Transients & Variables Science Unit update (Eric Bellm ; #rubinobs-sciunit-transient-variable):
  • Planning a session at the PCW
  • Keith: we are working on the draft On-Sky Observing plan–comments are welcome.  Eric: the SU did provide updates on the relevant Confluence pages.
• DIA Solar System Science Unit update (Mario Juric ; #sitcom-solsys):
  • Mario is out today
• PreOps V&V (Colin Slater ): 
  • no further updates: anxious for more on-sky data from AuxTel
• Review CI (https://usdf-rsp-dev.slac.stanford.edu/chronograf/):
  • possible ~10% increase in subtractImages runtime during the shutdown, but a bit hard to tell from the sampling.  No obvious algorithmic or config drivers–we will check and see if this continues during the next week.
  • (Jenkins hardware is non-USDF so it's not affected but the shutdown)

• Next meetings (all at 2pm Pacific):
  
  • Cancel August 7 (PCW)
  • September 4 → September 11
• AOB
  • none

Action Items