AP Pipeline QA Meeting, 2023-10-02

2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

• Eric Bellm
• John Parejko 
• Ian Sullivan 
• Hsin-Fang Chiang 
• Keith Bechtol 
• Krzysztof Suberlak 
• Bryce Kalmbach 
• Bruno Sanchez 
• Erin Howard 
• Brianna Smart 
• Kurlander Jake 
• Kenneth Herner 
• Meredith Rawls 
• Harshit Rai 
• Colin Slater 
• Keith Bechtol 
• Michael Wood-Vasey 
• Nima Sedaghat 

regrets:

• Neven Caplar 
• Mario Juric 

Topics for discussion:

• Announcements
  • DM All-Hands on Friday
• AuxTel Processing (all):
  • Prompt Processing with AuxTel status (Hsin-Fang Chiang , Eric Bellm , Neven Caplar ):
    • run last week, imaging survey took data Weds/Thurs., but PP had bugs both nights: multithreading issue Weds. night, still awaiting a fix; Thursday a bug relating to new reference catalog. 
  • 10am processing status (Colin Slater , Meredith Rawls ):
    • Colin: not a ton of new imaging data gathered, so we haven't been focused on processing it. 
    • potential move to more auxtel nights per month will be an opportunity to reconsider cadence of right timescale for DRP-like AuxTel imaging
      • Keith: thinking about an ops rehearsal around AuxTel, so this would be relevant to discuss.
  • Keith: last month French team put new hardware (collimator/mask) in that better suppresses stellar light in spectra.    Opens the possibility to do both spectroscopy and imaging in high stellar-density fields.  Would then allow comparison of atmospheric params from fgcm to spectroscopy.  Proposal would be to have pairs of separated fields, each of which would get imaging and spectroscopy through the night
    • Eric: more regular imaging sounds good for PP!
    • Colin: sounds like a way to spend a lot of time slewing?  Keith: still early in the planning process. Please feel free to join 2023-10-03 Rubin Data Reduction and Planning Meeting for further discussion and/or see #rubinobs-sitcom-surveys.
• Precursor processing (AP team):
  • analysis_tools metrics afterburner sprint happening this week 
  • Switch to preconvolution (Ian Sullivan ):
    • DM-39976 filters out SAT/BAD/EDGE in preconvolution.  tests on several cases when science seeing is better than template
      • decrease in raw number of DIAObjects, but number of unflagged DIASources is roughly constant
      • looking at counts of sources suggests that it hasn't harmed the case when template seeing is better
      • attempting some coarse cuts on ML reliability score
      • larger-scale tests planned
      • Eric: do you see this as this default for all AP?  What other tests would you want to see first?
        • Ian: preconvolution is ready; there are some things to run down with comparison to auto-mode.  Thinks consistent measurement in preconvolution case is an advantage over auto-mode (since measurements get made on different difference.  (Auto mode is more competitive than he expected, and it runs faster)
        • Bruno: doing some tests with the middle-third seeing templates, looks better than expected.  Ian: in some cases these still have better seeing than the science, not as much of
      • MWV: What further work, if any, is necessary to improve the measurement tasks to correctly return correctly calibrated flux measurements from the DIA subtraction?
        • Eric: we get calibrated fluxes on DIA already, but the measurements won't be good if there are eg deconvolution artifacts.  We haven't yet done enough analysis of the photometric accuracy; Bruno has started on some flux pulls
        • Keith: where are the requirements?  Eric: OSS.  which SU are they assigned to?  Should they be in the DIA SU?  Keith: to discuss.
  • streak-masking in AP single-frame (Meredith Rawls , Brianna Smart ):
    • Bri: we've now added filters for trailed sources: trailed sources that touch the edge of the chip, or trailed sources longer than the provided threshold, are now rejected.  Meredith: tasks
    • Meredith: DM-32769: streak rejection now in both Characterize/Calibrate pipeline and new Calibrate.  Testing on 5 HSC images that are known to have streaks. 
      • just running Clare's maskStreaks task which only previously ran in coadd/warping pipeline (runs on detection bits, not image pixels).  Some cleverness of edge handling there may need to be handled
      • Colin: there's some weirdness in parallel to the streak?  MR: thinks its "real" in the image but it needs investigation.  Colin: where is background subtraction
      • Eric: correct that  we are not yet rejecting detections on new STREAK mask plane?  MR: correct, have to understand
• SITCOM team updates (Andrew Connolly, Mario Juric , Bryce Kalmbach , Krzysztof Suberlak , Neven Caplar  ):
  • Bryce: imsim migration nearly complete, closed loop converging, debugging coordinate systems with masking tape.
    • question for Ian about PP on wavefront sensor processing?  Ian: had conversations with Tiago at PCW.  Sounds like it should be straightforward, but trying to understand requirements in case there is something we're missing.
• DIA Transients & Variables Science Unit update (Eric Bellm ; #rubinobs-sciunit-transient-variable):
  • Processing of DECam data at NERSC – Bob Armstrong is making good progress on DECam DECAT-DDF COSMOS field reprocessing.
  • Discussion of verification planning at meeting tomorrow
  • Things are going well.  Shu and I spent some quality time with Ian Sullivan going through ip_diffim code to better understand where to try out new algorithms.
• DIA Solar System Science Unit update (Mario Juric ; #sitcom-solsys):

  • We're hopefully a day away from establishing a replica of the MPC database at USDF.

  • Jake has started writing the ephemerides cache tool for nightly precomputation of asteroid positions so we can quickly associate them in PP.

  • Ari is continuing to swat down corner cases in Heliolinc performance.

  •  Talked to Robert about AuxTel runs, think we'll request some observations near the ecliptic to ensure we have asteroids to associates. Contingent on finishing the association code.

• PreOps V&V (Colin Slater ): 
  • said above
• Review CI (https://usdf-rsp-dev.slac.stanford.edu/chronograf/):
  • DM-39796 (config updates) changed the metrics
  • Ian: roughly 50 sources changing. about half of these are junk detections going in or out; perhaps not a big concern.  But other half were sources that moved by up to half an arcsecond!  Not always clear which version was correct.  But biggest impact was on source location.  Not yet clear the cause, we'll have to track it down.
  • John: take a look the the config difference on the ticket–we should consider if we trust the old version, so we might be improving DECam in this case to be closer to HSC.  Ian: at least a few cases where the previous measurement was better

• Next meetings (all at 2pm Pacific):
  
  • Nov. 6
  • Dec. 4
• AOB

Action Items