• When: 1pm PT 50min 
• Where: zoom: https://princeton.zoom.us/my/yusra

Attending

Operations Data Production Algorithms and Pipelines folks

• • • Yusra AlSayyad
    • Nate Lust 
    • Dan Taranu 
    • Eli Rykoff
    • Hsin-Fang Chiang 
    • Clare Saunders
    • Lee Kelvin  
    • Ian Sullivan 
    • John Parejko 
    • Fred Moolekamp
    • Jim Bosch
    • Brianna Smart
    • Joshua Meyers
    • Merlin Fisher-Levine
    • Nima Sedaghat 
    • Erfan Nourbakhsh
    • Pierre-François Leget 
    •  Agnès Ferté (10min late)

Apologies

• Eric Bellm (Rubin ToO workshop)
• Arun Kannawadi 
• Christopher Waters
•  Andrés Alejandro Plazas Malag

Agenda

• • Announcements:
    • Think whether there are any non-quality metrics we you would feel proud of during steady state operations: https://rtn-077.lsst.io
  • How did last time go?
    • Other than the ComCamSim Pipeline sprint what got done?
      • DM-41929 - Getting issue details... STATUS
      • Made some backports
      • DM-42088 - Getting issue details... STATUS
      • DM-42774 - Getting issue details... STATUS
      • DM-41108 - Getting issue details... STATUS Added the analysis_tools afterburner for AP, which is now run in CI
      • LATISS Prompt Processing has updated to the default ApPipe of the w_2024_10 stack on Mar 7th, so it has started to use calibrateImage task.  
      • Prompt Processing now tests ComCamSim processing in dev.   Waiting for Scheduler and what nextVisit messages we can expect. 
      • Not done yet but in review--DM-43083 is adding the option to run maskStreaksTask to detectAndMeasure, so it partially addresses DM-42087 below. A separate ticket will turn of streak masking in characterizeImage and turn it on in detectAndMeasure.
      • Lots of StarTracker stuff inc a nice new TMA pointing model
      • Standing up Rapid Analysis at USDF, hopefully in time for OR3.
        • Many many things happening in parallel, hopefully they all converge just before OR3:
          
          • Moving to RubinTV v2
          • Moving to redis for work distribution
          • Adding ComCamSim to RubinTV and making workers for new instrument
          • Standing up RA at USDF and processing ComCamSim data there
          • Making RA run real pipelines the same way pipetask run  does.
      • Finished DM-41764 - Getting issue details... STATUS . Ready for review.
      • DM-42157 - Getting issue details... STATUS Fixed umpteen multiprofit bugs; much closer to filing RFC & getting into lsst_distrib & test-med-1 reprocessing at least.
      • DM-43176 : Added a check in meas_extensions_scarlet to flag and skip sources that cannot be loaded in a meas or forced_src catalog due to having no flux after re-distribution. This only happens when something else has gone wrong, like PSF modeling, but is worth catching to prevent a patch processing from grinding to a halt.
      • DM-43238 - Getting issue details... STATUS
      • DM-43187 - Getting issue details... STATUS .  Is this the reason for the wperp oddities?  I will investigate!
      • DM-43332 - Getting issue details... STATUS is in reviewable state; substantial speedup in a (currently DC2-only) task that didn't take very long so nobody is likely to notice.
      • DM-43247 - Getting issue details... STATUS

• • • What REMAINING emergent issues did we fail to fix or questions we didn't answer?
      • There's already a "refcats" run collection The butler on the summit needs some surgery, but it doesn't have to be a Jim Bosch level surgeon: DM-42295 - Getting issue details... STATUS  
      • YA owes Fred a resource usage analysis on DM-41790 - Getting issue details... STATUS
      • DM-42087 - Getting issue details... STATUS
      • DM-42606 - Getting issue details... STATUS
      • DM-43160 - Getting issue details... STATUS
      • DM-42920 - Getting issue details... STATUS I misrepresented this at the last meeting, apCorr already HAS a 10 minute heartbeat. It just prints out nonsense numbers.
      • DM-43256 - Getting issue details... STATUS This has been reviewed and is almost ready to merge, and then we can start having daily / monthly metrics
      • DM-43357 - Getting issue details... STATUS multiprofit PSF fitting seems much slower than meas_modelfit; still investigating
      • New set of errors from calibrateImage.  Maybe no further actions are needed? DM-43248 - Getting issue details... STATUS DM-43306 - Getting issue details... STATUS
    • What NEW emergent issues came up?
      
      • We've gotten some error reports from NAOJ from the ongoing HSC PDR4 processing, but they look like issues we've already fixed and backported at first glance.  Trying to confirm the version they're using now.
      • DM-43366 - Getting issue details... STATUS
      DM-43179 - Getting issue details... STATUS
      Need to check that the psf matching kernel correctly uses spatial variation

• Decisions for the ComCamSim Pipelines in advance of March 20:
  • Are the 3 days of 60 visits each region for templates OK?
  • Which skyMap. We need to remember to reduce the required number of dynamic detection sky objects to 10. 
  • Question: when will the "final" ops-rehearsal data be decided and transferred to /repo/embargo and collections defined there?  Answer: around Mar 27. 
• Next meeting: April 1 (back on same phase as before)