AP Pipeline Meeting, 2023-07-31

2pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

• Ian Sullivan 
• Krzysztof Findeisen 
• Nima Sedaghat 
• John Parejko 
• Ari Heinze 
• Brianna Smart 
• Bruno Sanchez 
• Meredith Rawls 

regrets

• Eric Bellm (vacation)
• Erin Howard (in Chile!)

Topics for discussion:

• Project updates (Eric Bellm , Ian Sullivan ):
  • No updates
• Auxtel processing status (all):
  • Another Auxtel run this week, running ApPipe again
• Pipelines Infrastructure (Krzysztof Findeisen , John Parejko ):
  • JP: Merged CalibrateImageTask, running final tests
    • New pipeline should run in parallel to old ap_verify (but not with Fakes)
    • Old Fake source injection pipeline not compatible with new CalibrateImageTask.
    • BOS: New source injection package appears feature complete, but will require major re-write of Fakes pipeline
• Precursor processing & Campaign Management ( Kenneth Herner , Erin Howard, Meredith Rawls ):
  • MR: Looking at Auxtel runs from DRP processing. Will consult with Colin about investigating Prompt Processing images.
• Satellites (Meredith Rawls ):
  • Noirlab is hiring a software engineer (Michelle Dadighat) to work on Sathub etc..
  • Started looking at adding streak detection to AP.
    • Aware of simpler streak detection from Tony Tyson's group. Might be faster?
  • Giving a talk on satellites/Rubin at PCW, would welcome questions to guide overview talk.
• Image differencing algorithms, DCR  ( Bruno Sanchez , Ian Sullivan ):
  • BOS: Masking is now avoiding Fakes being used for PSF-matching. Looking at flags
  • Attempting to process dataset with sufficient Fakes for good statistics
• alert distribution (Brianna Smart , Eric Bellm ):
  • BS: Dan S has a functioning alert broker that is separated from the Science Platform, may be running by end of week
    • IDF version of the alert broker has started exhibiting memory issues again, despite no changes
• transinet (Nima Sedaghat Harshit Rai , Eric Bellm ):
  • Pushing on Zooniverse to automate, and produce visual guide for volunteers.
  • Looking into correctness of ground-truth data for DC2, Bruno is assisting
• Solar-system processing (Ari Heinze ):
  • Press release today: https://www.washington.edu/news/2023/07/31/heliolinc3d/ 
  • Discussion of new columns needed in diaSource tables ( DM-36625 - Getting issue details... STATUS )
    • From the ticket:
      • When new, more advanced versions of heliolinc are written than make use of trailed source information, I anticipate that the following additional values will be needed:
        • Trail length and its uncertainty (in arcsec)
        • Celestial position angle of trail, and its uncertainty (in RA and Dec)
          • DM-15180 to get RA/Dec errors in output catalogs
        • Cross-trail and along-trail astrometric uncertainties
      • AH: notes that the CD matrix is in the source table, but not in the diaSource table. Would be very useful to have the CD matrix in the diaSource table
      • KF: Do we want any pixel units in the schema, or all in sky coordinates?
• Review CI (https://usdf-rsp-dev.slac.stanford.edu/chronograf/):
  • Drop in SubtractTask timing around 7/25, cause unknown
• Review outstanding action items
• QA meeting September 11:
  • Eric on detection thresholds/fakes forced photometry?
  • streak masking/long trail rejection (Bri & Meredith)
  • CalibrateImagesTask in AP (John)
• AOB
  • No meeting next week, for PCW

Action Items