AP Pipeline QA Meeting, 2021-06-07

4pm Pacific Time

BlueJeans: https://bluejeans.com/426716450

attending:

• Eric Bellm
• Andrew Connolly
• Unknown User (cmorrison)
• Ian Sullivan
• Jeffrey Carlin
• John Parejko
• Kenneth Herner
• Krzysztof Findeisen
• Yusra AlSayyad
• Zeljko Ivezic
• Meredith Rawls
• Unknown User (gkovacs)
• Leanne Guy
• Bryce Kalmbach
• Krzysztof Suberlak

regrets:

Topics for discussion:

• Precursor processing (AP team):
  • Kenneth Herner with Meredith Rawls and John Parejko DC2 data run through ap_pipe
    • This is a more full-fledged “end to end” AP Pipeline than we’ve done before, since it starts from raws, builds templates, and does differencing etc.!   Also provides a third dataset (after HiTS and COSMOS) that we can use for testing.
    • lots of pipeline infrastructure changes to handle the new dataset
    • DM-30003 captures the run being shown
    • <1% failure rates prior to imageDifference, > 95% success in ImageDifference, small failures thereafter
    • flags dominated by slotShape, which is catching negative sources
    • spatial density plots mostly uniform but a few linear features which need investigation–disappear with more flags or larger number of constituent visits
    • showing a handful of postage stamps–small handful shown look solid
    • still getting lots of technical (re)work as Gen3 matures under us
  • Eric Bellm / Krzysztof Findeisen fakes
    • fakes now running in CI–to discuss during our our look at Chronograph
• SITCOM team updates (Andrew Connolly  et al.):
  • focusing on prep for AOS integration.  Continuing to refine phosim replication of what Rubin will be generating
  • Chris S: Catalogs and images for AOS testing – big
• PreOps V&V (Colin Slater ): 
  • mainly focused on DP0.1 release at the end of June.–DP0.2 plan to come later
• Review CI (https://chronograf-demo.lsst.codes/):
  • fakes now injected in gen3 COSMOS–about 20 per CCD-visit with a flat magnitude distribution
  • need to adjust metrics for non-synthetic sources to avoid bumps in the object counts
  • 75-100% in the 20-22 and 22-24 mag bins–may have some small number statistics at play here, need to investigate a bit further
• Next meetings (all at 4pm Pacific):
  
  • first Mondays of the month: but July 5 is a holiday.  Defer to August 2 (PCW preview)?
    • general agreement to skip July
• AOB
  • none

Action Items