AP Pipeline QA Meeting, 2021-05-03

4pm Pacific Time

BlueJeans: https://bluejeans.com/426716450

attending:

• Ian Sullivan
• Leanne Guy
• Yusra AlSayyad
• Unknown User (cmorrison)
• John Parejko
• Andrew Connolly
• Colin Slater
• Zeljko Ivezic
• Bryce Kalmbach
• Kenneth Herner
• Meredith Rawls
• Unknown User (gkovacs)
• Krzysztof Suberlak
• Eric Bellm

regrets:

Topics for discussion:

• Precursor processing (Meredith Rawls ):
  • DM-29422: gen3+bps HSC Cosmos run
    • sample of random 50 diasource cutouts
    • Zeljko points out that many of the HiTS sample alerts had single pixels with very negative values–we seem to see similar effects here
  • sky plot with 3+ good DIASources looks pretty good–only a little bright star and edge structure.  With all good DIASources it's a bit worse
  • using a larger list of bad flags–but still majority of the
  • plots per visit still showing some per-CCD and edge troubles
  • seems like next step is starting to explore where the weird effects appear in spatial positions/flags/look at cutouts, etc.
  • equivalent HiTS processing awaiting port into shared gen3 repo
• SITCOM team updates (Andrew Connolly  et al.):
  • Krzysztof Suberlak shared work on ComCam simulations.
    • Simulations of donuts (phosim) using ComCam and LSSTCam, processed using stack Gen 3 pipeline
    • Also processing the equivalent in-focus images
    • Note that crosstalk is not yet turned on
    • Do not need special camera mappers anymore
• PreOps V&V (Colin Slater ): 
  • nothing today
• Review CI (https://chronograf-demo.lsst.codes/):
  • Observed difference in diaSource to direct source ratio is probably due to diffim variance scaling in DECam, not clear where the difference comes from in HSC
    • Likely somewhere in single frame measurement
    • For HSC Gen 3, there are differences in the number of sources detected on the calexp and the calexps themselves have differences at the pixel level
    • Yusra: in DRP the Gen 2 and Gen 3 calexps for HSC are now bitwise identical, the fix was changing how the calibs are read in. Lauren MacArthur would be able to explain more. 
      • Lauren here:
        • anyone wanting to follow along/catch up on gen2 vs. gen3 up to SFM is welcome have a look at  DM-28858 - Getting issue details... STATUS ,  DM-28985 - Getting issue details... STATUS   DM-28936 - Getting issue details... STATUS ,  DM-29881 - Getting issue details... STATUS , and  DM-29819 - Getting issue details... STATUS  (and  DM-29820 - Getting issue details... STATUS  &  DM-29821 - Getting issue details... STATUS  for the start of the fgcm and jointcal comparisons, respectively).  All are listed under the drp-parity Jira label.
        • As for the issue here, at least one cause (and fingers crossed the only) is that gen2 ap_verify has doBrighterFatter=False, whereas it is True in the gen3 version.  See updates on  DM-29344 - Getting issue details... STATUS )
        • Feel free to ping me if further details are desired.
    • There's also a bump in the ratio of DIASources to science sources on 4/25-4/27 in both cameras in both gens that we don't have annotated...
• Next meetings (all at 4pm Pacific):
  
  • first Mondays of the month: June 7 is our next meeting
• AOB
  • none today

Action Items