AP Pipeline QA Meeting, 2021-12-06

4pm Pacific Time

Zoom: https://washington.zoom.us/j/98846655020

attending:

• Eric Bellm
• Ian Sullivan 
• Unknown User (cmorrison) 
• Colin Slater 
• John Parejko 
• Krzysztof Findeisen 
• Leanne Guy 
• Zeljko Ivezic 
• Kenneth Herner 
• Meredith Rawls 
• Andrew Connolly 
• Jeffrey Carlin 

regrets:

Topics for discussion:

• Precursor processing (AP team):
  • Meredith Rawls gen3 HiTS with fakes (DM-32245)
    
    • multiple tracts, gen3, fake source injection.  First full rerun since January!
    • everything processed
    • chips 3 and 47 show consistently higher number counts
    • we see negative totfluxes, which requires further investigation
      • Eric Bellm ticket investigation of negative totfluxes 13 Dec 2021 
        
    • some improvements seen due to calib imrpovements
    • an unmasked satellite trails
    • fakes!
      • input fakes are all over due to HSC rings skymap (though they are surprisingly structured)
        • these are coming from the matchedDIASource catalogs–Chris thinks these shouldn't be there, since they are trimmed to the bbox of the detector/visit
          • Eric Bellm ticket to understand why matchedDIASources are being stored outside the visit bbox (is this the source of our low completeness?)  see https://github.com/lsst/pipe_tasks/blob/aca5567d74a05fb166ef81240f5dd9cf5a539e7c/python/lsst/pipe/tasks/matchFakes.py#L151 13 Dec 2021 
      • completeness is is about 50-60% at 6-sigma, which is too low, but similar to what we see in HSC–need to sort this out.
        • algorithms workshop performance was better, don't expect we should be doing worse–bookkeeping issue?
        • scatter plot of nondetected sources
      • does diffim propagate the fakes pixel mask plane from calexp/coadd?  if so, we should propagate up to diffim catalog
        • can't use it to select fakes without contamination, but can use it to exclude fakes
          • Eric Bellm ticket to propagate up fake mask plane into diffim catalog 13 Dec 2021 
            
    • some discussion of whether we should switch to a custom skymap here
  • Kenneth Herner Saha Bulge reprocessing PREOPS-597
    
    • ran successfully with multi-tract diffim (u/kherner/PREOPS-597_20211026 in /repo/main)
    • had to disable brighter-fatter.  (Krzysztof Findeisen --this should be turned off by default in obs_decam?  we've never supported this for decam, strange you had to do anything.  was he including ApPipe.yaml from obs_decam?  Ken to try to reproduce.)
    • also adapted DMTN-171 config changes
    • output is ~40 GB on disk–so have to be thoughtful about how to analyze in RSP
    • why are there large missing regions?  Some of this is due to a large number of very bright stars, not clear if it's all of it
      • Eric Bellm ticket investigation of swirly missing DIASources in Saha Bulge 13 Dec 2021 
        
    • 2013 and 2015 have different limiting magnitudes, different distributions of psFlux
      • Eric Bellm ticket investigation of 2013 vs 2015 differences in Saha Bulge (also astrometry, photometry, etc...) 13 Dec 2021 
  • Kenneth Herner preconvolution & convolve science tests
    • aiming to find configuration that lets us assess diffim when the science has better seeing than the template (via carefully selected visits, with worse seeing than best seeing templates, but better seeing than the worse templates)
    • see very similar results between best-seeing templates and preconvolution cases.
    • convolve science shows a different pattern, and larger median diffim variance
    • restricting just to bright sources (SNR > 20, to avoid threshold effects) we see again same detections in all but the convolve science case.
    • stamps for convolve science shows lots of ringing artifacts with dark central pixels–suggests this is not working correctly algorithmically
• SITCOM team updates (Andrew Connolly  et al.):
  • everyone is out on parental leave!
  • starting (with John Franklin) testing a neural net for fitting Zernikes
• PreOps V&V (Colin Slater ): 
  • need access to lots of summary visualizations to see where processing succeeds and fails
  • showing "plot navigator" which combines Tim Morton's web interface with Sophie's code, analysis_drp
  • pngs written out by analysis_drp, metadata stored in the butler–so can change collections, etc.
  • still very much shared-risk, but something useful to think about building on for monitoring AP processing
  • since it runs in the SP, simple to turn on for IDF, SLAC, etc.
  • can display multiple plots at once
  • analysis_drp shows how it's done, but it's mainly writing pngs to the butler. 
  • can run as an afterburner
• Review CI (https://chronograf-demo.lsst.codes/):
  • no changes

• Next meetings (all at 4pm Pacific):
  • no meeting in January (DM quiet period)
  • Next meeting February 7.  (conflicts with Ops review, but it should be done by 4pm Pacific)
• AOB
  • None

Action Items