BlueJeans: https://bluejeans.com/426716450

attending: Eric Bellm Unknown User (emorganson) Meredith RawlsUnknown User (gkovacs)Ian SullivanUnknown User (cmorrison)

Topics for discussion

• false positive rates (Eric Bellm ):
  • Eric compared current HiTS processing to DMTN-006 and DMTN-021 (currently in https://github.com/lsst-dm/ap_pipe-notebooks/blob/u/ebellm/false-positive-rates/false-positive-rates.ipynb, to move to DM-19292).  DMTNs looked at a single DECam chip, and with 5.0 sigma detection and decorrelation on got ~50 merged DIASource detections.  Our ap_pipe processing is giving us strictly more, with several hundred detections per chip and a tail up to 2000+ DIASources.
  • Configuration problem?
  • Colin Slater stops in to report that the DMTN-006 & 21 used "entirely stock" configs but did use InstCals (post-ISR DECam CCD processed with the CP pipelines).  So we're not doing an apples-to-apples comparison.  Also those were image-to-image differences, not image-template.
    
    • so we could run imageDifference.py on instcals to see if we can reproduce past results.  Might we able to just retarget to obsDecam.nullIsr as a configuration in ap_pipe?
    • and we should push ahead with stack-generated cals to see if that makes a difference.
• HiTS reprocessing (Meredith Rawls):
  • last week, we found that photometric calibration configs weren't being set to reasonable defaults.  Pulled over some from obs_subaru.  Doesn't seem to have made a big difference in the calibration factors, but made a substantial change in the number of DIAObjects that have many DIASources (Update: this is a change in the flagging behavior in the matcher).  Investigation ongoing to see if the calibration has changed.
  • Eric thinks we need to capture some metrics for photometric calibration
• DES processing (Unknown User (emorganson)):
  • preparing for the Ops Rehearsal with Hsin-Fang
• ap_association (Unknown User (cmorrison)):
  • pandas is now an option for input and return for PPDB–some concern about handling mixed types in the columns
  • next step is to do some timing tests, first on sqlite and then Andy will chase other dbs
  • template forced fluxes/background (Eric Bellm )
• Metrics & ap_verify (Krzysztof Findeisen ):
  • (out today)
• image differencing (Unknown User (gkovacs) ):
  • actions from call with Yusra:
    

    • Unknown User (gkovacs) check for decorrelation in our dataset.  To do

    • Unknown User (gkovacs) figure out where “bright source” cutoff lies: To do

    • check # detections against expectation

      • see above & DM-19292

    • find truly bad visits and debug them

      • Eric: ccdVisits with > 2000 DIASources in cw_processed3 (and 4):

      • ccdVisitId
        41137103    3062.0
        41979116    2198.0
        41980411    2074.0
        41980415    2501.0
        41980416    2632.0
        41980440    2226.0
        41980447    2482.0

  • invite David back to chat about differencing over lunch sometime
  • John Parejko reports that there were some algorithmic changes in the unmerged diffim cleanup from David
  Gabor has been plotting kernel solution coefficients, hasn't chased DMTN-006/kbmod configs
• DCR (Ian Sullivan ):
  • Eric sent him chasing after DIASource detections rather than just dipoles as a part of the review of DM-18709
  • quick summary is that with variable PSFs + DCR model, many artifacts aren't detected by image differencing, which makes metrics seem to "get better" with worse seeing even when there are visual artifacts
  • some subset of images have better seeing than the template, so require special handling
• Review CI (https://chronograf-demo.lsst.codes/):
  • no changes
• AOB
  • Ian may be on jury duty next Monday
  • Chris want to sprint plan this week or next since he will be traveling week after next
  • Meredith is out next week