...
- false positive rates (Eric Bellm ):
- Eric compared current HiTS processing to DMTN-006 and DMTN-021 (currently in https://github.com/lsst-dm/ap_pipe-notebooks/blob/u/ebellm/false-positive-rates/false-positive-rates.ipynb, to move to DM-19292). DMTNs looked at a single DECam chip, and with 5.0 sigma detection and decorrelation on got ~50 merged DIASource detections. Our ap_pipe processing is giving us strictly more, with several hundred detections per chip and a tail up to 2000+ DIASources.
- Configuration problem?
- Colin Slater stops in to report that the DMTN-006 & 21 used "entirely stock" configs but did use InstCals (post-ISR DECam CCD processed with the CP pipelines). So we're not doing an apples-to-apples comparison. Also those were image-to-image differences, not image-template.
- so we could run imageDifference.py on instcals to see if we can reproduce past results. Might we able to just retarget to obsDecam.nullIsr as a configuration in ap_pipe?
- and we should push ahead with stack-generated cals to see if that makes a difference.
- HiTS reprocessing (Meredith Rawls):
- last week, we found that photometric calibration configs weren't being set to reasonable defaults. Pulled over some from obs_subaru. Doesn't seem to have made a big difference in the calibration factors, but made a substantial change in the number of DIAObjects that have many DIASources (Update: this is a change in the flagging behavior in the matcher). Investigation ongoing to see if the calibration has changed.
- Eric thinks we need to capture some metrics for photometric calibration
- DES processing (Unknown User (emorganson)):
- preparing for the Ops Rehearsal with Hsin-Fang
ap_
association (Unknown User (cmorrison)):- pandas is now an option for input and return for PPDB–some concern about handling mixed types in the columns
- next step is to do some timing tests, first on sqlite and then Andy will chase other dbs
- template forced fluxes/background (Eric Bellm )
- Metrics & ap_verify (Krzysztof Findeisen ):
- (out today)
- image differencing (Unknown User (gkovacs) ):
- actions from call with Yusra:
Unknown User (gkovacs) check for decorrelation in our dataset. To do
Unknown User (gkovacs) figure out where “bright source” cutoff lies: To do
check # detections against expectation
- see above & DM-19292
find truly bad visits and debug them
- Eric will do this: ccdVisits with > 2000 DIASources in cw_processed3 (and 4):
ccdVisitId 41137103 3062.0 41979116 2198.0 41980411 2074.0 41980415 2501.0 41980416 2632.0 41980440 2226.0 41980447 2482.0
- invite David back to chat about differencing over lunch sometime
- John Parejko reports that there were some algorithmic changes in the unmerged diffim cleanup from David Gabor has been plotting kernel solution coefficients, hasn't chased DMTN-006/kbmod configs
- actions from call with Yusra:
- DCR (Ian Sullivan ):
- Eric sent him chasing after DIASource detections rather than just dipoles as a part of the review of DM-18709
- quick summary is that with variable PSFs + DCR model, many artifacts aren't detected by image differencing, which makes metrics seem to "get better" with worse seeing even when there are visual artifacts
- some subset of images have better seeing than the template, so require special handling
- Review CI (https://chronograf-demo.lsst.codes/):
- no changes
- AOB
- Ian may be on jury duty next Monday
- Chris want to sprint plan this week or next since he will be traveling week after next
- Meredith is out next week