BlueJeans: https://bluejeans.com/426716450
attending: Eric Bellm Unknown User (emorganson) Ian Sullivan Krzysztof Findeisen John SwinbankUnknown User (gkovacs) Unknown User (cmorrison)Meredith Rawls
Topics for discussion
- HiTS processing (Meredith Rawls ):
- discussion of DM-20534 - Getting issue details... STATUS : comparing ap_pipe run on HiTS 2015 with standard processing, instcal-instcal (CP-processed data), calexp-calexp
- most data processed. instcals had some substantial failures–some astrometry failures and some diffim matching failures, and some crappy single CCDs. Calexp-calexp was in general better but a few missing CCDs
- standard processing has about an order of magnitude more sources than calexps, which go away when your filter on flags:
- instcal-instcals have fewer pixel flags, but that's because nullIsr is not setting pixel flags on the instcals
- why the big jump in flagged sources (and hence raw DIASource counts) when going from calexp-calexp to calexp-coadd?
- calexp-coadd is ~193 raw DIASources/ccd, calexp-calexp is ~76
- is some of this the issue of the variances in the coadds being underestimated, as Yusra identified?
- we should check with Yusra next week about rescaling variance, maybe see if Meredith can take on some of that work (although @gabor had the ticket DM-20558)
- still lots of dipoles in the fit
- DES processing (Unknown User (emorganson)):
- putting together some testing code for moving around kafka alerts
- some question of whether alert contents are sufficiently in-memory–we think it is since association is already computing stats on all the
- John asserts that doing the packaging is "fun and easy"
- Eric requests to be involved in this "fun and easy" part when it comes time to do it
ap_association (Unknown User (cmorrison)):- DIAForcedSources are now in the database as their own table (psFlux on diffim and totFlux on calexp, both at existing DIAObject position)
- DCR (Ian Sullivan ):
- trying to run all of HiTS with DCR, but everything failed with strange errors, working on fixing it
- Image differencing (Unknown User (gkovacs) ):
- clearing ticket backlog; didn't save screenshots from processing so having trouble backfilling the tickets.
- looking next at metadata ticket and scalevariance ticket. thinks he wants to write a wrapper to make the scale variance in a standalone step.
- Review CI (https://chronograf-demo.lsst.codes/):
- AOB
- Meredith Rawls will give a short talk about DIA processing at PCW
- Krzysztof Findeisen wrote a metric for memory usage–one gotcha is that it isn't useful to run on subtasks–only runs in a cumulative way, so can't break it down lower than ap_pipe. appears on ap runtime chronograph
- Ian Sullivan will write small DCR talk for same session
- no meeting next week due to PCW. Eric will be out the subsequent week (Hotwired) so hopes John Swinbank will take over.