Skip to end of metadata
Go to start of metadata


attending: Eric Bellm Eric Morganson Ian Sullivan Krzysztof Findeisen John SwinbankGabor Kovacs Chris MorrisonMeredith Rawls

Topics for discussion

  • HiTS processing (Meredith Rawls ):
    • discussion of DM-20534 - Getting issue details... STATUS : comparing ap_pipe run on HiTS 2015 with standard processing, instcal-instcal (CP-processed data), calexp-calexp
    • most data processed.   instcals had some substantial failures–some astrometry failures and some diffim matching failures, and some crappy single CCDs.  Calexp-calexp was in general better but a few missing CCDs
    • standard processing has about an order of magnitude more sources than calexps, which go away when your filter on flags:
      • instcal-instcals have fewer pixel flags, but that's because nullIsr is not setting pixel flags on the instcals
      • why the big jump in flagged sources (and hence raw DIASource counts) when going from calexp-calexp to calexp-coadd?
        • calexp-coadd is ~193 raw DIASources/ccd, calexp-calexp is ~76
        • is some of this the issue of the variances in the coadds being underestimated, as Yusra identified?
        • we should check with Yusra next week about rescaling variance, maybe see if Meredith can take on some of that work (although @gabor had the ticket DM-20558)
        • still lots of dipoles in the fit
  • DES processing (Eric Morganson):
    • putting together some testing code for moving around kafka alerts
    • some question of whether alert contents are sufficiently in-memory–we think it is since association is already computing stats on all the
    • John asserts that doing the packaging is "fun and easy"
    • Eric requests to be involved in this "fun and easy" part when it comes time to do it
  • ap_association (Chris Morrison):
    • DIAForcedSources are now in the database as their own table (psFlux on diffim and totFlux on calexp, both at existing DIAObject position)
  • DCR (Ian Sullivan ):
    • trying to run all of HiTS with DCR, but everything failed with strange errors, working on fixing it
  • Image differencing (Gabor Kovacs ):
    • clearing ticket backlog; didn't save screenshots from processing so having trouble backfilling the tickets.
    • looking next at metadata ticket and scalevariance ticket.  thinks he wants to write a wrapper to make the scale variance in a standalone step.
  • Review CI (
  • AOB
    • Meredith Rawls will give a short talk about DIA processing at PCW
    • Krzysztof Findeisen wrote a metric for memory usage–one gotcha is that it isn't useful to run on subtasks–only runs in a cumulative way, so can't break it down lower than ap_pipe.  appears on ap runtime chronograph
    • Ian Sullivan will write small DCR talk for same session
    • no meeting next week due to PCW.  Eric will be out the subsequent week (Hotwired) so hopes John Swinbank will take over.