AP Pipeline Meeting, 2020-06-22

2pm Pacific Time

BlueJeans: https://bluejeans.com/426716450

attending:

• Eric Bellm
• Unknown User (cmorrison)
• Unknown User (gkovacs)
• Ian Sullivan
• John Swinbank
• Krzysztof Findeisen
• Meredith Rawls
• Spencer Nelson

regrets:

Topics for discussion:

• sample alerts for brokers (Eric Bellm , Unknown User (cmorrison) , Spencer Nelson ):
  • Eric Bellm README is most of the way there
  • Unknown User (cmorrison)
    • cutouts implemented–will be some issues with WCS at poles, still tbd.  Plus XY0 is missing
    • started incorporating ForcedSources–has to do some apdb schema modifications to include filtername
    • next is template cutouts
  • John Swinbank are we still waiting on the SAC?  Eric Bellm no, we can release these whenever we're ready
• Gen 3 conversion (all):
  • Krzysztof Findeisen --got reviews back on Gen 3, not much progress otherwise

• Precursor processing (Meredith Rawls ):
  • did June ap_pipe HiTS reprocessing.  Last month there was an increase in objects that we thought was an ISR bug now fixed; but counts haven't changed.  Did lose erroneous edge bug
  • could be decorrelation afterburner changes?
  • was using new templates for months, but CI was on old templates until recently–we should keep these sync in the future to aid us in tracking down changes
• report from NCSA (Unknown User (emorganson)  + Unknown User (manninga) et al):
  • out today
• Alert Distribution (Spencer Nelson ):
  • not much.  no word from Carl
• ap_association/SDM/fakes/alerts (Unknown User (cmorrison)):
  • (discussed above)
• Crowded field photometry (Ian Sullivan ):
  • cleared lots of reviews
  • looking at DECam & HSC crowded field datasets
• Image differencing (Unknown User (gkovacs) ):
  • still drafting tech note
    • wondering about padding with white noise rather than zeros
    • wondering about making gaussian PSF in Fourier space–makes diffim more appealing, but doesn't really matter for detection if you detect on Scorr image
    • will try to circulate it so we can talk to RHL next week
  • needs to respond to Ian's review
• Review CI (https://chronograf-demo.lsst.codes/):
  • looking at memory usage, which increased due to higher resolution template.
  • what is the actual limit we have to watch out for?   Some worries that we might already be close, especially with multiple fourier space copies etc.
    • John Swinbankping KT about AP memory capacity per CPU
  • "we should assume that AP processing will take place on systems with no more than 5-6 GB of RAM per core."
• Review outstanding action items
• AOB
  • Meredith Rawls wonders if there is any general documentation on running e.g. ImageDifference.py
    • Unknown User (gkovacs) reminds us that this will be phased out in current form in Gen 3

Action Items