AP Pipeline Meeting, 2020-08-03

2pm Pacific Time

BlueJeans: https://bluejeans.com/426716450

attending:

• Eric Bellm
• Ian Sullivan
• John Swinbank
• Krzysztof Findeisen
• Meredith Rawls
• Unknown User (gkovacs)

regrets:

• Unknown User (emorganson)

Topics for discussion:

• Gen 3 conversion (all):
  • Krzysztof Findeisen:
    • gen3 running within ap_verify code, so just needs to hook it up to the CLI

• Precursor processing (Meredith Rawls ):
  • not much to report; sat in on SST this morning with Simon et al demo of "new" metrics system
    • John Swinbank wishes he had a better sense of the context;
    • Krzysztof Findeisen says gen3→ SQUASH is new
      • Looking at the slides from the meeting, it appears that Simon et al. worked around DM-21916 - Getting issue details... STATUS by using a script to convert the Gen 3 measurements back to Gen 2
  • next monthly QA meeting is in two weeks: maybe will show Postgres, better HSC processing
• report from NCSA (Unknown User (emorganson)  + Unknown User (manninga) et al):
  • out today
• Alert Distribution (Spencer Nelson ):
  • out today
• ap_association/SDM/fakes/alerts (Unknown User (cmorrison)):
  • out today
• Crowded field photometry (Ian Sullivan ):
  • returning after a week away
  • processing the DECam Saha data on lsst-dev.  has been running for ~2 days.  of 13k exposures, 6k have had fatal errors.  Almost all in PSF modeling (all PSF candidates get rejected)–have been putting together a few notes, will spawn some tickets to look into individual errors.  Failures concentrate in first five CCDs–on both fields, so not just location on sky.  10x worse in i-band than in g-band.
  • Meredith: are we fringe correcting?  (sounds like not, but Ian isn't seeing the effect immediately on fields that process)
  • did need to apply some config tweaks (increasing the reduced chisq for PSF candidates–to 20).  Ian thinks they shouldn't be applied more generally, but has captured them
  • next on to the HSC data.
  • thinks we just need a new way to determine PSFs in crowded fields
• Image differencing (Unknown User (gkovacs) ):
  • ZOGY implementation merged
    • John Swinbank worries about bitrot.  Should this be in CI?  Should Meredith run it also in monthly processing?
    • Eric in favor of ticketing CI implemetnation but thinks it's premature to schedule an algorithm bakeoff.  not sure it's worth running in Meredith's processing if she's
  • found a small bug when there's little template overlap, fixing that now
  • next to small improvements in decorr. afterburner
  • returning again to the mask convolution RFC
• Review CI (https://chronograf-demo.lsst.codes/):
  • no changes
• Review outstanding action items

• AOB
  • new page on configs: AP Pipeline Configuration comments welcome
    • everyone should feel to edit.  Not meant to capture the current state of pipelines at all times, but a way to keep track of "knobs we found helpful to twiddle."
      • Eric Bellm check configurations page for investigations we can mark as "done" 05 Oct 2020 
  • no meeting next week (virtual PCW)

Action Items