AP Pipeline Meeting, 2020-10-12

2pm Pacific Time

BlueJeans: https://bluejeans.com/426716450

attending:

• Eric Bellm
• Ian Sullivan
• Unknown User (cmorrison)
• Ian Sullivan
• Unknown User (gkovacs)
• Meredith Rawls
• Kenneth Herner

regrets:

Topics for discussion:

• Gen 3 conversion (all):
  • Krzysztof Findeisen :
    • did a couple of cleanup tickets
      • dealt with slow import of the HiTS data
      • fixed Jenkins

• Precursor processing (Meredith Rawls ): 
  • (reminder: QA meeting next week)
  • Meredith has a review panel then–can send plots if she's not going to be able to attend
  • last week was busy with satellite meeting
• report from NCSA (Unknown User (emorganson)  + Unknown User (manninga) et al):
  • out today
• ap_association/SDM/fakes/alerts (Unknown User (cmorrison)):
  • showing astrometry tolerances on HiTS CI
    • successes up to shifts of 100 arcsec; failures are outside of pipeline defaults
    • big rotations around the boresight likely to fail (translate to big shifts)
      • refcat loading pads only 300 pix around initial WCS; hard to pad more without loading a lot more HTM
    • next steps are to actually compare cataloged sources one to one to calculate AM1 etc.
    • Eric and Chris M to find a time to chat with RHL to confirm we're heading the most useful direction
• Crowded field photometry (Ian Sullivan ):
  • ran Saha bulge data all the way through AP pipe... many CCDs are missing, perhaps due to sqlite timeouts?
    • Ian tried running it again with the timeout increased by 10x, didn't make a significant improvement
  • some of the coadds failed silently; fixed those failures, but going to investigate other potential failures
    • ran out of memory on the node, which killed the job and everything thereafter
• Image differencing (Unknown User (gkovacs) ):
  • looking at undersampled case, not ready to chat with Robert yet
  • having trouble running processCcd on ap_verify right now
    • Chris M says it's a config tweak.
• Preops project (Kenneth Herner  and all):
  • Ops bootcamp is this week; talks will be recorded.
    
• Review CI (https://chronograf-demo.lsst.codes/):
  • one visit (410802)/two CCDs show a ~10% jump in memory usage on 10/1–-what happened since then?
    • these two CCDs reverted to previous behavior
  • both HSC visits showed a ~10% jump in memory use–let's check back next week.
  • we think we should be running PSFEx by default now–can we confirm it?  any visible changes?
    • no, it's not in yet, DM-5304 - Getting issue details... STATUS
• Review outstanding action items

• AOB
  • none

Action Items