2pm Pacific Time

BlueJeans: https://bluejeans.com/426716450



Topics for discussion:

  • Pipelines Infrastructure (Krzysztof Findeisen ):
    • discuss RFC-775 - Getting issue details... STATUS
      • Meredith has already been getting lots of comments from confused users about where pipelines live
      • concerns:
        • design is operations-focused (and positive in that vein), doesn't capture ad-hoc science users, and maybe development/construction process
        • concern about config sharing between AP & DRP
          • Eric has some hope this will be eased by greater focus on single cameras/datasets in commissioning, and in ops DRP configs will be updated on very definite timescales
  • Precursor processing (Meredith RawlsKenneth Herner ):
    • not much from Meredith; Ken fighting some postgres schema issues–defaulting to write into the public schema rather than the user schema.
    • ken will ticket this to raise the point rather than silently fixing it
  • Difference imaging algorithms  (Unknown User (gkovacs)Ian Sullivan ):
    • Gabor working on debugging decorrelation afterburner
      • three orders of magnitude difference in noise between science and template.
      • having trouble isolating decorrelation afterburner effect because we also ran with doRescaleVariance=True–so needs to rerun
      • we're also not getting a good test here because we ran vs. the best seeing template–so this is the wrong case to convolve the science image
      • so multiple things to disentangle
      • also getting new and unusual crashes from dipole fitting
    • discussing mask handling with Robert–wants to discuss again with RHL
    • status of Gen3 DECam repo?
      • Meredith: Jim planning to make a DECam repo in the shared repo, but it is not there yet–Meredith can give access to her private repo in the meantime (no linearity or crosstalk, though)
    • Ian: will be scheduling more diffim sprint followup tickets in May sprint planning
  • report from NCSA (Unknown User (emorganson)  + Unknown User (manninga) et al): 
  • ap_association/SDM/fakes/astrometry (Unknown User (cmorrison)):
    • three tickets in review
    • going to start pulling validate_drp data to look at AM metrics with distortion-model fitter
    • preparing to gut mapApData once SDM merges
    • reviews
  • real-bogus (Eric BellmJohn Parejko ):
    • finished writeup of data flow, draft tasks to write for RB infrastructure
    • next to talk to some domain experts
  • Review CI (https://chronograf-demo.lsst.codes/):
    • lots of confusing movement in the ratio of DIASources/science sources in both datasets and gen2 vs gen3
    • we think  that doScaleVariance was false for all combinations prior to April 21; at that time HiTS was ~identical in gen2 vs gen3 but COSMOS was discrepant
    • changes April 21 moved gen3 in both datasets (scale variance on)–both datasets now discrepant
    • ticket on April 22 turned on gen2 ScaleVariance for COSMOS but it doesn't match gen 3
    • so currently neither agree, and gen2 HiTS is the only dataset without scaleVariance
    • Chris can look at source catalogs to see if SFM is different
    • Meredith to continue investigation ticket.
  • Review outstanding action items
  • AOB
    • QA meeting next week.
    • please fill out sprint planning


Action Items

DescriptionDue dateAssigneeTask appears on
  • Eric Bellm write up retention policy and sizing for APDB  
17 Jul 2023Eric BellmAP Pipeline Meeting, 2023-06-26
  • Eric Bellm dig into alert serialization test in ap_association and ticket ways to improve it  
11 Sep 2023Eric BellmAP Pipeline Meeting, 2023-08-21
  • Eric Bellm review broker communication channels  
25 Sep 2023Eric BellmAP Pipeline Meeting, 2023-08-14