2pm Pacific Time

BlueJeans: https://bluejeans.com/426716450



Topics for discussion:

  • Gen 3 conversion (all):
    • Krzysztof: Gen3 CI is uploading to squash!
    • DECam linearity got merged
    • DECam crosstalk still in review with Meredith, should be close.
    • We put DM-21888 (Gen3 HiTS CI) in April for Krzysztof–after which point we think we are at gen3 parity

Key Summary T Created Updated Due Assignee Reporter P Status Resolution

  • Difference imaging sprint (Ian Sullivan ):
    • Ken: looking at good/middle/worse seeing coadds
    • Meredith: updating plotting tools in ap_pipe-notebooks
    • Ian: worked some on diffim unit tests for simulated data, will be ready for metrics soon
    • Eric: with Meredith, got cutout→Zooniverse pipeline working on Postgres + Gen3, also did some prototype calcuations/metrics on the cutouts
    • Gabor: put metadata exploration notebook on Trello board (

      diffimTests/tickets/DM-28201_diffim_metadata/inspect_diff_metadata.ipynb).  Doesn't seem to be super useful, though, since the metadata system hasn't been extensively used in later pipeline development–might be useful place for us to put new entries going forward.  Also thinking some about number of sources detected w/ decorr. afterburner–doesn't think it's a simple increase/decrease.  Also working on technote, thinking about preconvolution.

  • report from NCSA (Unknown User (emorganson)  + Unknown User (manninga) et al): 
    • out
  • ap_association/SDM/fakes (Unknown User (cmorrison)):
    • working on functors, doesn't have reviewers yet
    • Lee Kelvin asking about AP fakes
  • Review CI (https://chronograf-demo.lsst.codes/):
    •  COSMOS shows a small bump in the DIAsrc/source ratio on Mar 1
    • gen2 and gen3 COSMOS don't agree on object counts–we will ticket this and pick it up in the next sprint planning once we have a bit more history to look at
      • Lauren MacArthur just popping in here...I'm not entirely sure of the context for these object counts, but if they rely on Gen2/Gen3 parity through singe frame processing we are not quite there yet, but I am working on it (and getting close!) See  DM-28858 - Getting issue details... STATUS  and  DM-28936 - Getting issue details... STATUS .  Ignore me if this is entirely irrelevant for this issue!
    • Gen3 COSMOS runtime looks a little bit better, but this may be because Gen3 doesn't account for I/O time the same way (it's not a walltime calculation)
    • todos:
      • Eric Bellm write ticket for Meredith Rawls ( TBC) to investigate differences in object counts between Gen2 and Gen3 COSMOS CI  
      • Krzysztof Findeisen put together a gen2 vs gen3 dashboard on squash ( DM-29113 - Getting issue details... STATUS  
  • Review outstanding action items
    • Ian: Saha bulge catalogs: group has just released a paper on this dataset.  Catalog is for their internal use right now, but Ian is going to ask for access for our testing.
    • External catalog of the DECam galactic bulge dataset will be released soon. Details: https://academic.oup.com/mnras/article/499/2/2357/5900983
  • AOB
    • Meredith: gen3 inter-chip crosstalk may require some extra work since it's going to require two pipetask run calls


Action Items

DescriptionDue dateAssigneeTask appears on
  • Eric Bellm write up retention policy and sizing for APDB  
17 Jul 2023Eric BellmAP Pipeline Meeting, 2023-06-26
  • Eric Bellm dig into alert serialization test in ap_association and ticket ways to improve it  
11 Sep 2023Eric BellmAP Pipeline Meeting, 2023-08-21