Date 15 Feb 2022
Location Zoom alias : https://ls.st/rsv
Zoom: https://noirlab-edu.zoom.us/j/96120781994?pwd=bCtpT3Q3b2RsU1ZBRUFaSnUrZXo3Zz09#success
Attendees Peter Ferguson
Jeffrey Carlin
Keith Bechtol
Regrets Useful links Metric tracking dashboards
Rubin science pipelines weekly change log and Github repo: https://github.com/lsst-dm/lsst_git_changelog
Status of jenkins jobs and slack channle for notifications: #dmj-s_verify_drp_metrics slack channel
Discussion items Item Who Pre-meeting notes Minutes News, announcements and task review CI status, bugs, issues of the week
All The filtering before multi-match branch was merged last week. No feedback yet on memory consumption yet.Is there a way we can test this more directly ourselves? What stats are most useful? The metric values computed on nighly runs have, in general, slightly improved as evidenced on SQuaSH. Our hypothesis is that we are getting better quality matches. wPerp. Jeff started work on a ticket branch. wPerp is currently loading full tract object catalogs in three bands from FITS files. We think this is the only current task that is loading full tract object catalogs. Faro Development status Fall 2021 epic Spring 2022 epic Backlog epic: Progress on conversion to parquet New multimatching approach
N-way matcher (RFC-824) Another matcher (see #dm-science-pipelines; see here )FGCM strategy to store the indices. We should see how Eli implements. Take object table and find the best match in each source Dan Taranu has 1:1 matcher Peter will do some research and report back in 2 weeks so we can have a more concrete discussion. Converting to parquet format
Photometric repeatability (Peter is working on this DM-32613)Is there a way to re-use the calculations from PA and PF? Forced source table only has PSF fluxes. Will we want other flux measurements? Aperture fluxes? Extended model fluxes?
What other infrastructure is needed before we start having many more developers and adding many more metrics?
Diagnostic capability. Ability to load the in-memory objects in a notebook. Additional diagnostic capability. Ability to save a figure. Metric naming convention and metric package specifications Documentation Unit tests I/O for matching to be able to swap algorithmsN-way (e.g., photometric repeatability) 1:1 matching (e.g., object detection completeness versus external catalog) Synthetic source injection Proposed prioritization
Matching Metric naming conventions Data processing campaigns
RC2/DC2 reprocessing epic : w_2021_40 RC2: w_2021_44 DC2 : AuxTel AP/DiffIm DP0.2 AOB Potential co-working session ideas here
List of tasks (Confluence)