2022-02-15 Science Metric Development Agenda and Meeting notes

CI status, bugs, issues of the week

• Discussion on slack about wPerp requiring a large amount of memory: https://lsstc.slack.com/archives/C011E3ZUZAB/p1643131727008000

• The filtering before multi-match branch was merged last week. No feedback yet on memory consumption yet.
  • Is there a way we can test this more directly ourselves? What stats are most useful?
    
    • See https://github.com/erykoff/fgcm/blob/66f97d1a4ccb31ddf4bd31ac11195c0c0c8b1a4a/fgcm/fgcmUtilities.py#L60-L86
  • The metric values computed on nighly runs have, in general, slightly improved as evidenced on SQuaSH. Our hypothesis is that we are getting better quality matches.
• wPerp. Jeff started work on a ticket branch. wPerp is currently loading full tract object catalogs in three bands from FITS files. We think this is the only current task that is loading full tract object catalogs.

New multimatching approach

• N-way matcher (RFC-824)
• Another matcher (see #dm-science-pipelines; see here)
  • FGCM strategy to store the indices. We should see how Eli implements.
• Take object table and find the best match in each source
• Dan Taranu has 1:1 matcher
• Peter will do some research and report back in 2 weeks so we can have a more concrete discussion.

Converting to parquet format

• Photometric repeatability (Peter is working on this DM-32613)
  • Is there a way to re-use the calculations from PA and PF?

Forced source table only has PSF fluxes. Will we want other flux measurements? Aperture fluxes? Extended model fluxes?

What other infrastructure is needed before we start having many more developers and adding many more metrics?

• Diagnostic capability. Ability to load the in-memory objects in a notebook.
• Additional diagnostic capability. Ability to save a figure.
• Metric naming convention and metric package specifications
• Documentation
• Unit tests
• I/O for matching  to be able to swap algorithms
  • N-way (e.g., photometric repeatability)
  • 1:1 matching (e.g., object detection completeness versus external catalog)
• Synthetic source injection

Proposed prioritization

1. Matching
2. Metric naming conventions

Data processing campaigns  

Potential co-working session ideas here