Zoom alias :



Peter Ferguson 

Jeffrey Carlin 

Keith Bechtol 


Metric tracking dashboards

Rubin science pipelines weekly change log and  Github repo:

Status of jenkins jobs and slack channle for notifications:  #dmj-s_verify_drp_metrics slack channel 

Discussion items

ItemWhoPre-meeting notesMinutes
News, announcements and task review


CI status, bugs, issues of the week

  • The filtering before multi-match branch was merged last week. No feedback yet on memory consumption yet.
  • wPerp. Jeff started work on a ticket branch. wPerp is currently loading full tract object catalogs in three bands from FITS files. We think this is the only current task that is loading full tract object catalogs.
Faro Development status 
  • Fall 2021 epic DM-30748 - Getting issue details... STATUS
  • Spring 2022 epic  DM-33385 - Getting issue details... STATUS
  • Backlog epic:  DM-29525 - Getting issue details... STATUS
  • Progress on conversion to parquet

New multimatching approach

  • N-way matcher (RFC-824)
  • Another matcher (see #dm-science-pipelines; see here)
    • FGCM strategy to store the indices. We should see how Eli implements.
  • Take object table and find the best match in each source
  • Dan Taranu has 1:1 matcher
  • Peter will do some research and report back in 2 weeks so we can have a more concrete discussion.

Converting to parquet format

  • Photometric repeatability (Peter is working on this DM-32613)
    • Is there a way to re-use the calculations from PA and PF?

Forced source table only has PSF fluxes. Will we want other flux measurements? Aperture fluxes? Extended model fluxes?

What other infrastructure is needed before we start having many more developers and adding many more metrics?

  • Diagnostic capability. Ability to load the in-memory objects in a notebook.
  • Additional diagnostic capability. Ability to save a figure.
  • Metric naming convention and metric package specifications
  • Documentation
  • Unit tests
  • I/O for matching  to be able to swap algorithms
    • N-way (e.g., photometric repeatability)
    • 1:1 matching (e.g., object detection completeness versus external catalog)
  • Synthetic source injection

Proposed prioritization

  1. Matching
  2. Metric naming conventions

Data processing campaigns  

  • RC2/DC2 reprocessing epic : DM-26911 - Getting issue details... STATUS
    • w_2021_40 RC2:   DM-32248 - Getting issue details... STATUS
    • w_2021_44 DC2 : DM-32391 - Getting issue details... STATUS
  • AuxTel
  • AP/DiffIm


Faro for DP0.2


Potential co-working session ideas here

Next meeting  

List of tasks (Confluence)

DescriptionDue dateAssigneeTask appears on
  • Leanne Guy to talk to Science Pipelines (Yusra) about when do this transfer  
19 Oct 2021Leanne Guy2021-09-28 Science Metric Development Agenda and Meeting notes
  • Leanne Guy arrange to disucss at a future meeting if there are metrics from PDR3 & this paper that we might want to include in faro.   
26 Oct 2021Leanne Guy2021-08-31 Science Metric Development Agenda and Meeting notes
  • Colin to ask about capturing ideas for improvement to the stellar locus algorithm   
30 Nov 2021 2021-11-09 Science Metric Development Agenda and Meeting notes
  • Colin Slater to make a preliminary draft agenda for a workshop to clarify visualization use cases for science verification and validation
Colin Slater2022-04-19 Science Metric Development Agenda and Meeting notes
  • Jeffrey Carlin to review metric specification package organization and the relationship to formal requirements documents
Jeffrey Carlin2022-04-19 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol Schedule a time to have focused discussion on verification package, potentially next status meeting
Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol to make a ticket to better understand mapping of these camera and calibration products characterization efforts to verification documents and the focus of these efforts. Discuss with the SCLT
Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes