Skip to end of metadata
Go to start of metadata




Zoom alias :



Erik Dennihy Peter Ferguson Colin Slater Keith Bechtol Michael Martinez Robert Lupton


Leanne Guy Jeffrey Carlin 

Metric tracking dashboards

Rubin science pipelines weekly change log and  Github repo:

Status of jenkins jobs and slack channle for notifications:  #dmj-s_verify_drp_metrics slack channel 

Discussion items

ItemWhoPre-meeting notesMinutes
News, announcements and task review


CI status, bugs, issues of the week



Faro Development status 
  • Spring 2022 epic  DM-33385 - Getting issue details... STATUS
  • Backlog epic:  DM-29525 - Getting issue details... STATUS
  • Progress on conversion to parquet
  • Remaining infrastructure in faro to support adding many more metrics
    • Relationship with visualization
      • Visualization capability within faro
      • Connection to analysis_drp – understanding the relationship between these two software packages
      • Infrastructure for visualizing ensembles of metric values / statistics / correlations
    • Diagnostic capability
      • Partly addressed with shelf
      • Other tooling for understanding an unexpected metric value – related to visualization above
    • Metric naming convention and metric package specifications
    • Review of documentation
    • Review of unit tests / coverage
    • List of metrics to be implemented / review of metric specifications
    • Any additional analysis contexts needed?
    • Persisting metrics as lsst.verify.Measurement objects
      • Performance question when scaling up to thousands of metric values
    • Matching routine for using external catalogs.
  • MatchedCatalogTable base class merged last week
  • Rough example of PA1 using MatchedCatalogTableMeasurement (Git branch tickets/DM-32613
    • Just using aperature photometry 12 as the measure since the PsfFlux column in not available in current matched table.
  • DM-33959 has been bumped up the priority list for pipelines. We're expecting a fix to land in w15. DM-31855 is essentially a duplicate so this has been marked as 'won't fix. This would apply astrometric and and photometric calibrations to source parquet tables. 
    • Would a second matching step need to run to make table of matched sources?
    • Join once and persist or join every time we run a metric.
    • There is a value is
    • We could profile the joining now using the isolatedStarAssociationTask keys as an index

Data processing campaigns  

  • RC2/DC2 reprocessing epic : DM-26911 - Getting issue details... STATUS
    • w_2021_40 RC2:   DM-32248 - Getting issue details... STATUS
    • w_2021_44 DC2 : DM-32391 - Getting issue details... STATUS
  • AuxTel - Erik to give update
  • AP/DiffIm
  • Prior issues with AuxTel photometric calibration issues now understood, a bug to the calibrated fluxes written to parquet tables was discovered and a fix is under development (see DM-34019 - Getting issue details... STATUS ). For now, we can still calibrate fluxes using AFW tables and photoCalib object. Calibration is good but shows some systematics in scatter. 
  • I need to generate/register a skymap for the different fields in order perform co-addition, is there a way to do this outside of repo/main to allow for experimentation? 
    • butler register-skymap --config-file /path/to/file /path/to/repo (could experiment on rc2subset, example config in obs_lsst )
    • Discrete skymaps will become more work than worth at some point.
    • Option would be to make a small repository and do a test with discrete skymap. Then make skymap in repo/main
  • Are there any single-visit/detector level metrics are implemented now in Faro that are worth running on the afw tables we have already generated?
  • Keith and Erik to schedule a meeting to talk about next steps for computing metrics to support AuxTel observing
    • What quantities are most useful to AuxTel during the night
    • Merlin is probably the person to talk to about current AuxTel data processing and RubinTV visualization of metric values
    • Short term
  • analysis_drp, producing many QA plots. Person is Sophie Reed. Potentially generate a webpage of useful plots. Another source of


Potential co-working session ideas here

Next meeting   

List of tasks (Confluence)

DescriptionDue dateAssigneeTask appears on
  • Leanne Guy to talk to Science Pipelines (Yusra) about when do this transfer  
19 Oct 2021Leanne Guy2021-09-28 Science Metric Development Agenda and Meeting notes
  • Leanne Guy arrange to disucss at a future meeting if there are metrics from PDR3 & this paper that we might want to include in faro.   
26 Oct 2021Leanne Guy2021-08-31 Science Metric Development Agenda and Meeting notes
  • Colin to ask about capturing ideas for improvement to the stellar locus algorithm   
30 Nov 2021 2021-11-09 Science Metric Development Agenda and Meeting notes
  • Colin Slater to make a preliminary draft agenda for a workshop to clarify visualization use cases for science verification and validation
Colin Slater2022-04-19 Science Metric Development Agenda and Meeting notes
  • Jeffrey Carlin to review metric specification package organization and the relationship to formal requirements documents
Jeffrey Carlin2022-04-19 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol Schedule a time to have focused discussion on verification package, potentially next status meeting
Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol to make a ticket to better understand mapping of these camera and calibration products characterization efforts to verification documents and the focus of these efforts. Discuss with the SCLT
Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes