Data Management
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata

Date

 

Location

Zoom alias : https://ls.st/rsv

Zoom:  https://noirlab-edu.zoom.us/j/96120781994?pwd=bCtpT3Q3b2RsU1ZBRUFaSnUrZXo3Zz09#success


Attendees

Regrets

  • Simon Krughoff

Metric tracking dashboards

Rubin science pipelines weekly change log and  Github repo: https://github.com/lsst-dm/lsst_git_changelog

Status of jenkins jobs and slack channle for notifications:  #dmj-s_verify_drp_metrics slack channel 

Discussion items

ItemWhoPre-meeting notesMinutes
News, announcements and task review

All

Faro paper submitted to SPIE. Draft: dmtn-211.lsst.io . Acceptance notification from SPIE is 06 Apr 2022, so nothing more to do until then. 


  • Along with other changes in Rubin leadership, there will be a reorganization of SIT-Com. Plan by the AMCR.

CI status, bugs, issues of the week

All
  • CI job has not been running for a week. There is an issue with Jenkins. Jobs so slow that they don't finish. KT is on it.
  • The new weekly should have the fix for non-deterministic wiggles in metric values. Since dashboard hasn't been running, Jeff is running his own tests.
  • When source tables are combined into a single parquet table, they are combined in a random order. Eli implemented change to enforce ordering by detector so that repeatable. Before fix, FGCM was getting slightly different calibrations each run. The algorithms have an order dependence.
    • This gives us a sense of the systematic floor of the algorithms
  • Separate issues. Require 50 repeated measurements to compute photometric repeatability. Jeff will change configuration for the CI jobs to be 20. yaml configs.
Faro Development status 

Leanne Guy

  • Fall 2021 epic DM-30748 - Getting issue details... STATUS
  • Backlog epic:  DM-29525 - Getting issue details... STATUS
  • Progress on conversion to parquet
  • Last week merged ticket so that refcat loader gives a single merged dataframe that includes refcat columns + proper motion corrected astrometry and color-term-corrected photometry 
  • Dan Taranu had developed a matcher. To use, we would do matching before the measurement task DM-32034 - Getting issue details... STATUS
  • Jeff and Peter working on selectors. There is PR. Once this is merged we can finish converting metrics to use parquet format DM-32611 - Getting issue details... STATUS  
  • ATLAS REFCAT2 RFC. General support to move forward. Closes tomorrow. Already shared up in the desired format.

Data processing campaigns  

Jeffrey Carlin Keith Bechtol

  • RC2/DC2 reprocessing epic : DM-26911 - Getting issue details... STATUS
    • w_2021_40 RC2:   DM-32248 - Getting issue details... STATUS
    • w_2021_44 DC2 : DM-32391 - Getting issue details... STATUS
  • AuxTel
  • AP/DiffIm

DP0.2

Colin Slater 

Leanne Guy  

Faro for DP0.2


AOB

Leanne Guy

Potential co-working session ideas here

Next meeting Homework task for everyone to take a look at tickets from the previous development epic and we'll do a big review of status + planning for new development cycle next week.


List of tasks (Confluence)

DescriptionDue dateAssigneeTask appears on
  • Leanne Guy to talk to Science Pipelines (Yusra) about when do this transfer  
19 Oct 2021Leanne Guy2021-09-28 Science Metric Development Agenda and Meeting notes
  • Leanne Guy arrange to disucss at a future meeting if there are metrics from PDR3 & this paper that we might want to include in faro.   
26 Oct 2021Leanne Guy2021-08-31 Science Metric Development Agenda and Meeting notes
  • Colin to ask about capturing ideas for improvement to the stellar locus algorithm   
30 Nov 2021 2021-11-09 Science Metric Development Agenda and Meeting notes
  • Jeffrey Carlin to review metric specification package organization and the relationship to formal requirements documents
 Jeffrey Carlin2022-04-19 Science Metric Development Agenda and Meeting notes
  • Colin Slater to make a preliminary draft agenda for a workshop to clarify visualization use cases for science verification and validation
 Colin Slater2022-04-19 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol Schedule a time to have focused discussion on verification package, potentially next status meeting
 Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol to make a ticket to better understand mapping of these camera and calibration products characterization efforts to verification documents and the focus of these efforts. Discuss with the SCLT
 Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes