Date

 

Location

Zoom alias : https://ls.st/rsv

Zoom:  https://noirlab-edu.zoom.us/j/96120781994?pwd=bCtpT3Q3b2RsU1ZBRUFaSnUrZXo3Zz09#success


Attendees

Regrets

Metric tracking dashboards

Rubin science pipelines weekly change log and  Github repo: https://github.com/lsst-dm/lsst_git_changelog

Status of jenkins jobs and slack channle for notifications:  #dmj-s_verify_drp_metrics slack channel 

Discussion items

ItemWhoPre-meeting notesMinutes
News, announcements and task review

All



CI status, bugs, issues of the week

All

Review of action items from last week:

  • Filtering before matching. This has been waiting on . All activity has been killed on NCSA machines.
    • Failure mode. Introducing new matched catalogs made CI jobs break. If jenkins job completes today, we should be able to merge.
  • wPerp switching over to using parquet tables. Jeff has started working on a ticket branch. Needs the data selectors to be merged first.
    • Discussed with Sophie Reed. Would like to adopt faro as the place where algorithms for calculating metrics live. Port functions into faro. Will need to confirm with Yusra. Don't expect problems. Is there an issue with duplicating selectors? The ones we have implemented so far in faro are similar to analysis_drp, but more general.

What is the order of operations:

Matching before filtering in FITS input is separate and trying to finish ASAP.

Selectors is a per-requisite for the wPerp fix.

Asking Sophie to review the selectors ticket. Keith will also take a look.

Jeff noticed that metric values shift by about 5% (in both directions) after the filtering before matching. We are keeping more data because we are throwing out fewer objects due to a single problematic measurement.

We should make sure that we have the capability to diagnose the behavior when

Faro Development status 

Leanne Guy

  • Fall 2021 epic DM-30748 - Getting issue details... STATUS
  • Spring 2022 epic  DM-33385 - Getting issue details... STATUS
  • Backlog epic:  DM-29525 - Getting issue details... STATUS
  • Progress on conversion to parquet

v23 will be cut this afternoon. Would not be unexpected to cut of v23.0.2. We still want wPerp by step 5. French group wants this and we want to be able to have it available as an option to backport.

"freeze-drying bugs". Capture enough information to be able to do full interactive exploration.

ATLAS REFCAT 2 suited for shallower AuxTel data

DELVE. Is this our plan going into ComCam commissioning?

Who is responsible for

  • identifying needed refcats
  • processing commissioning data
  • maintaining refcats

SIT-Com will drive the definition of needs, and may need to take on some of these tasks in practice. It will also need to have an operations role and name if these are activities that will be continued into operations.

Here is the current delve coverage, if having full grip coverage is vital we could try an prioritize that over the next few months https://data.darkenergysurvey.org/fnalmisc/decam_coverage.png

Data processing campaigns  

Jeffrey Carlin Keith Bechtol

  • RC2/DC2 reprocessing epic : DM-26911 - Getting issue details... STATUS
    • w_2021_40 RC2:   DM-32248 - Getting issue details... STATUS
    • w_2021_44 DC2 : DM-32391 - Getting issue details... STATUS
  • AuxTel
  • AP/DiffIm

DP0.2

Colin Slater 

Leanne Guy  

Faro for DP0.2


AOB

Leanne Guy

Potential co-working session ideas here

Next meeting  


List of tasks (Confluence)

DescriptionDue dateAssigneeTask appears on
  • Leanne Guy to talk to Science Pipelines (Yusra) about when do this transfer  
19 Oct 2021Leanne Guy2021-09-28 Science Metric Development Agenda and Meeting notes
  • Leanne Guy arrange to disucss at a future meeting if there are metrics from PDR3 & this paper that we might want to include in faro.   
26 Oct 2021Leanne Guy2021-08-31 Science Metric Development Agenda and Meeting notes
  • Colin to ask about capturing ideas for improvement to the stellar locus algorithm   
30 Nov 2021 2021-11-09 Science Metric Development Agenda and Meeting notes
  • Colin Slater to make a preliminary draft agenda for a workshop to clarify visualization use cases for science verification and validation
 Colin Slater2022-04-19 Science Metric Development Agenda and Meeting notes
  • Jeffrey Carlin to review metric specification package organization and the relationship to formal requirements documents
 Jeffrey Carlin2022-04-19 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol Schedule a time to have focused discussion on verification package, potentially next status meeting
 Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol to make a ticket to better understand mapping of these camera and calibration products characterization efforts to verification documents and the focus of these efforts. Discuss with the SCLT
 Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes