Data Management
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata

Date

 

Location

Zoom alias : https://ls.st/rsv

Zoom:  https://noirlab-edu.zoom.us/j/96120781994?pwd=bCtpT3Q3b2RsU1ZBRUFaSnUrZXo3Zz09#success

Attendees

Regrets

Metric tracking dashboards

Rubin science pipelines weekly change log and  Github repo: https://github.com/lsst-dm/lsst_git_changelog

Status of jenkins jobs and slack channle for notifications:  #dmj-s_verify_drp_metrics slack channel 

Discussion items

ItemWhoPre-meeting notesMinutes
News, announcements and task review

All

Computing clusters / data facilities

  • Rubin construction effort at NCSA will be shutting down 15 August 2022. Machines will be shipped out. A few people from NCSA will continue into LSST Operations. We will want to migrate work. Datasets will be migrated in a planned manner
  • SLAC is looking to be ready at end of July. SLAC will do full reprocessing at HSC PDR3. SLAC USDF will support commissioning activities. Not exactly clear when AuxTel data processing will transition to USDF.
    • USDF has understanding that it is responsible for commissioning, including AuxTel
  • Data Previews at IDF
  • Prompt Processing Prototype at IDF. Mainly for development purposes with intent to switch to using USDF.
  • Relationship with Antu
  • Colin Slater Follow up with Fritz to check status of USDF readiness for AuxTel data  



CI status, bugs, issues of the week

All
  • Piff turn on resulted in small changes in metric values
  •  Most metric values changed between 30 and 31 March.
  • Questions of exactly what changed and how would we know. Star selection? Measurements of photometry? We currently don't have quick ways of diagnosing.
Faro Development status 

Leanne Guy

  • Spring 2022 epic  DM-33385 - Getting issue details... STATUS
  • Backlog epic:  DM-29525 - Getting issue details... STATUS
  • Progress on conversion to parquet
  • Status of DM pipeline tickets
    • Bug in the application of zeropoints to source catalogs resolved DM-34019 - Getting issue details... STATUS
    • Feature: Photometric and astrometric corrections applied to source catalogs DM-33959 - Getting issue details... STATUS
  • Performance metrics to support AuxTel imaging campaigns
    • Example diagnostics from image quality team
    • More ideas for single-visit performance metrics starting to collect in FAFF context
    • Comparison with metadata and pixel-level image data? In a commissioning context, there are specific drivers for automated analyses that compare quantities from source catalogs and other sources

Data processing campaigns  

Jeffrey Carlin Keith Bechtol

  • RC2/DC2 reprocessing epic : DM-26911 - Getting issue details... STATUS
    • w_2021_40 RC2:   DM-32248 - Getting issue details... STATUS
    • w_2021_44 DC2 : DM-32391 - Getting issue details... STATUS
  • AuxTel  (Erik Dennihy )
  • AP/DiffIm
  • Erik developed a workflow - a ticket is written prior to each run, lays out scheduler conditions, etc. Followup ticket in SITCOm prj. to track the details - source collections, etc. 
    • Example ticket of imaging survey observing requests - CAP-868 - Getting issue details... STATUS
    • Example ticket of imaging survey data processing - SITCOM-253 - Getting issue details... STATUS
DP0.2

Colin Slater  



AOB

Leanne Guy

Potential co-working session ideas here

Next meeting  

Faro work planning meeting is scheduled for next week.  Please add your thoughts to 2022-04-12 Science Verification - Planning Meeting Agenda and Meeting notes so that we can make most efficient use of the meeting time


List of tasks (Confluence)

DescriptionDue dateAssigneeTask appears on
  • Leanne Guy to talk to Science Pipelines (Yusra) about when do this transfer  
19 Oct 2021Leanne Guy2021-09-28 Science Metric Development Agenda and Meeting notes
  • Leanne Guy arrange to disucss at a future meeting if there are metrics from PDR3 & this paper that we might want to include in faro.   
26 Oct 2021Leanne Guy2021-08-31 Science Metric Development Agenda and Meeting notes
  • Colin to ask about capturing ideas for improvement to the stellar locus algorithm   
30 Nov 2021 2021-11-09 Science Metric Development Agenda and Meeting notes
  • Jeffrey Carlin to review metric specification package organization and the relationship to formal requirements documents
 Jeffrey Carlin2022-04-19 Science Metric Development Agenda and Meeting notes
  • Colin Slater to make a preliminary draft agenda for a workshop to clarify visualization use cases for science verification and validation
 Colin Slater2022-04-19 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol Schedule a time to have focused discussion on verification package, potentially next status meeting
 Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol to make a ticket to better understand mapping of these camera and calibration products characterization efforts to verification documents and the focus of these efforts. Discuss with the SCLT
 Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes