Logistics

Date , 10:00 PT

BlueJeans (note the new number): https://bluejeans.com/583102117

Attendees 

Regrets

Goals

Weekly meeting of the DM-SST to discuss scientific aspects of the Data Management System. One in four meetings us  dedicated to the discussions about the LSST Science Platform. 

Discussion items


ItemWhoNotesConclusions and action items
Introduction

Leanne Guy Gregory Dubois-Felsmann

Introduction to the scope and goals of this once-every-4-week dedicated LSP SST meeting


Definition of scope of LSP product owner

Leanne Guy

Does product ownership cover LSP development or are we also POs of running an operating service?

  • Formally operations is outside of our scope, however there is agreement that, in the absence of other guidance from senior management, it is natural and appropriate for us to act as the POs for a pre-operations service in commissioning. This is very important for scientific oversight; we need to know that we are meeting users needs.
  • Simon Krughoff: This is also the only real way that we can meet verify/validate the platform.
  • Gregory Dubois-Felsmann: We need to focus on catalog based access more than has been possible thus far. How do we stage meaningful user testing of the science platform towards the majority of use cases, which are around catalogs, interfaces to catalogs, datasets.

  • Datasets: we need to produce catalogs. Which datasets should be used to produce catalogs for validation testing?

    • SDSS stripe8 is old. Assess if it is worth the effort (hand-massaging) to reprocess it or whether we should just move to HSC. Zeljko Ivezic thinks that there is still value in the SDSS forced photometry.
    • WISE & Gaia Fritz needs to ingest the WISE & Gaia DR2 datasets.
    • HSC: DR2 release is slated for early 2019. HSC has very little deep multi-epoch data. Stripe 82 is an ultra deep slice of data and provides a nive proportional multi-epoch sample to LSST
  • DM-SST LSP Product owners will act as POs for a pre-operations LSP service in commissioning
  • DM-SST will continue to run and support tutorials, bootcamps, support stack-club, promote the LSP to SCs and commissioning; all of which provide us with essential user feedback on the product and contributes to scientific validation. Experiences from these interactions should be fed into the scientific validation plans
  • Simon Krughoff Update the stack-club roster to schedule SST people once every 2 weeks. We will not schedule anyone in the off-weeks



LSP review planning

Gregory Dubois-Felsmann

Plans for the LSP review : Review Charge

The last and only review was 2017 - 1.5 yrs ago was the only review. Since this review, 2 key aspects of the platform have matured significantly

  • The VO centric design for DAX interfaces .
  • Next-to-data processing; we have a much more concrete picture for N2D processing as well as a  meaningful design and prototype

Additionally we need to ensure that the descope is fully and appropriately a addressed in the review


Science Data Model talk: Colin Slater is this not too backend-focused for this review? Gregory Dubois-Felsmann : This is not a DPDD-like talk but rather a presentation that we will have a system with robust metadata on the data products; this forms part of the user experience. We will generate a user facing doc page, 

Qserv: We should clearly expose topics like the overlap radius and query restrictions of Qserv. The goal of this review is to gain feedback from the community and make whatever changes are needed to ensure LSST science now.


Demos.

Simon Krughoff there will be priorities on which systems need work for these demos/

  • TAP to portal must be done
  • A&A Do we need an implementation for review ? Agreement that we need to demonstrate

    cross-aspect single sign-on for single sign-on but that we can present the design and future plans for other aspects of A&A

Sample queries: Simon Krughoff The API team need sample queries. They need them at least a month before the review and in a templated form so they can be fuzzed. See these examples.

Colin Slater requests that all those who need to update documents be informed of what/when ASAP.

The venue dates are now fixed: Tucson, 10,11,12 April 2019 and the review is on Review Hub

Overdue or undated DM Science Team LSP tickets

Gregory Dubois-Felsmann

See list below - still 34 unreviewed and undated tickets
Open actions concerning the LSP

Gregory Dubois-Felsmann



AOB

Leanne Guy

Please read and comment on RFC-571

Next LSP meeting will be 2019-03-08

List of SST tasks (Confluence)

DescriptionDue dateAssigneeTask appears on
  • Robert Lupton Clarify the meaning of time in the object table. 1 sentence description in sdm_schemas, can link to a short DMTN.  Update 2022-02-09: Meeting to resolve this on 2022-02-21  
28 Feb 2022Robert Lupton2018-11-05 DM SST F2F Agenda and Meeting notes
  • Gregory Dubois-Felsmann check if SDM standardization is adequately represented in project documents, and whether DMTN-067 should be required.
31 Mar 2022Gregory Dubois-Felsmann2022-02-14 DM-SST Virtual F2F Agenda and Meeting notes
28 Feb 2023Leanne Guy2023-01-23 DM-SST Agenda and Meeting Notes
  • Leanne Guy talk to Steve R about presenting plans for the ShearObject table to PST and SciCollab chairs   
20 Mar 2023Leanne Guy2023-02-27 DM-SST Agenda and Meeting Notes
31 Mar 2023Jim Bosch2023-02-27 DM-SST Agenda and Meeting Notes
  • Leanne Guy  talk to Gregory Dubois-Felsmann to review the original intent of the AFS-related Portal requirements before deciding on a course of action  
29 May 2023Leanne Guy2023-05-01 DM-SST Focus Meeting - Brokers in Commissioning
  • Leanne Guy Prepare to consult the PST on the question of providing compressed PVIs for AP outputs, to cover the period before the data become available in a DR.  
02 Jun 2023Leanne Guy2023-03-27 DM-SST Agenda and Meeting Notes
  • Jim Bosch Incorporate 30-60 day period for raws on disk into the strawman proposal and present to KT  
26 Jun 2023Jim Bosch2023-05-08 DM-SST Agenda and Meeting Notes
  • Parker Fagrelius Patrick Ingraham  how long will it take to do a scan as described? No need to scan the whole WL range but will require additional points outside nominal lambda range.  
30 Jun 2023Parker Fagrelius2023-03-27 DM-SST Agenda and Meeting Notes
31 Jul 2023Colin Slater2023-07-10 DM-SST Agenda and Meeting Notes
  • Eli Rykoff , Leanne Guy  Develop a proposal for what calibration processing, hardware, data we actually need and what will be needed for DR1. This has implications for the ORR and for prioritisation of work in commissioning  
31 Jul 2023Eli Rykoff2023-01-30 DM-SST Agenda and Meeting Notes
  • Yusra AlSayyad will look to see if there is any effort to help on option 1  
28 Aug 2023Yusra AlSayyad2023-08-14 DM-SST Agenda and Meeting Notes
  • Jim Bosch  Provide a physical example of that a  up on cell table would look like fo the Colin Slater / DAX team to review  
31 Aug 2023Jim Bosch2023-02-27 DM-SST Agenda and Meeting Notes
  •  "What is the pathway to defining the data products that are required to meet DMS-REQ-0266" Jeffrey Carlin   
30 Nov 2023Jeffrey Carlin2023-10-23 DM-SST vF2F Agenda and Meeting Notes
30 Nov 2023Gregory Dubois-Felsmann2023-10-23 DM-SST vF2F Agenda and Meeting Notes
30 Nov 2023Leanne Guy2023-10-23 DM-SST vF2F Agenda and Meeting Notes
  • Jeffrey Carlin follow up with KT on DMS-REQ-0176 and DMS-REQ-0315 to update/disaggregate this for latest base/summit infrastructure split.  
30 Nov 2023Jeffrey Carlin2023-10-23 DM-SST vF2F Agenda and Meeting Notes
  • Jim Bosch Follow up on the possibility of investigating further the ability to process 2 collections in parallel.   
31 Jan 2024Jim Bosch2023-12-04 DM-SST Agenda and Meeting Notes
31 Jan 2024Jeffrey Carlin2023-12-04 DM-SST Agenda and Meeting Notes
Gregory Dubois-Felsmann2023-10-23 DM-SST vF2F Agenda and Meeting Notes


Overdue or Undated DM Science Team tickets

Key Summary T Created Updated Due Assignee Reporter P Status Resolution
Refresh

LIT tickets of interest to DM Science

T Key Summary Assignee Reporter P Status Resolution Created Updated Due
Refresh