Versions Compared

Old Version 50

changes.mady.by.user Yusra AlSayyad

Saved on

compared with

New Version 51

changes.mady.by.user John Swinbank

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Agenda (Draft)

Day 0: 2018-11-05

DM-SST meeting. Schedule will be provided by Leanne Guy.

Day 1: 2018-11-06

TimeTopicChairNotes
09:00WelcomeWil O'Mullane
  • Review outstanding actions
  • Review & adjust agenda
09:15Impact of scope changesWil O'Mullane
  • What portal development will happen, and when?
  • How does this impact on other aspects of the DM system (e.g. who provides the UI for specifying and managing alert filters)?
  • What other scope options are in the offing?
  • Is further work to define new scope options required?
10:30Break (Refreshments Provided)
11:00The LSST Data Model

Colin Slater

  • Provide an overview of work which has been undertaken in the DM-SST.
  • DMLT to agree on direction of travel, deliverables, etc.
12:00“DPDDification”Yusra AlSayyad
  • Review the tools which have been produced since the last meeting to turn pipeline outputs into something resembling the DPDD.
  • Describe how this relates to work on the Data Model.
12:30Lunch (Provided)
13:30Transition to CommissioningWil O'Mullane
  • “As discussed at the PST F2F meeting in September, DM should review the names of people going into commissioning” (requested by Leanne Guy )
  • Not immediately clear whether this means permanent reassignment of staff, or those DM folks who are temporarily assigned to assist the Commissioning Team with specific activities.
14:!5Transition to OperationsWil O'Mullane
  • Early Ops funding is now available, and during FY19 (ie, this year) the ADs for both Data Facility and Science Operations (Margaret Gelman and Wil O'Mullane) are funded at 0.25 FTE. That's half an FTE coming out of DM management. How are we handling that?
  • Similarly there's a total of ~15 FTEs funded across Data Facility and Science Ops during FY20.
  • Please review:
    • The plans and schedule for transitioning staff;
    • The activities which the Operations Team will be carrying out with this effort, and how they relate to ongoing DM construction and the commissioning effort.
15:00Break (Refreshments Provided)
15:30Product Tree and Document TreeUnknown User (gcomoretto)
  • Based on the modeling work done before the review this year, review proposed product tree, components characterization and relation with the document tree
16:00Management of externally contributed packagesWil O'Mullane
  • The proposal is to develop a policy for handling packages which have been developed externally and which their authors offer up for inclusion in pipeline processing, the Science Platform environment, or elsewhere in the DM system.
  • Obvious examples might be scientific algorithms contributed by the community.
  • There is history of external users refusing to make contributions like this due to the demands of DM engineering (code quality, review, tests, etc...)
16:30Release ProcessUnknown User (gcomoretto) / Leanne Guy
  • Proposed changes to the release process.
  • Requirements for back-porting bug fixes to release branches, in support of science collaborations, commissioning, etc.
17:00Close

Day 2: 2018-11-07

09:00Middleware & Workflow

Margaret Gelman & Fritz Mueller

  • Where are we on the selection & deployment of the workflow management system?
  • What's the development timeline for next generation middleware (Butler Gen 3, PipelineTask)?
  • What's the overall roadmap for middleware and workflow over the next ~4 years?
10:00Test Approach Using JiraUnknown User (gcomoretto)
  • How to create test specifications and perform test runs with Adaptavist Test Management in Jira.
10:30Break (Refreshments Provided)
11:00Level 3 MilestonesJohn Swinbank
  • We're lacking L3 milestones in PMCS which describe the availabilities of services and capabilities which we know are coming, particularly later in construction.
  • Obvious examples include:
    • Butler Gen 3 / PipelineTask as the regular production environment;
    • Roll-out of the Pegasus WMS (or some other WMS);
    • Data back bone capabilities.
  • In addition, some existing milestones seem unclear about who is delivering what. For example:
    • “DM-SUIT-16: Commissioning DAC“ — is that really a SUIT deliverable?

    • “DM-SQRE-5: Notebook service ready for general science“ — is that really a SQuaRE deliverable?

  • We should have milestones describing the delivery of effectively everything in LDM-148.
  • This session is about expectation setting and setting a timeline for developing these milestones, not about generating them in real time!
11:30Review of Calibration plansRobert Lupton
  • What raw data is being taken?
  • What products are being generated?
  • What are the possible CPP execution periodicities?
12:30Lunch (Provided)
13:30Summit and Base Data Center Status and PlanningJeff Kantor
  • Brief overview of Summit and BDC construction status and move-in schedule
  • Identification of planned deployments of DM equipment to BDC, visits, tests/rehearsals in FY19 (and IT support required)
14:30DBB and Consolidated DB SLAKian-Tat Lim
  • Are there ways of providing greater uptime and lower latency using rolling upgrades, schema evolution with backfill, planned Observatory maintenance windows and daytime maintenance, etc.?
  • What are the features and timeline for DBB?

15:00Break (Refreshments Provided)
15:30Review of L2 Milestones during calendar 2019John Swinbank
  • Which milestones are scheduled?
  • Who is responsible for them?
  • What other milestones might be added?
16:00Plans for Operations RehearsalRobert Gruendl
  • Update on the plans for, and timing of, the early-2019 Operations Rehearsal (LDM-503-09).
17:00Close

EveningGroup dinner at 287 Hamilton Ave.

Day 3: 2018-11-08

09:00Plans for S19John Swinbank
10:30Break (Refreshments Provided)
11:00Plans for S19John Swinbank
  • Feedback from DMLT members on the plans heard earlier.
  • Finalize cross-team priorities.
  • Agree development plans for S19.
12:30Review action items & plans for next meeting

Wil O'Mullane


13:00Close


Pre-Meeting Planning

Suggested topics for discussion


TopicRequested byTime required (estimate)NotesIncluded in schedule above
Review of the DM people transitioning to commissioning

Leanne Guy


As discussed at the PST F2F meeting in September, DM should review the names of people going into commissioning

✅

Milestone test plans

Kian-Tat Lim


Continue work on content and dates for upcoming milestones
Descope optionsKian-Tat Lim

✅
DBB and Consolidated DB SLAKian-Tat Lim
Are there ways of providing greater uptime and lower latency using rolling upgrades, schema evolution with backfill, planned Observatory maintenance windows and daytime maintenance, etc.?✅
Review of Calibration plansKian-Tat Lim
What raw data is being taken? What products are being generated? What are the possible CPP execution periodicities?✅
Unusual Commissioning modesKian-Tat Lim
What things that we normally think of as slowly-changing (e.g. camera geometry, voltage setpoints) might not be during Commissioning?
Review of L2 milestones due during calendar 2019

John Swinbank

1 hour
✅
Review of plans for S19 development cycle

John Swinbank

2 hours
✅
Review impact of portal scope changesJohn Swinbank / Wil O'Mullane1.5 hours
  • What portal development will happen, and when?
  • How does this impact on other aspects of the DM system (e.g. who provides the UI for specifying and managing alert filters)?
✅
Status of DPDDificationJohn Swinbank0.5 hours
  • Update on work started at the May DMLT F2F.
✅
LSST Data ModelJohn Swinbank1 hour
  • Work has been ongoing in the DM-SST to develop a formal DM data model and to refine the way in which LSE-163 (DPDD), the cat pacakge, LDM-153 (the baseline schema) are generated and managed.
  • This should be presented to and agreed by the DMLT.
✅
Transition to operationsJohn Swinbank1 hour
  • Early Ops funding is now available, and during FY19 (ie, this year) the ADs for both Data Facility and Science Operations (Margaret Gelman and Wil O'Mullane) are funded at 0.25 FTE. That's half an FTE coming out of DM management. How are we handling that?
  • Similarly there's a total of ~15 FTEs funded across Data Facility and Science Ops during FY20.
  • What are the plans and schedule for transitioning staff?
✅
Product Tree and Document TreeUnknown User (gcomoretto)0.5 hour
  • Based on the modeling work done before the review this year, review proposed product tree, components characterization and relation with the document tree
✅
Test Approach Using Jira

Unknown User (gcomoretto)

0.5 hour
  • How to do test specs and test runs with ATM in Jira
✅
Summit and Base Data Center Status and PlanningJeff Kantor1 hour
  • Brief overview of Summit and BDC construction status and move-in schedule
  • Identification of planned deployments of DM equipment to BDC, visits, tests/rehearsals in FY19 (and IT support required)
✅
Workflow management

Leanne Guy

1 hour
  • Where are we on the decision and implementation of a WMS?
✅
L3 milestonesJohn Swinbank / Wil O'Mullane0.5 hour
  • We're lacking L3 milestones in PMCS which describe the availabilities of services and capabilities which we know are coming, particularly later in construction.
  • Obvious examples include:
    • Butler Gen 3 / PipelineTask as the regular production environment;
    • Roll-out of the Pegasus WMS (or some other WMS);
    • Data back bone capabilities.
  • In addition, some existing milestones seem unclear about who is delivering what. For example:
    • “DM-SUIT-16: Commissioning DAC“ — is that really a SUIT deliverable?

    • “DM-SQRE-5: Notebook service ready for general science“ — is that really a SQuaRE deliverable?

  • We should have milestones describing the delivery of effectively everything in LDM-148.
  • The half hour time estimate is for Wil O'Mullane to state the above, set expectations, and develop a timeline for delivery. If we actually start creating milestones during the meeting, it could take arbitrarily long.
✅
Management of externally contributed packagesWil O'Mullane0.5 hour
  • The proposal is to develop a policy for handling packages which have been developed externally and which their authors offer up for inclusion in pipeline processing, the Science Platform environment, or elsewhere in the DM system.
  • Obvious examples might be scientific algorithms contributed by the community.
  • There is history of external users refusing to make contributions like this due to the demands of DM engineering (code quality, review, tests, etc...)
✅
Next generation middleware timelineFritz Mueller0.5 hour
  • When do we expect BG3, PipelineTask, etc to be actively being used by developers? What's the roadmap for getting there?
✅
Early considerations on Release processUnknown User (gcomoretto)0.5 hour
  • The aim is to share some preliminary considerations on the release process, taking in account the actual approach and giving a look on how this can evolve.
✅ (albeit in a potentially tightly squeezed slot...)


  • Attached Documents

Attachments