Note that the “Dinky” — the train from Princeton Junction to Princeton — will be suspended during the DMLT meeting. A substitute bus service will be provided by NJ Transit. Please allow extra time for your journey.
We discussed whether some DM work can be pushed into the future as effectively a “no cost extension” to construction; the practicalities of this are not clear...
In terms of what's acceptable to the agencies.
And in terms of whether there will be staffing in the operations era to work on this.
If there are people in your team/institute who would be interested in moving to AURA positions, please let Wil O'Mullane know.
There are practical issues here, regarding which states are acceptable for people to reside, and whether there is office space available for them.
And of course the downsides to breaking up groups.
DM10 alert filters interface may actually represent a scope increase in 02C.03 in terms of providing a mini-broker interface.
We believe that the current portal is sufficient for commissioning data releases (after finishing the closeout plan).
We discussed increase staffing load resulting from making commissioning data widely available: this is expected to be significant.
“As discussed at the PST F2F meeting in September, DM should review the names of people going into commissioning” (requested by Leanne Guy )
Not immediately clear whether this means permanent reassignment of staff, or those DM folks who are temporarily assigned to assist the Commissioning Team with specific activities.
The commissioning support activities listed are validation of the DM system, but verificationof the LSST system.
None of the commissioning activities listed cover aspects of the system outside Science Pipelines.
Concern expressed about the impacts of people being reassigned to commissioning on the rest of the DM team.
Often the people who might be most effective with commissioning are also those most necessary for facilitating efforts within DM; concern expressed that this will have a disproportionate impact on the DM schedule.
Detailed definition of the contents of the commissioning tests is with Leanne Guy and Keith Bechtol . Expectation that they will often involve repeating tests that have been performed within DM.
Early Ops funding is now available, and during FY19 (ie, this year) the ADs for both Data Facility and Science Operations (Margaret Gelman and Wil O'Mullane) are funded at 0.25 FTE. That's half an FTE coming out of DM management. How are we handling that?
Similarly there's a total of ~15 FTEs funded across Data Facility and Science Ops during FY20.
Please review:
The plans and schedule for transitioning staff;
The activities which the Operations Team will be carrying out with this effort, and how they relate to ongoing DM construction and the commissioning effort.
Some milestones (e.g. ops rehearsals) are effectively duplicated between DM and pre-operations funding; where possible, they will migrate from DM to pre-ops.
There's a worry that commissioning data maybecome available to the public more quickly than we currently plan; DM should be ready to scale up to address this.
And a feeling that simply making the data available for download will not adequately address this need.
But we acknowledge the cost and schedule impacts of this.
There's also a concern about what level of end-user support is implied by this.
How do we handle folks being required in DM and commissioning and pre-ops?
This is a matter of ongoing planning. There may be some overlap/double counting.
Discussion of how open the access to commissioning data & facilities should be, balancing getting input on commissioning from members of the community with the support load and “chaos” of wide access.
Wil O'Mullane — coordinate the writing of a memo describing what community DM can support during commissioning.
The proposal is to develop a policy for handling packages which have been developed externally and which their authors offer up for inclusion in pipeline processing, the Science Platform environment, or elsewhere in the DM system.
Obvious examples might be scientific algorithms contributed by the community.
There is history of external users refusing to make contributions like this due to the demands of DM engineering (code quality, review, tests, etc...)
Based on the modeling work done before the review this year, review proposed product tree, components characterization and relation with the document tree
Are there ways of providing greater uptime and lower latency using rolling upgrades, schema evolution with backfill, planned Observatory maintenance windows and daytime maintenance, etc.?
We're lacking L3 milestones in PMCS which describe the availabilities of services and capabilities which we know are coming, particularly later in construction.
Obvious examples include:
Butler Gen 3 / PipelineTask as the regular production environment;
Roll-out of the Pegasus WMS (or some other WMS);
Data back bone capabilities.
In addition, some existing milestones seem unclear about who is delivering what. For example:
“DM-SUIT-16: Commissioning DAC“ — is that really a SUIT deliverable?
“DM-SQRE-5: Notebook service ready for general science“ — is that really a SQuaRE deliverable?
We should have milestones describing the delivery of effectively everything in LDM-148.
This session is about expectation setting and setting a timeline for developing these milestones, not about generating them in real time!
Are there ways of providing greater uptime and lower latency using rolling upgrades, schema evolution with backfill, planned Observatory maintenance windows and daytime maintenance, etc.?
Work has been ongoing in the DM-SST to develop a formal DM data model and to refine the way in which LSE-163 (DPDD), the cat pacakge, LDM-153 (the baseline schema) are generated and managed.
This should be presented to and agreed by the DMLT.
Early Ops funding is now available, and during FY19 (ie, this year) the ADs for both Data Facility and Science Operations (Margaret Gelman and Wil O'Mullane) are funded at 0.25 FTE. That's half an FTE coming out of DM management. How are we handling that?
Similarly there's a total of ~15 FTEs funded across Data Facility and Science Ops during FY20.
What are the plans and schedule for transitioning staff?
Based on the modeling work done before the review this year, review proposed product tree, components characterization and relation with the document tree
We're lacking L3 milestones in PMCS which describe the availabilities of services and capabilities which we know are coming, particularly later in construction.
Obvious examples include:
Butler Gen 3 / PipelineTask as the regular production environment;
Roll-out of the Pegasus WMS (or some other WMS);
Data back bone capabilities.
In addition, some existing milestones seem unclear about who is delivering what. For example:
“DM-SUIT-16: Commissioning DAC“ — is that really a SUIT deliverable?
“DM-SQRE-5: Notebook service ready for general science“ — is that really a SQuaRE deliverable?
We should have milestones describing the delivery of effectively everything in LDM-148.
The half hour time estimate is for Wil O'Mullane to state the above, set expectations, and develop a timeline for delivery. If we actually start creating milestones during the meeting, it could take arbitrarily long.
The proposal is to develop a policy for handling packages which have been developed externally and which their authors offer up for inclusion in pipeline processing, the Science Platform environment, or elsewhere in the DM system.
Obvious examples might be scientific algorithms contributed by the community.
There is history of external users refusing to make contributions like this due to the demands of DM engineering (code quality, review, tests, etc...)
The aim is to share some preliminary considerations on the release process, taking in account the actual approach and giving a look on how this can evolve.
✅ (albeit in a potentially tightly squeezed slot...)