Expect DM developers outside the existing middleware team to start engaging with the “Gen3 porting” at the PCW; no requirement that they begin before that.
Expect “DMLT checkpoints” showing status relative to Fritz Mueller 's slides at the PCW, and potentially another in ~October.
Previously-announced plans for a PipelineTask UI review are on hold for now; the Middleware Team is not convinced they would learn much from such a review. Should aim to solidify plans and set expectations on this by (say) the PCW.
Not yet clear if there will be a separate “user-batch” service that is separate from the batch production service. BaBar experience suggests this is a risky idea.
Planning an LSST-wide validation effort, rather than something DM specific. A document describing this will be forthcoming, produced in conjunction with Chuck Claver.
There is no LDM-240-equivalent “glide path” on metrics; we think this is probably not a problem for science numbers, but we should be careful to ensure we are working towards realistic performance numbers.
An overview of the current status of, and future plans for, the product & document trees, together with an understanding of how requirements flow between them. In particular, this should address questions like:
What is the plan for finishing/adopting DMTN-104?
Will the document tree be linked to the product tree (e.g., with one requirements doc per product)?
Are documents like LDM-602 obsolete, on the basis that there is not an “Alert Production” product? If so, when will new documents be created and where should requirements live?
Given that this may be a work in progress, an understanding of goals and timelines is the aim.
Should remove the “coming in 2018” labels from the DocTree!
While we are happy with the idea that not every component requires a detailed requirement flowdown, we note that product owners for subsidiary components must understand how their work is constrained by higher-level requirements.
We note the product-to-requirement mapping is shown in LDM-148.
We do require test cases explicitly for each product, even if there is no separate set of flowed-down requirements for that product.
We draw a distinction between a “product”, which is a major component which will undergo its own release process, and a “component”, which (in software terms) map to single repositories, but may not be released independently.
We expect design documentation to live separately from the component repository in general (but adding it to the repository is may be appropriate on occasion).
Kian-Tat Lim — Reconciliation of LDM-148 naming with the product tree.
Summarise the conclusions of LDM-672 and DMTN-106.
Outline plans for short-term and longer-term changes to the DM release process.
Discussion about which of our existing repositories are well-formed products. Is daf_butler? daf_base? afw? Balancing the desire to reduce the number of products with wanting to make fixes in one repository without forcing releases of many apparently unrelated packages. The answer seems to be that the product tree needs to be design “intelligently”.
There are various concerns about the definitions that Gabriele Comoretto proposes, which may be best addressed in a focused technical follow-up session or breakout meeting.
The Commissioning Team is part of the “non-operational” stakeholders per Gabriele Comoretto definitions. However, we note that Commissioning will require stable services: arguably, they will require different policies applied to (say) Pipelines (which require quick fixes) and services.
At least ~50 of our existing repositories are updated at least every few months.
SQuaRE requests that each team nominate a “release engineer” who can help resolve build problems.
What does the recently-announced data framework mean for DM?
What sort of in-kind contributions could enhance or add to the expected capabilities of the DM system? We should prepare a list to feed into future discussions.
E.g. crowded fields, ...
“Contributions that will offset NSF or DOE costs for operations” is the key line.
There are things we need in operations which could make useful contributions.
Worry that often external contributions “mostly work”.
We will still need the on-project team to do the core work.
One could imagine grey areas around the edges, like QA.
New algorithms which expand the scientific output of the survey would be welcome, assuming they don't increase our costs.
There is some ambiguity about whether in-kind contributions can apply to construction (per “talking points” memo page 1) or only to operations (per page 2).
Blum suggests we would consider innovative contributions to construction, but it's hard to see what that would be.
A new data rights policy document will be forthcoming; Bob Blum has authorized the release of a draft to DMLT members.
Bob tells us that there will be a process within Ops to identify things which could make useful contributions; the DM list can feed into this. This list will then be audited for risks, and presented to NSF and DOE.
Bob says: forget about the LSST Resource Board; it will not approve in-kind contributions.
We note the the DM system will need to be technically capable of providing the access specified in the data rights policy.
There is scope for both “offset” and “added-value” (upscopes) from the in-kind contributions.
Wil O'Mullane — create and circulate a Google Document for DMLT members to suggest possible in-kind contributions.
The LSST@Asia conference was held in Sydney, Australia at the University of New South Wales from 20-23 May 2019.
The project was represented by Tony Tyson, Leanne Guy, Robert Blum and Robert Lupton.
Tony presented the project status, Leanne the LSST science drivers and data products, Bob the new framework for access to LSST data by the international community and Robert Lupton on HSC reprocessing using the LSST science pipelines.
Robert and Leanne jointly ran an session demoing the LSST Science Platform
It was made clear that going forwards, international access to LSST data would be in the form of ‘in-kind’ contributions and that it would not be possible to pay for data access rights.
Most of the discussion focused around the new data access framework and how international communities could get involved, both now during the construction project and in operations.
Regional scientists were most interested to understand what would count as an ‘in-kind’ contribution, e.g telescope time for follow-up, and how in-kind contributions would be translate to FTE
Some concern was expressed about what would happen to people with data rights now but for whom no in-kind contribution could be identified in operations.
A lot of the regional research interests are focused around Galaxy studies and cosmology and many of the talks from regional scientists focused on synergies with LSST with talks
talks focused on what they can contribute, including followup ideas, and contributions to commissioning. Many have joined science collaborations already.
David Trilling of the SSSC pointed out that as the Solar System data products will be sent to the MPC, that effectively all Solar System data is public.
European LSST colleagues shared their experiences of building and an LSST community outside of the USA; there is enormous interest in the Asia-Pacific region in building a regional LSST LSST community. The community has been named LSST@A^3 (Asia, Australia, Africa) .
The next LSST@A^3 meeting will take place at Peking University, Kavli led by Hu Zhan.
We note that not all in-kind contributions would necessarily directly integrate with LSST: other contributions of capabilities to the US astronomical community might also be possible.
We note that the current baseline does not send all solar system data to the MPC.
Date for the next meeting has not yet been set (may be in either one or two years).
In the April monthly report there are over 40 DM milestones listed as being delayed.
We will review this list, confirm the status of each milestone and what must be done to accomplish it (or to drop it from the plan), and assign action items to substantially reduce this list ahead of this summer's reviews.
Michelle Butler — redefine LDM-503-08b to use AuxTel, rather than CCOB data, and then complete the milestone.
Wil O'Mullane — encourage Victor to approve LCRs corresponding to outstanding ICDs.
John Swinbank — update due date on the DM-DRP-8 to couple to AuxTel delivery.
John Swinbank — LCR DM-DRP-12 & DM-DRP-11 & DM-DRP-29 into the future