Meeting at 14:00 (Project time).

Participants

Present

Apologies

Eric Bellm

Agenda

Working Group Scope

Per the charge, this WG is aims to deliver a plan for “QA” tooling and activities in support of the construction and verification of DM's Science Pipelines.

I suggest we interpret “QA” quite broadly — this could be things that DM needs to build, activities that DM needs to carry out, etc, in order to make sure that the scientific results produced by DM are delivered to meet our requirements for both runtime performance and scientific validity, as well as tools that we should provide to make developers' lives easier.

However, we don't need to define all the tools that will be needed to commission, operate or do science with the LSST system.

WG Members

Per the charge (LDM-622):

Eric Bellm (AP)
Hsin-Fang Chiang (LDF)
Angelo Fausti (SQuaRE)
Simon Krughoff (SQuaRE)
Lauren MacArthur (DRP)
Unknown User (tmorton) (DRP)
Trey Roby (SUIT)
John Swinbank (AP & System Management)

This is your last chance to bail out if you want to escape...!

Regular Meeting Schedule

We will meet every week at the same time — 14:00 Project. Additional meetings may be scheduled as required.

This means we have only nine meetings until the end of June when the WG is due to wind up!

Slack

Slack channel #dm-qawg. Please join!

Key Topics

Based on the previous request for use cases (from Simon, in February), together with some thoughts of my own, DMTN-074 identified the following areas of activity:

Developer Support
- Test Datasets
- Visualization, plotting & debugging frameworks
- Support for running & debugging at scale
- Notifications and dashboards
Code Quality
- Unit tests
- Integration tests
- Static analysis, code linters, etc
Metric Verification
- Runtime performance analysis
- High-level metric tracking
- Ad-hoc metric calculation and tracking
Scientific Assessment
- Drill-down

(The DMTN defines what each of those means in more detail; we can run through that verbally now.)

Question for the group: what are we missing? Aim for next meeting: flesh these out by developing a collection of “use cases”. Discuss what form those use cases should take.

(Note that for now I'm keeping these independent of implementation; we need to collect metrics regardless of whether we use SQuaSH or stone tablets; we need to make plots, regardless of whether we use matplotlib or pen & paper.)

Working Group Timetable

Meeting Date	Topics
2018-05-02	WG scope WG members Meeting schedule Slack Key topics Timetable
2018-05-09	Use cases Refined list of key topics Approach to planning for each key topic Assign WG members to each key topic Start narrative overview of topic
2018-05-16	Review narrative progress; consider restructuring or reorganizing topics as required.
2018-05-23	DMLT meeting this week; QAWG meeting may be cancelled. Watch this space.
2018-05-30	Review completed narratives. Map narratives to existing software & services. Identify missing pieces. Begin writing requirements documentation.
2018-06-06	Review requirements documentation progress.
2018-06-13	Complete draft of requirements documentation. Assign editing and review tasks.
2018-06-20	Progress review.
2018-06-27	Completion of WG deliverables.

Actions for Next Meeting

Make sure you are familiar with LDM-522 and DMTN-074.
Brain-dump all of the “use cases” which you think are relevant to the work of this WG.
- Consider using a structure similar to §3 of DMTN-074.
- Provide as much detail as you can. If you know a detailed procedure, the appropriate test data, or whatever else, specify it; if you just have a vague feeling that we ought to be worried about something, say that.
- Let's collect input here.

Next Meeting

Wednesday 2018-05-09, 14:00 (Project), https://bluejeans.com/426716450