Who is the point of contact for usability issues in middleware? Middleware support channel is the first stage of triage (for project-internal users), then a linear combination of Gregory and Jim. Product owner role is to prioritize tickets, e.g. in order to balance user needs vs needs of pipelines execution, etc.
From #dm-sst channel. Agreement that the scope of the middleware PO covers interfaces to the underlying execution/production system, but not the execution/production system itself (e.g PanDA, HTCondor).
Does the requirement on "once and only once" processing mean that we supply some software framework, or is it met by providing compute resources plus "cookbook" of recipes that guides people towards the "right way" of accessing the data once-and-only-once.
RHL: HSC experience was that users needed to reprocess images at the position of their candidate objects. Users had to write their own code to take those catalog and then activate batch processing of their image processing code. Would like users to not have to write their own slurm submissions to do this type of processing.
LPG: What have we promised for DP1? Unlikely that we will have anything like this on that short of a timescale.
RHL: we did say that we would provide a solution for this. GPDF: did not say we would have a cradle-to-grave solution for this, we said we would provide the tools to enable users.
CTS: need to be more specific about what an MVP solution would look like in order to converge on whether we could build and support that.
LPG: What are commissioning requirements on this? Treat commissioning as using project systems, doesn't need to be "user batch".
LPG: I am thinking about the AO / community contributions to the commisioning data processing and analysis and not the standard project processing of the data.
Eric: Solar systems objects will come "for free" from the SSP pipeline, because the data comes from the Minor Planet Center.
Eric: don't expect transient/variable catalogs to be sufficient to verify completeness and purity at the faint end, will need to depend on injected sources or other technical means to verify. Might be feasible at the bright end.
Why UV and IR? Keith: UV helpful for calibration of u band, IR for photo-z and S/G sep.
How much should we standardize the refcat columns on input vs. apply processing when they're loading? Experience with the Gaia catalogs was that we tried to standardize them to be SDSS-like, but that made it eventually harder to use.
Agreement was that standardizing on refcat creation is a bad idea, instead should focus on analysis code that can handle heterogenous inputs.
Are catalogs provided via qserv in the RSP in-scope for this discussion? In-scope for the LIT ticket that triggered this, but focusing on commissioning needs right now. Will need a follow-up on the RSP catalogs.
Reconciling the DPDD with the required information for pipeline QA.
Will we provide an ObjectId in the Source table - need a clear statement on this.
Can we remove detailed schema information from the DPDD so it is no longer serving as source of truth (painful after RFC-807, for instance). Point instead to https://dm.lsst.org/sdm_schemas/browser/and/or LDM-153.
DM-22078-Reconciling the DPDD with the required information for pipeline QA.IN PROGRESS
ObjectID in the Source table:
We want an ObjectId in the Source table but we can't guarantee that every Source will have a (good) match. Provide a set of flags that tell the quality of the match.
Can Sources match to Solar System Objects. Yes, must be possible. (Then we definitely cannot partition by that match)
Need a follow-up to address the Solar System topic more thoroughly.
Ran out of time, remaining topics deferred to a later SST meeting.
Slide 16, the P2 and P3 dates seem stronger than the definitions of P2 and P3 (in that they could be deferred much later than the dates in the bullet point).
Could we make one of the auxtel runs one of the ops rehearsals? Sounds like yes.
Integration milestones have explicit prerequisites, but haven't written those down for ops rehearsals. Robert would like the ops rehearsals to only use components that have already been demonstrated/verified by construction.
Need to add the RSP's Level 2 milestones, LDM-503-RSPa and LDM-503-RSPb, and clarify whether the latter is a predecessor of "DM Ready for Science Verification" (as I've been assuming) or a peer of it.
Question of whether the construction requirements are sufficiently related to the milestones to make them verifiable as part of claiming the milestone, or whether a separate acceptance campaign would provide a better connection to requirements.
Eric: sense that the upcoming milestones are large and vague, making it difficult to tie to development specifics.
We have many tickets open on special programs and a draft TN. Much of special programs can be left to ops but there are some topics that need to be resolved in construction. We could use this time to conclude on open topics
At PCW2021 this topic was raised. For some science cases (e.g., searching for Solar System objects where there are no templates) even severely paired down source catalogs (just ra, dec, psfMag) would be extremely useful .
DM-15307-Include Sources in Prompt Products DatabaseTO DO