Unknown User (pdomagala) make a specific recommendation for the fields to use in these tickets (i.e., component, label, etc.) to ensure that they are seen by the right person and to facilitate summary queries
HSC PD1 (February public data release) has been loaded to /datasets/hsc/repo - see
DM-9683
-
Getting issue details...STATUS
,
RFC-297
-
Getting issue details...STATUS
Still finalizing exactly what to run. Brief discussion of what to do about meas_mosaic. John Swinbank: it should not be run at this time. Later in the cycle we are likely to run a variety of tests of jointcal and meas_mosaic, and this will be looked at as part of QA work on the pipelines.
Planning to start run from latest weekly build but then hold version constant, if possible, for the whole dataset
Interested in learning more about the plans for data (file) storage, whether object stores will be used, how metadata queries can be turned into DataIDs usable in the Butler.
Relevant to some processing cases that don't obviously fit into the current DM pipeline paradigm: MOPS and alert distribution
Donald Petravick: current picture is that there need to be three databases involved in this area, enabling
Finding data by provenance
Finding data by metadata describing the file
Mapping data identity to data location (so that the "on-disk" arrangement of data can be changed)
Donald Petravick: working with Kian-Tat Lim to develop a process for working out requirements, initially, and then design, for provisioning for storage (as well as for computing); will present an introduction at Monday DMLT
Donald Petravick Make presentation at DMLT meeting on the process for defining requirements
Gregory Dubois-Felsmann: the question of whether the "all-powerful Butler" model for data access works for all parts of AP, including the ones Simon Krughoff mentioned, is likely to come up soon in the SuperTask/Butler working group as well.
Seems like some specific Architecture Group attention to how MOPS and alert distribution will be structured may be needed.
Thanks to everyone for getting PDAC to the point where it was possible to demonstrate at the JTM. Everyone's work is very much appreciated.
Will be discussing medium-term plans for PDAC at Monday's DMLT. Not ready to discuss in any detail today. Next week's PDAC meeting should be a good venue.
WISE data partially loaded - Object-like and ForcedSource-like data loaded by Igor Gaponenko and various Qserv issues that were exposed have been solved. Waiting for information from IPAC on how to access table column metadata.