02C.06.01.01 Catalogs, Alerts and Metadata (Schema and Structure)

02C.06.01.02 Image and File Archive

02C.06.02.01 Data Access Client Framework (Butler)

Request fromRelevant issuesTarget dateSummary

DLP-496 - Getting issue details... STATUS DM-2404 - Getting issue details... STATUS

DLP-21 - Getting issue details... STATUS

W16 (requested; see summary)

DRP is prioritizing a re-write of the top-level pipeline tasks. This is a priority as we are incurring technical debt in terms of inconsistent interfaces, configurations and outputs as we merge from HSC through S15. We would like to take this opportunity to update these tasks to use the "new" Butler interfaces, rather than performing a second round of rewriting when that becomes available.

Note that work is ongoing on the Butler through W16, resulting in a deliverable scheduled for S16.

DRP DLP-496 - Getting issue details... STATUS W16Again in support of the pipeline tasks re-write we request the capability to perform spatial lookups in the Butler (e.g. find CCDs which overlap a particular patch/tract).
DRP DLP-483 - Getting issue details... STATUS W17

To support multi-coadd deblending, we'll need moderately efficient access to object postage stamps over several coadds (~20). We expect to not be able to process full images for a patch due to memory constraints, so we're planning to do this in groups of nearby objects. We can supply boundaries that are irregular, and consider it an implementation detail whether we get rectangular regions or just the pixels we requested. Many of the coadds we'll be requesting postage stamps from will not be kept long term, and it may be useful to only save them to local scratch rather than a distributed filesystem.

At this stage, the algorithmic details of the processing are somewhat uncertain, and we do not require I/O performance at the level eventually needed for production. By DLP-483 - Getting issue details... STATUS (tentatively scheduled for W18), we expect to have those algorithmic questions answered at the level necessary for the Data Access team to start building a production-ready service for this requirement.

02C.06.02.02 Web Services



Relevant issues




DLP-564 - Getting issue details... STATUS

S15SUI group has been working with database group in defining the data access APIs. We would like to be able to exercise some APIs to get some data
SUI DLP-565 - Getting issue details... STATUS S16SUI should be able to use all the data access APIs to get catalog and images

02C.06.02.03 Query Services

Request fromRelevant issues



AP DLP-268 - Getting issue details... STATUS S17For association AP needs to query Level 2 and Level 1 databases. This is a request to have collaborate on the APIs so that they can be stable by the point the association is slated to be implemented.

02C.06.02.04 Image and File Services

Request fromRelevant issues



AP DLP-271 - Getting issue details... STATUS S19The current plan is to include images in the alert packets. Even if images are not ultimately embedded in the alert packet itself, AP will still need access to the image service to include URLs.

02C.06.02.05 Catalog Services

Other or Unknown



  • No labels

1 Comment

  1. Simon Krughoff, the Level 1 will NOT be served through Qserv. (we can talk about it if you'd like details)