...
- need to plan epics / metaepics
- writing images from AP ... local disk? shared fs? object store? Prefer shared fs over object store. Probably don't want to serve images to users from the location where AP writes. So writing to local disk might be best
- when images saved, need to save provenance. It is either via butler "put", or separate step. Better via butler, it is more atomic that way
- in butler put, it should be possible to turn off writing metadata, we have a usecase where someone might want to write just images say on a laptop
- need to understand how to hook provenance to butler
- butler probably does not need to add to provenance, hopefully not
- querying provenance: preexisting "query templates", user fills bits and pieces, we expand it to internal queries that know how to query provenance. That can be implemented through plugin to butler
usecase: we push alerts, but fail to push images and/or catalog data. Later we reprocess that thing that failed. Obey observing order, also, have to remember processing order! So, provenance needs to keep track of processing order for AP (this is not the case for DRP, as Jim Bosch mentioned in the thread on community.lsst.org)
- archive manager
- access via buter, webserv, ingestion daemon that ingests data from pipeline. Maybe exposed to users for ingesting images
- also, see picture of whiteboard related to that discussion, attached to this page:
