int
/long
, PropertySet
, ExposureF
, WCS
, PSF
, set
/list
/dict
calexp
, src
, icSrc
FitsStorage
, SqlStorage
file:
, http:
, sqlite:
, mysql:
DataId
DataRef
DataId
packaged with a Butler
for access to datasetsDataRefSet
DataRef
sDataRef
s can be used with other dataset typesDataRef
s point to an existing input dataset at time of generationMapper
Butler
DataId
with additional key/value pairs (fixed and/or as-needed) needed to expand location templateButlerLocation
DataId
s to numeric identifiersDataId
key's valueIdStorage
Provides utilities for subclasses
Maintain templates for dataset types in repository configuration
Look up key/value pairs using equality or range joins in registry databases
Glob for key/value pairs in filesystem
Record metadata of new datasets in registries
Maintain registry of registries
ButlerLocation
DataId
Butler
Mapper
to translate DataId
into location__init__(outputRepo, inputRepos=None)
outputRepo
is a repository URL (string)inputRepos
is a map from role name (string) to repository URLget(self, datasetType, dataId={}, **kwArgs)
dataId
with keyword argument overridesput(self, obj, datasetType, dataId={}, **kwArgs)
obj
using dataId
with keyword argument overridesButler
(or actually either its Mapper
or a Storage
) is allowed to notice that the identical obj
has been persisted before and not persist it againobj
s, which can be persisted as a reference to the originalgetKeys(self, datasetType=None)
DataId
keys appropriate for datasetType
or all keys known for output repositorygetDatasetTypes(self)
createDatasetType(self, datasetType, datasetClass, pathTemplate, **kwArgs)
datasetType
based on the datasetClass
using the provided path template and keyword argumentsgetRefSet(self, datasetType, partialDataId={}, **kwArgs)
DataRefSet
enumerating all existing datasets of datasetType
using partialDataId
with keyword argument overridesdefineAlias(alias, datasetType)
@alias
" with this Butler
becomes equivalent to datasetType
ButlerFactory
is gone.
getDatasetTypes
and createDatasetType
are passed through from the Mapper
.subset
is renamed to getRefSet
to be more descriptive; its functionality subsumes the old queryMetadata
.datasetExists
is gone, since getRefSet
only returns datasets that exist.level
arguments have been removed, as the concept turned out to be useless in practice.put
can handle duplicates (for configurations and provenance or for sharing objects).Task
s to handle, e.g., any "src
"-like dataset.Interface used by Butler
:
__init__(self, repo)
repo
is an output repositorymap(self, datasetType, dataId)
ButlerLocation
corresponding to the dataId
for the given datasetType
getKeys(self, datasetType)
DataId
keys appropriate for datasetType
or all keys known for output repositorygetDatasetTypes(self)
createDatasetType(self, datasetType, datasetClass, **kwArgs)
datasetType
based on the datasetClass
using the keyword argumentslistDatasets(self, datasetType, partialDataId={}, **kwArgs)
DataId
s enumerating all existing datasets of datasetType
using partialDataId
with keyword argument overridescanStandardize(self, datasetType)
True
if the datasetType
can be standardizedstandardize(self, obj, datasetType, dataId)
obj
, given its datasetType
and dataId
Interface for subclasses:
createDatasetType
has been added.listDatasets
replaces the old queryMetadata
.validate
was never used and is gone. (It was originally supposed to do something like CameraMapper
's Mapping
's need()
.)IdStorage
.