THIS PAGE IS A WORK IN PROGRESS.  PLEASE REFRAIN FROM EDITING, COMMENTING ON, OR BELIEVING ANY OF THIS UNTIL IT IS RELEASED.

In the next major release of the LSST stack, the system for measuring the properties of sources will be replaced by a new one, housed mostly in the new meas_base package.  In addition to providing new Task classes and a new plugin interface for measurement algorithms, this overhaul also includes changes to the schemas of the catalogs produced by the measurement framework.  The new framework will not be entirely complete as of the next release, but it will have sufficient functionality to replace the old framework as the default.  The old system will remain available via config on the S14 final release, but will be removed entirely before the W15 release.

New Tasks

The main entry point for the new measurement framework is the new SingleFrameMeasurementTask class (in meas_base), which is intended as an almost drop-in replacement for the current SourceMeasurementTask (in meas_algorithms).  It's not drop-in replacement, because it has different plugins, a slightly different slot system, and an entirely different output schema - so we will use it everywhere SourceMeasurementTask was used before, but many other additional configuration settings need to be modified when retargeting the measurement task to keep it all working.  We've provided configuration files to make that easy, as described in the next section.

SingleFrameMeasurementTask combines the work previously done by two classes, the old SourceMeasurementTask and the C++ MeasureSources class.  It initializes the plugins (which determines the schema) in its constructor, then invokes the plugins on each source in the image, replacing neighbors with noise as it does so (this is delegated to the NoiseReplacer class, which replaces the ReplaceWithNoiseTask subtask).  Unlike SourceMeasurementTask, SingleFrameMeasurementTask does not do forced photometry (see below).  Sources are also processed in a slightly different order (see Simultaneous Multi-Object Measurement).

Reverting to the Old Framework

TODO

Schema Changes and Versioning

The changes to afw::table's Schema objects to support the new measurement framework are handled by a new versioning mechanism, which allows the old behavior (necessary for the old framework) to coexist with the new behavior in the stack.  A Schema with version=0 corresponds to one appropriate for use with the old SourceMeasurementTask, while version>0 corresponds to SingleFrameMeasurementTask and the new forced photometry tasks.  Here are the differences between versions:

• In version 0, periods in field names were translated to underscores when writing to FITS, making it impossible to use underscores in field names.  This was due to a misunderstanding of the FITS standard, and was never necessary.  This translation is not done for version>0 schemas, and hence both periods and underscores will work with the code.
• In version>0, our naming conventions use underscores instead of the periods used in version 0.  So while both are permitted by the code in version>0 (which allows us to read external FITS files with fewer restrictions), only underscores should be used in LSST code, and underscores will now be used as the delimiter by the SubSchema class and Schema::operator[] operator when joining pieces of field names.

In version>0, we have stronger naming convention for fields generating by plugin algorithms, which tie the plugin name and location to its outputs.  These conventions are used by the 

• Names should start with "<package-abbr>_<class>", where "<package-abbr>" is an abbreviated version of the package name that provides enough information to specify the package location.  For instance, plugins in meas_base use "base" as their prefix, while those in ip_diffim would use simply "diffim".  The "class" part of the name is simply the name of the class that defines the algorithm, minus any explict "Plugin" or "Algorithm" suffix.  For instance, the field name prefix used by meas_base's SdssShapeAlgorithm class would be "base_SdssShape".
• Flux measurements should end with "_flux", even if that's repetitive with the name of the class (e.g. "base_PsfFlux_flux").
• Position measurements should consist of two fields, ending in "_x" and "_y".
• Ellipse measurements should be saved using the "Quadrupole" parametrization (even if they aren't measured as moments), using fields ending with "_xx", "_yy", and "_xy".
• Uncertainties on individual fields should be saved with names that end in "_<p>Sigma", where "<p>" is the suffix of the field the uncertainty corresponds to (e.g. "_fluxSigma" or "_xSigma").
• Uncertainty covariances should be saved with names that end in "_<p1>_<p2>_Cov", such as "_x_y_Cov" for the uncertainty on "_x" and "_y".
• Flag fields should have names like "_flag_<reason>", where "reason" is a camelCase string identifying what went wrong.  A special flag that is simply the algorithm prefix + "_flag" is additionally set for any fatal error (but this will likely be changed on prior to the S14 release).

Aliases and Slots

We have added an alias feature to the Schema class, via an AliasMap object it holds.  Aliases are handled simply as an extra stage in Key lookup - when a Key is requested via a string, its beginning is compared against all aliases to see if it should be replaced.  Partial replacements are allowed, but only at the beginning, and multiple replacements are not.  For example, if the AliasMap includes the mapping "slot_PsfFlux"->"base_PsfFlux", then a Key lookup on "slot_PsfFlux_flux" will resolve to "base_PsfFlux_flux", but a lookup on "r_slot_PsfFlux_flux" will not be affected.  Because Key lookup happens implicitly when getting a value from a record via string, this makes it even more important to do Key lookup once in advance rather than use a string to access the same value repeatedly.

The alias system is available for both version 0 and version > 0 tables, and is now used to define the slot mechanism in SourceTable/SourceRecord/SourceCatalog in both cases; each slot corresponds to a predefined field name prefix that is mapped to an actual measurement via an alias.  These predefined names are the name of the slot, starting with a capital letter (e.g. "PsfFlux", "Centroid"), prefixed with "slot_" (or "slot." for version 0).  For instance, a typical AliasMap containing slot definitions would have the following mappings for version 1:

• slot_PsfFlux -> base_PsfFlux
• slot_ApFlux -> base_SincFlux
• slot_Centroid -> base_SdssCentroid
• slot_Shape -> base_SdssShape

Slot definitions are now persisted simply as aliases, with slots saved via older versions of the stack translated into aliases when read from disk.  Changing one of these aliases in an AliasMap attached to a SourceTable will notify the table that slots have changed and the cached Keys that correspond to them must be updated (in fact, the old slot definers, such as SourceTable::definePsfFlux() are now implemented by simply changing the alias and letting the notification callback do its work).

Note that this means that slot values can now be accessed via string, just like any normal measurement.

Simultaneous Multi-Object Measurement

TODO

Forced Photometry

TODO

Python and C++ Measurement Plugins

TODO

Error-Handling and Diagnostics

TODO

Missing Features and Known Problems

For issues that we expect to complete before the S14 release, please see the following epics:   These include improvements to error-handling, miscellaneous code cleanups, FunctorKeys for array fields, and a multiple-aperture photometry plugin.

For issues that we expect to complete in W15, please see: .  These include reimplementing aperture corrections, reimplemented database ingest, converting extension algorithms to the new plugin system, and removing the old measurement framework and all remaining code that depends on it.