Warning |
---|
|
This page is frozen for the v13 release. Please do not edit! |
These draft notes cover the major updates made by Data Release Production to the LSST stack since release 12 (Winter/Extra 2016). Please add material here which should ultimately be incorporated into the notes accompanying the Fall 2016 release.
...
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-5429 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6123 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6124 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6125 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-2253 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-5867 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-5432 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6719 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6724 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6300 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6398 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6788 |
---|
|
...
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-4926 |
---|
|
Added ability to run ctrl_pool processes without a batch system
The option --batch-type none
(or None
) runs the command in your process as normal. This means that e.g. the pipe_drivers calibration scripts need not rely on MPI.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7900 |
---|
|
Bug Fixes
Corrected configuration dictionary comparisons
Resolved a bug whereby comparison of (intrinsically unordered) configuration dictionaries incorrectly compared their ordering rather than just their contents. This lead to unexpected failure of re-runs due to incorrectly identified “mismatched configurations”.
Jira |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6661 |
---|
|
Correct CModel table schema units
HSC R2 filter
The obs_subaru package now includes support for the R2 filter on Hyper Suprime-CamUnits for the time field changed from seconds to second to comply with the Astropy convention.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6282 |
---|
|
Restore capability of calculating temporary local backgrounds in source detection
Added DirectMatchTask
The new DirectMatchTask
matches sources against a reference catalog without performing any rotation, offset or WCS fitting. This provides a simpler and less error-prone option for matching sources for quality assessment. The MeasureMergedCoaddSourcesTask
has been modified to make use of this new DirectMatchTask
Our SourceDetectionTask
provides the optional capability of fitting and temporarily removing a local background before detecting sources. This capability was accidentally disabled in the X/W16 release (v12); it has now been restored. It is disabled by default.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-5323 |
---|
|
Resolve a memory error in convolution
Added --show history=config.foo.bar
You can now specify a command line argument –show history=config.foo.bar
to see where configuration parameter foo.bar
is set and over-ridden (the leading config.
is optional). The output may not be very intuitive, but it is what pex_config
provides and is sometimes just what you need. N.B. pipe to cat
to turn of colourization (or investigate lsst.pex.config.history.Color
)A memory handling error could cause our convolution code to generate incorrect results on some platforms (problems observed on OS X when the code is built without optimization; no errors seen on Linux, or when building with -O2
or -O3
on OS X). This appears to originate in the Boost Generic Image Library, an external dependency of the LSST stack. We have been unable to isolate the problem in Boost, but have worked around it for our particular use case.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-5822 |
---|
|
Patched GalSim to raise an exception in some error conditions
On certain error conditions, GalSim, a third party package used by the LSST stack, generates an assertion failure which is hard for our code to handle appropriately. LSST's version of GalSim has been patched to raise a more appropriate exception instead. This change has been submitted for inclusion in GalSim itself.
Jira |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7114 |
---|
|
Ensure star selector results are reproducible
frame
defaults to None
when creating a new Display
When using the afw.display
system to create a new Display
object, the frame
argument is now optional: it will default to None
if not specifiedResolved an issue whereby the star selection could change depending on the number of CPU cores in use.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7040 |
---|
|
Single frame measurement exploits multiple CPU cores
Resolved an issue whereby the SingleFrameDriverTask
failed to use more than one CPU core regardless of configuration.
Jira |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7134 |
---|
|
Fix typo in CoaddSrcTransformTask
Attribute access to Display objects is passed through to the underlying implementation
Some afw.display
device types may provide implementation-specific functionality. This may now be accessed directly through the Display
object, which will delegate to implementation-specific methods, if they exist, when a non-standard method is called on the Display
A typo which rendered CoaddSrcTransformTask inoperable was resolved. In the process, the tests for this task were significantly upgraded to avoid a repeat of this, or other, problems.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DMRFC-6635 |
---|
|
When persisting objects to FITS, we were incorrectly writing duplicate AR_NAME
headers. This has been resolved.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7221 |
---|
|
Be more rigorous about units when persisting angles
We now correctly write TUNIT
headers describing the units used when storing angles, and are more careful about checking units when reading them.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7221 |
---|
|
Infinite recursion in afw table
New fitter for astrometric distortions.
A new fitter task, meas.astrom.FitSipDistortionTask
has been added that will eventually replace the current fitter, meas.astrom.FitTanSipWcsTask
. The new fitter should be more robust and certainly uses a better-motivated algorithm, but it has not yet received enough testing to be used as the default and hence must be enabled explicitly. When running processCcd.py
, this can be done with the following config file:
Code Block |
---|
|
from lsst.meas.astrom import FitSipDistortionTask
config.charImage.astrometry.wcsFitter.retarget(FitSipDistortionTask)
config.calibrate.astrometry.wcsFitter.retarget(FitSipDistortionTask) |
The new fitter is not compatible with meas.astrom.ANetAstrometryTask
A bug has been discovered in the afw table library which could lead to infinite recursion. While the ultimate cause of the bug is still under investigation, a temporary workaround has been put in place to mitigate its impact.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-73773549 |
---|
|
Unified exposure metadata
Exposures now contain a new VisitInfo object that contains useful information about the overall visit (none of it specific to the individual CCD), including the exposure time, date and ID, boresight position and airmass. See Unified exposure metadata for more information.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DMRFC-7385 |
---|
|
Fixed a division by zero error in measurement of aperture corrections
Attempting to calculate an aperture correction for a measurement of zero flux would previously result in an unhandled division by zero.
Jira |
---|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7180 |
---|
|
Fix dataset lookup in multibandDriver.py
...
Convenient access to catalog metadata
It is now possible to read the metadata, length and schema of a catalog through a Butler interface without loading the whole catalog. For example, given a particular data ID (dataID
) and catalog dataset type (someCatalog
), one can run:
Code Block |
---|
|
butler.get("someCatalog_schema", dataId)
butler.get("someCatalog_len", dataId)
butler.get("someCatalog_md", dataId) |
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7716 |
---|
|
Build & Code Improvements
Migration from Boost to standard library equivalents
Continuing work started in the previous release, we have audited the stack for use of the Boost libraries. Where possible, Boost has been replaced with C++11 standard library equivalents.
Jira |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-5880 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6089 |
lsst.afw.image.readMetadata
now returns a PropertyList
, which maintains the intrinsic ordering of the header keywords in the FITS file it is reading. Previous versions used a PropertySet
, which is unordered and hence lost this information.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM- |
---|
|
6090
Measurement without deblending
The deblender sometimes runs into trouble with cluster galaxies, leading to inaccurate fluxes. To mitigate, we can now provide measurements performed without any deblending having taken place. To activate this functionality, specify the measurement plugins to be run without deblending in the undeblended
registry attached to the measurement class (this is analogous to the plugins
registry used when performing measurement on deblended sources). By default, no undeblended measurement is performed. This functionality is available in both forced and single-frame measurement.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6091 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-60926785 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6093 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6094 |
---|
|
Warp then match when building PSF-matched coadds
WarpAndPsfMatchTask has been updated so that, by default, it will first warp inputs to the coadd projection and then perform PSF matching; this is the reverse of its previous behaviour. The new behaviour is preferable, but a configuration option makes it posisble to revert to the old method on request.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-60958088 |
---|
|
Reduce galaxy “shredding” when deblending
If galaxies have substructure, such as face-on spirals, the process of identifying peaks can “shred” the galaxy into many pieces. The templates of shredded galaxies are typically quite similiar because they represent the same galaxy. We try to identify these “degenerate” peaks by looking at the inner product (in pixel space) of pairs of templates. If they are nearly parallel, we only keep one of the peaks an reject the other. This behaviour is enabled by the removeDegenerateTemplates option to SourceDeblendConfig; by default, it is disabled.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee, |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6096 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution | serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM- |
---|
|
6097
Bug Fixes
Corrected configuration dictionary comparisons
Resolved a bug whereby comparison of (intrinsically unordered) configuration dictionaries incorrectly compared their ordering rather than just their contents. This lead to unexpected failure of re-runs due to incorrectly identified “mismatched configurations”.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6325 |
---|
|
Properly log warnings in ObjectSizeStarSelector
Correct CModel table schema units
Units for the time field changed from seconds to second to comply with the Astropy convention.Previously, non-fatal NumPy warnings of the form RuntimeWarning: invalid value encountered in less could appear on the console, but not be caught and logged anywhere. It was therefore easy to miss problems arising when running the algorithm. These warnings are now ingested into the LSST logging system, and are hence recorded properly.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-5428 |
---|
|
Unit tests for colorterm framework
A set of unit tests have been written which properly exercise the colorterm framework which was adapted from Hyper Suprime-Cam in the S15 cycle.
Restore capability of calculating temporary local backgrounds in source detection
Our SourceDetectionTask
provides the optional capability of fitting and temporarily removing a local background before detecting sources. This capability was accidentally disabled in the X/W16 release (v12); it has now been restored. It is disabled by default.
Jirajira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-2837 |
---|
|
FlagHandler accessible from Python
Resolve a memory error in convolution
A memory handling error could cause our convolution code to generate incorrect results on some platforms (problems observed on OS X when the code is built without optimization; no errors seen on Linux, or when building with -O2
or -O3
on OS X). This appears to originate in the Boost Generic Image Library, an external dependency of the LSST stack. We have been unable to isolate the problem in Boost, but have worked around it for our particular use caseThe FlagHandler system for efficiently and safely setting flags corresponding to measurement algorithm failures on source records is now accessible from Python. Previously, it was only available for use in C++ code.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-4009 |
---|
|
Exact image comparison utility code
Patched GalSim to raise an exception in some error conditions
On certain error conditions, GalSim, a third party package used by the LSST stack, generates an assertion failure which is hard for our code to handle appropriately. LSST's version of GalSim has been patched to raise a more appropriate exception instead. This change has been submitted for inclusion in GalSim itselfThe assertImagesEqual
and assertMaskedImagesEqual
methods have been added to added to LSST's test cases. These provide explicit mechanisms for checking for exact equality of image data.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6641 |
---|
|
Add integration test for stellaricity of sources used to measure PSF
Ensure star selector results are reproducible
Resolved an issue whereby the star selection could change depending on the number of CPU cores in use.Added a test to the ci_hsc package to monitor the fraction of sources used to model the PSF in when processing a CCD image and ensure that the vast majority of them are ultimately classified as stars. This provides an automated check that we are not inadvertantly estimating the PSF based on extended objects.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6051 |
---|
|
Use CatalogCalculation to handle aperture corrections
Single frame measurement exploits multiple CPU cores
Resolved an issue whereby the SingleFrameDriverTask
failed to use more than one CPU core regardless of configuration.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7134 |
---|
|
Fix typo in CoaddSrcTransformTask
A typo which rendered CoaddSrcTransformTask inoperable was resolved. In the process, the tests for this task were significantly upgraded to avoid a repeat of this, or other, problems.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6635 |
---|
|
When persisting objects to FITS, we were incorrectly writing duplicate AR_NAME
headers. This has been resolved.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7221 |
---|
|
Be more rigorous about units when persisting angles
We now correctly write TUNIT
headers describing the units used when storing angles, and are more careful about checking units when reading them.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7221 |
---|
|
Infinite recursion in afw table
A bug has been discovered in the afw table library which could lead to infinite recursion. While the ultimate cause of the bug is still under investigation, a temporary workaround has been put in place to mitigate its impact.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7377 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7385 |
---|
|
Fixed a division by zero error in measurement of aperture corrections
Attempting to calculate an aperture correction for a measurement of zero flux would previously result in an unhandled division by zero.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7180 |
---|
|
Fix dataset lookup in multibandDriver.py
The multiband driver script was previously checking for the existence of the pre-calexp version of the coadd. This is not guaranteed to exist if the coadds were built using the driver scripts (where there is no need to persist both the pre-calexp and calexp versions) and it is only the calexp coadds that are used in the multiband tasks, so it is the calexp version whose existence should be checked for in multibandDriver.py.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7716 |
---|
|
Fix incorrect definition of mask when building fringe frames
The code in pipe_drivers to construct fringe frames, FringeTask
, was incorrectly using a mask plane when in fact a mask bit was required. This has now been corrected.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-8033 |
---|
|
Fix failures in MPI-based processing
Resolved an issue whereby the ctrl_pool distributed processing middleware was unable to transmit large volumes of data over MPI due to a limitation of the Python 2.x “pickle” system.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-8021 |
---|
|
Fix failure to retrieve optimizer state when fitting DoubleShapeletPsf
Resolved an issue whereby the optimizer state was being read incorrectly, which could lead to fitting failures.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6562 |
---|
|
Build & Code Improvements
Migration from Boost to standard library equivalents
Continuing work started in the previous release, we have audited the stack for use of the Boost libraries. Where possible, Boost has been replaced with C++11 standard library equivalents.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-5880 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6089 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6090 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6091 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6092 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6093 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6094 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6095 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6096 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6097 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6325 |
---|
|
Properly log warnings in ObjectSizeStarSelector
Previously, non-fatal NumPy warnings of the form RuntimeWarning: invalid value encountered in less could appear on the console, but not be caught and logged anywhere. It was therefore easy to miss problems arising when running the algorithm. These warnings are now ingested into the LSST logging system, and are hence recorded properly.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-5428 |
---|
|
Unit tests for colorterm framework
A set of unit tests have been written which properly exercise the colorterm framework which was adapted from Hyper Suprime-Cam in the S15 cycle.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-2837 |
---|
|
FlagHandler accessible from Python
The FlagHandler system for efficiently and safely setting flags corresponding to measurement algorithm failures on source records is now accessible from Python. Previously, it was only available for use in C++ code.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-4009 |
---|
|
Exact image comparison utility code
The assertImagesEqual
and assertMaskedImagesEqual
methods have been added to added to LSST's test cases. These provide explicit mechanisms for checking for exact equality of image data.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6641 |
---|
|
Add integration test for stellaricity of sources used to measure PSF
Added a test to the ci_hsc package to monitor the fraction of sources used to model the PSF in when processing a CCD image and ensure that the vast majority of them are ultimately classified as stars. This provides an automated check that we are not inadvertantly estimating the PSF based on extended objects.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6051 |
---|
|
Use CatalogCalculation to handle aperture corrections
The CatalogCalculation facility was added to the stack in the previous (v12) release (where it was referred to as "afterburners"); it provides a capability to run algorithms akin to our regular measurements, but which only take inputs from catalogs (i.e. not pixel data). This has now been used to clean up the logic used to apply aperture corrections within our regular data processing flow. This old method for handling aperture corrections was a persistent source of bugs and confusion: these should be avoided by the new approach.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-5877 |
---|
|
Builds are now optimized by default
Following an RFC, sconsUtils was updated to enable optimization by default.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | RFC-202 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7010 |
---|
|
Fixed a build failure of mpi4py on some versions of Mac OS X
Added extra logic to ensure we target a version of the OS X SDK which is compatible both with the version of Mac OS installed and the Python interpreter in use.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6133 |
---|
|
Centralised Butler datasets
Butler dataset definitions which are common to multiple cameras have been consolidated in a central location in daf_butlerUtils rather than being repeated in each camera packageThe CatalogCalculation facility was added to the stack in the previous (v12) release (where it was referred to as "afterburners"); it provides a capability to run algorithms akin to our regular measurements, but which only take inputs from catalogs (i.e. not pixel data). This has now been used to clean up the logic used to apply aperture corrections within our regular data processing flow. This old method for handling aperture corrections was a persistent source of bugs and confusion: these should be avoided by the new approach.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-5877 |
---|
|
Builds are now optimized by default
Following an RFC, sconsUtils was updated to enable optimization by default.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | RFC-202204 |
---|
|
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7010 |
---|
|
Fixed a build failure of mpi4py on some versions of Mac OS X
Added extra logic to ensure we target a version of the OS X SDK which is compatible both with the version of Mac OS installed and the Python interpreter in use. Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6133 |
---|
|
Centralised Butler datasets
Order in which SConscript files are read is configurable
The sconsUtils build system now reads SConscript
files, which describe the build, in a user configurable order. This makes it possible to specify that some components of a package must be built before othersButler dataset definitions which are common to multiple cameras have been consolidated in a central location in daf_butlerUtils rather than being repeated in each camera package.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-7179 |
---|
|
Ndarray is now an external package
The ndarray package in the LSST stack now installs vanilla upstream version 1.3.0.
Jira |
---|
server | JIRA |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-2005 |
---|
|
Jira |
---|
server | JIRA |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-6858 |
---|
|
Order in which SConscript files are read is configurable
SpherePoint now available
The SpherePoint class allows locations on the celestial (or any other) sphere to be represented and manipulated without the coordinate-system complications associated with CoordThe sconsUtils build system now reads SConscript
files, which describe the build, in a user configurable order. This makes it possible to specify that some components of a package must be built before others.
Jira |
---|
server | JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
---|
key | DM-71795529 |
---|
|