RFC on lossy-compressed PVIs (successor to RFC-325) 1) DMSR New subsection "Lossy-Compressed Visit Images" DMS-REQ-xxx1, Produce Lossy-Compressed Visit Images (Level 1), Priority 2 (functional requirement) Specification: The DMS shall produce and archive lossy-compressed versions of the Level 1 Processed Visit Image data products in time to replace all original XXXs before they are removed from the YYY cache. Discussion: As an implementation choice, this requirement may be satisfied by producing the lossy-compressed PVIs in the Prompt production system along with the original PVIs, or later, just in time for the cache expiration. The availability of lossy-compressed PVIs does not relieve the DMS of the requirement to provide software and services to re-create the original PVIs on-demand, but it is expected to greatly reduce the level of demand for those services. The compression algorithm will be chosen to preserve the scientific usefulness of the images for both visual inspection and algorithmic processing, with only modest degradation of quantitative performance. DMS-REQ-xxx2, Produce Lossy-Compressed Visit Images (Data Releases), Priority 2 (functional requirement) Specification: The DMS shall produce and archive lossy-compressed versions of the Processed Visit Images in each Data Release. DMS-REQ-xxx3, Retention of Lossy-Compressed Visit Images, Priority 2 (functional requirement) Specification: The DMS shall retain the Level 1 lossy-compressed Processed Visit Images on low-latency services at least until they are superseded by Processed Visit Images or lossy-compressed Processed Visit Images from a subsequent Data Release. DMS-REQ-xxx4, Service of Lossy-Compressed Visit Images, Priority 1b (functional requirement) Specification: The DMS shall make lossy-compressed Visit Images available to users through the same image metadata and image data services, and in the same formats, as the original PVIs. Discussion: Lossy-compressed PVIs will be available through IVOA-conformant ObsTAP and SIAv2 image metadata services, and available for image cutout creation from corresponding SODA services. They will also be available via the Butler in the Notebook Aspect. They will, therefore, be available for identification, access, visualization, and analysis through all Aspects of the Science Platform. This requirement is higher-priority than the others because it affects the design of the image services, so this capability needs to be in place earlier in order to avoid a possible change in interface when the actual lossy-compressed images become available. DMS-REQ-xxx5, Metadata Service for Expired Processed Visit Images, Priority 2 (functional requirement) Specification: The DMS shall ensure that metadata records are retained and available for service to users for Processed Visit Images which are deleted from the cache specified in DMS-REQ-0310, including metadata facilitating re-creation of the PVI as well as informing the user of the availability of a lossy-compressed alternative. Discussion: This requirement is expected to be satisfied in the API and Portal Aspect by means of the content of the DataLink "links service". It also implies a capability in the Butler to retain records of expired images. DMS-REQ-xxx6, Usability of Lossy-Compressed Visit Images, Priority 1b (functional requirement) Specification: The DMS shall support the use of lossy-compressed Processed Visit Images in all processing contexts where the original PVIs may be used. Discussion: Lossy-compressed PVIs are not meant only for visual inspection, but may be used for photometry, image differencing, custom coaddition, and other tasks, as long as the user is aware of the data quality penalties that may be incurred. This requirement implies that lossy-compressed PVIs must include mask planes, variance planes, PSF models, and so on. The use of lossy-compressed images may result in a modest degradation in performance, consistent with the quality specification in DMS-REQ-xxx7. This requirement is higher-priority than the others because it affects the design of the compression mechanism, and verification of the usability of compressed images should be demonstrated prior to their actual production. DMS-REQ-xxx7, Quality of Lossy-Compressed Visit Images, Priority 2 (performance requirement) Specification: The compression algorithm for lossy-compressed Processed Visit Images shall be chosen to maintain the following performance for isolated point-like sources: single-epoch detection limit reduced by no more than X; flux measurement uncertainty for sources with SNR>=20 worsened by no more than Y; and position resolution for sources with SNR>=20 worsened by no more than Z. DMS-REQ-xxx8, Format of Lossy-Compressed Visit Images, Priority 1b (functional requirement) Specification: The DMS shall produce lossy-compressed Processed Visit Images using a compression mechanism compatible with the FITS 4.00 standard. DMS-REQ-xxx9, Re-creation of Expired Processed Visit Images, Priority 2 (functional requirement) Specification: The DMS shall provide software and services for the re-creation of expired (non-lossy-compressed) PVIs on demand, for at least the most recent two Data Releases and for images acquired since the cutoff date of the most recent Data Release. Discussion: No specific performance requirement on this re-creation is specified; the operations organization will determine what performance is realistically possible to offer given the available resources.