One component of the end-to-end testing of DM systems, beginning in 2016, is the establishment of an integration environment for the storage and service of astronomical datasets, using early versions of the components intended for the final systems, running on project-provided infrastructure.

The Fall 2016 version of this project handled only static test data and involved the integration of the (non-IVOA-based) prototype DAX services with an early version of the Firefly-based Portal Aspect.  This was known as the "Prototype Data Access Center" (PDAC) effort, and produced a viable Portal prototype that was evaluated (reported in DMTR-052).

In early 2018, a prototype of the JupyterLab-based Notebook Aspect became available, and was made available in several deployments, including one hosted on the new Kubernetes Commons (available at https://lsst-lspdev.ncsa.illinois.edu/ ) and others hosted on cloud services.

Starting in mid-2018 work commenced on integrating the Portal, Notebook, and API Aspect as foreseen in the LSP Vision.  In late 2018 the PDAC hardware is being merged into the Kubernetes Commons, following which we will maintain two full three-Aspect instances of the LSP ("PDAC" and "lspdev") on the Commons, with the PDAC instance continuing to be augmented by a large-scale QServ instance behind it.

The series of meetings and associated work under which the PDAC was integrated is now continuing under the "LSP Integration and PDAC" title (LSPIP).

In early 2019 the PDAC (Portal + API) and stand-alone Notebook environments were merged, leading to a new deployment model, closer to the anticipated final form, in which there is now a user-facing instance, https://lsst-lsp-stable.ncsa.illinois.edu/ , and an inward-facing system integration instance, https://lsst-lsp-int.ncsa.illinois.edu/ .

In 2019 the main remaining goals are:

• Integration of the three Aspects under a single-sign-on authentication & authorization system;
• Migration of the DAX APIs to IVOA-based interfaces and of the Portal to use these interfaces;
• Demonstration of Portal-Notebook-API integration that begins to meet the goals of the LSP Vision; 
• Development of a configuration control mechanism for the integrated LSP that supports the needs of multiple LSP instances; and
• Loading of recently-processed data from the Science Pipelines into the PDAC instance, including a demonstration of the "Data Model Standardization" / "DPDD-ification" of the pipeline outputs, as foreseen in the designs put together during 2018.

The development of the LSP as an integrated system is overseen by Gregory Dubois-Felsmann as product owner (Science Platform Scientist) and Frossie Economouas "meta-CAM", coordinating the activities of the four institutional groups developing the Aspects and their deployment environment.

See also the Science Platform page tree.

Pages on this topic: