WBSTitleDescriptioncontent for LDM-294
02C.05Science User Interface and Tools

This WBS element is a summary element that contains the software, tools, documentation, and user interfaces implementing the Science User Interface and Tools. The definition of the elements below give more details.

This WBS element covers the work performed by the Science User Interface and Tools group. It covers the following areas:

  • The Firefly framework to support the infrastructure and common functionalities for all applications, including low-level Python API and JavaScript API;

  • Components to provide connection between the core data search/visualization components and the rest of the "LSST Science Platform";

  • The web portal aspect of LSST Science Platform, to enable the science user community to access, discover, explore, analyze, and download LSST data;

  • A web-based user interface to alert subscription and searches.



02C.05.00
  • Management, Leadership, and other LOE

The management work performed by the T/CAM and the lead scientist, and the necessary LOE work performed by developers and scientists are captured in this WBS element.

This WBS element covers Project Management, Control Account Management, and Scientific Leadership of the Science User Interface and Tools group. It includes effort to develop the overall schedule and cost for the work, perform day-to-day technical management of the team, coordinate with other parts of the WBS, provide support for invoices, contribute to the operation of the DM Subsystem Science Team, and LOE (for meetings ...) for team members. No products are defined at this level of the WBS. 

02C.05.06
  • Client-server query and visualization framework
Firefly framework to support the infrastructure and common functionalities for all applications.

This WBS element covers the construction of low-level, re-usable software components which form the core libraries (Firefly) underlying both the client and server sides of the LSST Science User Interface Portal. It includes software packaging, releases, and deployment, the production of test suites demonstrating the correct operation of the components, and the development of technical, developer-focused documentation describing their use. 

Firefly comprises a variety of capabilities, including the following:

  • Libraries for data display and visualization for tabular data and astronomical images, including various 2D charts, and a shared data model supporting data overlays on images, and brushing and linking among related displays.
  • Abstract search processor interface
  • Libraries for data query, retrieval, and export, including an abstract search processor interface as well as implementations providing common astronomical archive search forms and interfaces to standard (including VO) data query APIs 
  • Support for various common formats for astronomical tabular and image data
  • JavaScript and low-level Python APIs, used internally as well as providing for user control and customization
  • Identity and preferences management
  • Load balancing
02C.05.06.01

    • Infrastructure

  • JavaScript API
  • Lower-level Python API
  • Support ability to launch back-end processes, e.g., in Python
  • Identity and preferences management
  • Load balancing
  • Firefly packaging and releases

02C.05.06.02
    • Data query, retrieval, and export
  • Abstract search processor interfaces
  • Common search processors and forms
    • VO searches
    • Searches on well-known archives
    • Schema-driven "generic table" query form
  • Common tabular data file format support (for input and output)
  • Common image data file format support
  • Retrieval of data URIs

02C.05.06.03

    • Data display and visualization

  • Astronomical image display
    • WCS handling
    • Mask overlays
    • Multi-resolution images
    • Layering of images and graphics
    • Time series movies and image blinking
    • Image region selection and interaction
    • HEALPix data visualization, at different resolution levels, for all-sky image
  • Tabular data handling
    • Shared data model supporting linking of data across multiple plots
      • Related-column semantics
    • Display of tabular data as tables
      • Numeric formatting
      • Sorting
      • Filtering UI
    • Table manipulation:
      • Computed columns
      • Creation of 1D and 2D histograms, contour models, heatmap data
      • Statistics
      • Complex filtering (e.g., filtering on functions of multiple column values)
  • "Making pixels from data": generation of 2D and 3D graphics, organized by chart type
      • 2D scatter plots (x-y plot)
      • Symbols Types, Sizes, and Colors as representation of quantities
      • Line plots
      • Vector field plots
      • Contour plots
      • Density plots and mixed density-scatter plots
      • Surface plot
      • Heatmap plots
      • Time series plots
      • Histogram display (not histogram creation/accumulation)
      • 3D graphics (to be expanded)
      • Drawing tools / shape display
        • Includes DS9 region drawing
      • Plot region selection and interaction

02C.05.07
  • LSST science platform interfaces

Components that provide the connection between the core data search/visualization components and the rest of the "LSST DataSpace": the LSST stack, data model, data-access services, and compute and storage resources

This WBS element covers the construction of software components that provide the connection between the core data search/visualization components and the rest of the "LSST Science Platform": supporting the LSST data model, data-access services, and compute and storage resource access. It includes the development of associated documentation and test suites.

It incorporates the following:

  • Interface to LSST-specific user identity and management services
  • Access to LSST user workspace
  • Search processors matching the DAX interfaces
  • Search processors for the Engineering and Facilities Database
  • Interface for invoking LSST stack Python code to perform services needed by the Portal
  • Support for reading and displaying LSST-specific data formats (such as afw.table-format files) and data objects (e.g., masks, Footprints, PSF models)
  • Support for the afw.display interface to Firefly

02C.05.07.01
    • Infrastructure

  • Ability to launch back-end processes, e.g., in Python, including pipeline stack and user code, using LSST interfaces to computing resources

  • Interface to LSST-specific user identity and management services

  • Firefly back-end SuperTask activator

02C.05.07.02
    • Interfaces for LSST data services and storages
  • DAX search processors
  • Search processors for the Engineering and Facilities Database
  • Butler-based search processor framework in Python
  • Interface to next-to-the-database computation capabilities

02C.05.07.03
    • Visualizations and interpretation of LSST stack data
    • Pipeline visualization support

  • afw.display interface
  • Interpretation of afw.table data
  • Interaction with LSST WCS models that are not natively supported in Firefly
  • LSST-specific visualizations - examples:
    • PSF at a point, PSF model variation
    • Footprint, HeavyFootprint
    • LSST image masks (based on general mask overlay capability from 02C.05.02.03)
    • Galaxy model fits

02C.05.07.04
    • Portal-notebook-Python connections
  • Creation of notebooks from portal sessions
  • Two-way access between Python and Firefly data models
  • Jupyter widget interfaces to visualization components
  • Interface to obtaining compute resources for back-end computations

02C.05.08
  • Applications

Full deployments of SUIT components organized into user-facing services. Some are created and written by SUIT, others are supported by SUIT. Also includes specific science algorithm code required to implement one or more of the applications.

This WBS element covers the construction of the Portal Aspect of the LSST Science Platform software, including online help, deployment instructions, and other documents. The Portal uses the core Firefly components and the LSST-specific software interfaces in 02C.05.07, relying on infrastructure and data access services provided under other WBS elements, to deliver a portal for the science community to access, discover, explore, analyze, and download the LSST data. It covers the following:

  • The overall user interface layout and structure
  • Basic access to all LSST catalog and image data (Level 1, Level 2, user/Level 3, calibration, and Engineering and Facilities Database)
  • Scientifically motivated workflows guiding users to the available data and illuminating connections among tables and between tables and images
  • All-sky displays allowing exploration of the image data and of maps of the properties of the LSST survey across the sky
  • Access to the user workspace, supporting data sharing and and collaboration
  • Deployment packaging and configuration management
02C.05.08.01
    • Science algorithms support
  • Cross-matching and cross-identification tools
  • Time series manipulations, e.g., phase-folding
  • Periodogram calculation using public service

  • Solar System object orbit calculations


02C.05.08.02
    • LSST user-facing science data portals
  • Support for login and access to user management
  • Multiple versions appear over time:
    • PDAC, multiple versions with increasing capability
    • Commissioning data portal
    • Pre-DR1 portal (Level 1 data access only)
    • Full post-DR1 portal

  • Special portals with specific science emphases


02C.05.09
  • Alert interfaces
  • Support and access to user management
  • Subscription service
  • Filtering service
  • myAlert service (e.g., saving, annotation, linking) (To be discussed with other DM teams)

This WBS element covers the construction of a Web-based user interface for alert subscription and searches. It is based on the core Firefly package and uses interfaces provided by SUIT, DAX, and AP. It covers:

  • Alert subscription, setting filters and alert stream destination
  • Access to user management system
  • Alert searches and filtering
02C.05.10
  • Integration and test
  • UI manual test
  • Explore/setup the automated UI test
  • SUIT integration and test of all the applications, including portals, QA system, Jupyter NB, integrated science platform
This WBS element covers the integration and higher-level testing of all the deliverables from the 02C.05 sub-WBSs, including manual UI tests, tracking and reporting bugs in the software from other teams, possible automated UI test setup and instructions, and deployment of the software at the LSST Data Access Centers in collaboration with NCSA.
NEW work requiring BCRs


02C.05.11
  • LSST Camera I&T visualization support
  • Multi-panel wall display
  • Multiple WCSs support, e.g. expressing focal plane hierarchy
  • Over-scan representation and display
  • Special control functions needed for the multi-panel wall display

02C.05.12
  • (TBC) LSST Control Room image visualization (on behalf of Telescope & Site)
  • Still requires scope assessment and a change request
  • May extend to provide time-series visualization of telemetry integrated with the image visualization

02C.05.13
  • (TBC) EPO support
  • Support EPO activities in using Firefly

02C.05.14
  • (TBC) Visualization support for quality control and analysis tools
  • HEALPix data visualization
  • "QA Drill-down/Analysis portals Part I" corresponding to the "Part II" that is in the proposed 02C.10 WBS
  • Support for the use of SUIT components in the QC dashboards

  • No labels