Skip to end of metadata
Go to start of metadata

This is the page of the Solar System Processing team (Siegfried Eggl,Joachim MoeyensMario Juric), tasked with defining the delivering finished DM Products (i.e., software, documentation, operating procedures) needed to deliver the LSST Data Products to support the Solar System Science. It can be reached through the shortener via http://ls.st/ssp (mnemonic: solar system processing).

High-level Overview

Data ProductsPipelines

High level overview is at http://ls.st/doc-29545, screenshotted below:

Solar System Data Products

The overview can be found in the Data Products Definition Document.

The detailed schema is at https://docs.google.com/spreadsheets/d/1E0rTlvuJC0CvpLNsuWLK0x70uhpZww4v6GB5QkiQr-Q/edit#gid=1982710437

Solar System Pipelines

Design Documents

Repositories (Code)

High-level Development Plan

By the end of...... we aim to have:
2019Moving object linking (based on Heliolinc) running at design spec, tested on LSST simulation and ZTF data.
2020The ability to link objects, submit to the MPC, receive a new orbit catalog, and attribute observations to known objects.
2021The above, with the ability to compute LSST-specific table of asteroid physical quantities (absolute magnitudes, etc.). Run on ComCam data.
2022Run operations-ready pipeline with LSSTCam, and have the DRP Solar System pipelines ready.


Guidelines to Approach to Design and Development

  1. Resources and specialist expertise are in short supply; prefer collaborative projects with teams having similar needs.
  2. Time is of the essence; prefer to reuse and improve what's already out there.
  3. Enabling feedback is critical; make code trivial to install and run.
  4. The 2nd law of thermodynamics has not been repealed; aim to continuously integrate and test at every level.
  5. Custom procedures or obscure tooling reduce adoption and drain resources; follow community accepted standards and tools and evolve as they do.
  6. Things will go wrong; aim to tackle most important work first, update plans based on lessons learned.

Choices and standards

System Components
PurposeCore Tool / AlgorithmNotes
Moving object linkingpytrax (HelioLINC implementation)Developing collaboratively with the MPC
Orbit determinationOpenOrbContributing to Mikael Granvik's code
Orbit integration / ephemeris generationOpenOrbNote: the final ephemeris tool may use something else.
Development environment
PurposeToolNotes
Development environment and external package sourceAnaconda Python Distribution (with conda-forge, if needed)Where not required by LSST standards to use the LSST CI system (for packages developed collaboratively with external groups)
Packaging and Binary DistributioncondaWhere not required by LSST standards to use the LSST CI system (for packages developed collaboratively with external groups)
Continuous IntegrationAzure PipelinesWhere not required by LSST standards to use the LSST CI system (for packages developed collaboratively with external groups)
Code coverageCodecovWhere not required by LSST standards to use the LSST CI system (for packages developed collaboratively with external groups)
Unit test frameworkpytest-
Analysis reportsJupyter NotebooksNote: as supporting materials to DMTNs (and potentially a replacement)

Developer Infrastructure

LSST Solar System at UW

Datasets:

DatasetLocation on the LSST NCSA filesystem
JPL 3 month full density/epyc/users/moeyensj/datasets/jpl/fullDensity_3months/
JPL 3 month full density (MPC version)

/epyc/ssd/users/eggl/pytrax/pytrax_LSST.db; 

/epyc/ssd/users/moeyensj/pytrax/pytrax_LSST.db

LSST Solar System at NCSA

Verification cluster details and access: https://developer.lsst.io/services/verification.html

Datasets:

DatasetLocation on the LSST NCSA filesystem
JPL 3 month full density/project/solarsystem/jpl/fullDensity_3months/
JPL 2 month reduced density/project/solarsystem/jpl/reducedDensity_2months/
LDM-156 5 year/project/solarsystem/ldm156/
Small populations/project/solarsystem/smallPopulations/


Setting up a pytrax development environment

To have both pytrax and LSST stack running in the same environment, do something like the following:

source /software/lsstsw/stack3/loadLSST.bash
conda create -n pytrax-dev -c defaults -c conda-forge flann boost pandas pymysql matplotlib scipy astropy ephem healpy jupyter gxx_linux-64
source activate pytrax-dev

Once you have the environment, follow https://github.com/pytrax/pytrax#developing-with-anaconda-recommended to build the code.

Notice that gxx_linux-64 was added to the command line above, to bring in Fortran.

Installing a pre-built version of pytrax from the conda channel

Largely follow instructions at https://github.com/pytrax/pytrax#installing-from-mjurics-conda-channel

source /software/lsstsw/stack3/loadLSST.bash
conda create -n pytrax-test -c defaults -c conda-forge -c mjuric


Using sims_maf, sims_movingObjects

The lsst sims stack is now also installed at NCSA. Access it here:

source /software/lsstsw/stack_sim_20181015/loadLSST.bash
  • No labels