This is the page of the Solar System Processing team (Siegfried Eggl,Joachim Moeyens, Mario Juric), tasked with defining the delivering finished DM Products (i.e., software, documentation, operating procedures) needed to deliver the LSST Data Products to support the Solar System Science. It can be reached through the shortener via http://ls.st/ssp (mnemonic: solar system processing).
High-level Overview
| Data Products | Pipelines |
|---|---|
High level overview is at http://ls.st/doc-29545, screenshotted below:
|
Solar System Data Products
The overview can be found in the Data Products Definition Document. (note that this is temporarily out of date; should be updated by ~August 2020).
The detailed schema is at https://docs.google.com/spreadsheets/d/1E0rTlvuJC0CvpLNsuWLK0x70uhpZww4v6GB5QkiQr-Q/edit#gid=1982710437
Solar System Pipelines
Design Documents
- The change request document to update the Solar System baseline.
Repositories (Code)
- Pytrax (moving object linking): http://github.com/pytrax/pytrax
- Given a list of detections over a past few weeks, pytrax identifies detections (likely) belonging to the same moving object
- OpenOrb (orbit integration): http://github.com/oorb/oorb
- A general purpose orbit computation package: provides orbit determination and ephemeris calculation.
- Analysis code for MOPS, pytrax, thor: https://github.com/moeyensj/difi
Legacy MOPS (deprecated): https://github.com/lsst/mops_daymopsLegacy ephem calculation & association (deprecated): https://github.com/lsst/mops_nightmopsLegacy MOPS analysis code: https://github.com/moeyensj/analyzemops
High-level Development Plan
| By the end of... | ... we aim to have: |
|---|---|
| 2019 | Moving object linking (based on Heliolinc) running end-to-end, tested on LSST simulation. ✅ |
| 2020 | The ability to link objects, submit to the MPC, receive a new orbit catalog, and attribute observations to known objects. |
| 2021 | The above, with the ability to compute LSST-specific table of asteroid physical quantities (absolute magnitudes, etc.). Run on ComCam data. |
| 2022 | Run operations-ready pipeline with LSSTCam, and have the DRP Solar System pipelines ready. |
Guidelines to Approach to Design and Development
- Resources and specialist expertise are in short supply; prefer collaborative projects with teams having similar needs.
- Time is of the essence; prefer to reuse and improve what's already out there.
- Enabling feedback is critical; make code trivial to install and run.
- The 2nd law of thermodynamics has not been repealed; aim to continuously integrate and test at every level.
- Custom procedures or obscure tooling reduce adoption and drain resources; follow community accepted standards and tools and evolve as they do.
- Things will go wrong; aim to tackle most important work first, update plans based on lessons learned.
Choices and standards
| System Components | ||
|---|---|---|
| Purpose | Core Tool / Algorithm | Notes |
| Moving object linking | pytrax (HelioLINC implementation) | Developing collaboratively with the MPC |
| Orbit determination | OpenOrb | Contributing to Mikael Granvik's code |
| Orbit integration / ephemeris generation | OpenOrb | Note: the final ephemeris tool may use something else. |
| Development environment | ||
| Purpose | Tool | Notes |
| Development environment and external package source | Anaconda Python Distribution (with conda-forge, if needed) | Where not required by LSST standards to use the LSST CI system (for packages developed collaboratively with external groups) |
| Packaging and Binary Distribution | conda | Where not required by LSST standards to use the LSST CI system (for packages developed collaboratively with external groups) |
| Continuous Integration | Azure Pipelines | Where not required by LSST standards to use the LSST CI system (for packages developed collaboratively with external groups) |
| Code coverage | Codecov | Where not required by LSST standards to use the LSST CI system (for packages developed collaboratively with external groups) |
| Unit test framework | pytest | - |
| Analysis reports | Jupyter Notebooks | Note: as supporting materials to DMTNs (and potentially a replacement) |
Developer Infrastructure
LSST Solar System at UW
Datasets:
| Dataset | Location on the LSST NCSA filesystem |
|---|---|
| JPL 3 month full density | /epyc/users/moeyensj/datasets/jpl/fullDensity_3months/ |
| JPL 3 month full density (MPC version) |
|
LSST Solar System at NCSA
Verification cluster details and access: https://developer.lsst.io/services/verification.html
Datasets:
| Dataset | Location on the LSST NCSA filesystem |
|---|---|
| JPL 3 month full density | /project/solarsystem/jpl/fullDensity_3months/ |
| JPL 2 month reduced density | /project/solarsystem/jpl/reducedDensity_2months/ |
| LDM-156 5 year | /project/solarsystem/ldm156/ |
| Small populations | /project/solarsystem/smallPopulations/ |
Setting up a pytrax development environment
To have both pytrax and LSST stack running in the same environment, do something like the following:
source /software/lsstsw/stack3/loadLSST.bash conda create -n pytrax-dev -c defaults -c conda-forge flann boost pandas pymysql matplotlib scipy astropy ephem healpy jupyter gxx_linux-64 source activate pytrax-dev
Once you have the environment, follow https://github.com/pytrax/pytrax#developing-with-anaconda-recommended to build the code.
Notice that gxx_linux-64 was added to the command line above, to bring in Fortran.
Installing a pre-built version of pytrax from the conda channel
Largely follow instructions at https://github.com/pytrax/pytrax#installing-from-mjurics-conda-channel
source /software/lsstsw/stack3/loadLSST.bash conda create -n pytrax-test -c defaults -c conda-forge -c mjuric
Using sims_maf, sims_movingObjects
The lsst sims stack is now also installed at NCSA. Access it here:
source /software/lsstsw/stack_sim_20181015/loadLSST.bash