Users:
- Release engineer; need to be able to support building multiple products
- Science Pipelines users in application mode
- Developers working on single packages
- Developers working on multiple packages/interfaces
- CI for developers
Goals:
- Simplified, maintainable system: Jenkins, lsstsw, lsst-build, eupspkg
- Faster builds
- Reproducible builds
- Pushbutton releases based on known-good builds (with additional testing)
- Ability to add/layer environment-specific or user-specific conda/pip packages
- Remove versiondb; everything should be in eups.lsst.codes (tags) or GitHub
End states:
- All conda packages vs. eups packages
- python setup.py --develop
- Monorepo vs. multiple repos
- Semver vs. universal tag
- Stateless builds vs. stateful; lsstsw tarballs could make statelessness faster
lsstsw and scons are our build tools; could look into cmake or scons+submodules
- lsst_distrib, individual products, library, lower-level library
- ups table only has dependencies at same level
- dependency file has dependencies on lower-level products
- Possibly use manifest.remap?
- Dependent metapackages are sort of part of environment, installed using eups distrib install
- Every package must belong to exactly one product
- Most packages will belong to big products; only tiny packages for DMTN-104 products
- Release of big products will be similar to today's
- Need option to rebuild to build dependencies rather than using binaries
- Jenkins CI jobs can publish builds from known SHAs on master and possibly branches
- codekit should work since ups tables break its dependencies
Matthew Becker proposal:
- Need middle-ground between fixed explicit and bleed
- lsstsw and newinstall would have fixed and relaxed options
- Fixed Jenkins needs to pass to merge; relaxed gives blockers for next release
- Release scipipe_conda_env as conda-forge package
- Treat as a product, tag with semver
- Then easy to release on conda-forge
- Might be fixed, fixed without build, or relaxed
- bleed might become relaxed
- Unclear if we would use conda-forge release ourselves
Next steps:
- Brian Van Klaveren
- Relaxed conda-env
- Tooling to layer envs/packages
- Rebuild/lsst-build using binaries - possibly assisted by Kian-Tat Lim
- Unknown User (gcomoretto)
- lsstinstall
- Jenkins refactor
- Research metapackages