Users:
  • Release engineer; need to be able to support building multiple products
  • Science Pipelines users in application mode
  • Developers working on single packages
  • Developers working on multiple packages/interfaces
  • CI for developers
Goals:
  • Simplified, maintainable system: Jenkins, lsstsw, lsst-build, eupspkg
  • Faster builds
  • Reproducible builds
  • Pushbutton releases based on known-good builds (with additional testing)
  • Ability to add/layer environment-specific or user-specific conda/pip packages
  • Remove versiondb; everything should be in eups.lsst.codes (tags) or GitHub

End states:
  • All conda packages vs. eups packages
    • python setup.py  --develop
  • Monorepo vs. multiple repos
  • Semver vs. universal tag
  • Stateless builds vs. stateful; lsstsw tarballs could make statelessness faster

lsstsw and scons are our build tools; could look into cmake or scons+submodules

  • lsst_distrib, individual products, library, lower-level library
  • ups table only has dependencies at same level
  • dependency file has dependencies on lower-level products
  • Possibly use manifest.remap? 
  • Dependent metapackages are sort of part of environment, installed using eups distrib install
  • Every package must belong to exactly one product
    • Most packages will belong to big products; only tiny packages for DMTN-104 products
    • Release of big products will be similar to today's
  • Need option to rebuild to build dependencies rather than using binaries
  • Jenkins CI jobs can publish builds from known SHAs on master and possibly branches
  • codekit should work since ups tables break its dependencies

Matthew Becker proposal:
  • Need middle-ground between fixed explicit and bleed
  • lsstsw and newinstall would have fixed and relaxed options
    • Fixed Jenkins needs to pass to merge; relaxed gives blockers for next release
  • Release scipipe_conda_env as conda-forge package
    • Treat as a product, tag with semver
    • Then easy to release on conda-forge
    • Might be fixed, fixed without build, or relaxed
    • bleed might become relaxed
    • Unclear if we would use conda-forge release ourselves

Next steps:
  • No labels