Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Goals:

  • Enable the developer to easily bootstrap the development environment by downloading all the code necessary to build a specific package
  • Enable the developer to easily download and build the code for a ticket they're reviewing
  • Enable a tester to easily download and build the code they've been asked to test
  • Enable the developer to rebuild a set of packages from source, given a list (a manifest) of how they've been built before
  • Provide interface to buildbot to do periodic and on-demand builds
  • Enable automated builds and tagging of release branches 

 

Code Block
languagebash
titleExample workflows for the build tool
# Initializing the build directory (the directory into which the repositories will be cloned to)
bt init
 
# Configuring the build directory
bt config upstream.pattern <pattern list>			# where to find the git repositories (now known as REPOSITORY_PATTERN)
bt config exclusions <filename>						# list of exclusion regexes, when following the dependency chain
bt config versiondb.url <versiondb repository url>	# URL of the version database (mapping between (product,version,deps_versions) -> +N suffix)
bt config versiondb.writable <true|false>			# Whether versiondb is read only or writable as well (ro is the default, only the CI system can write)
bt config build.prefix <string>						# EUPS build tag prefix (defaults to username if unspecified)

# ... or do the config using a remote config file
bt init --config http://....

# Cloning all repositories required to build lsst_devel and lsst_sims products
# and their master. bt remembers that lsst_devel and lsst_sims are top-level products.
# bt build builds the remembered top-level products (and their dependencies)
bt pull lsst_devel lsst_sims
bt build

# Cloning/pulling like above, but checking out tickets/DM-1234 (if it exists, falling back to master otherwise)
# [note: if no repositories are specified on the command line, bt pull will pull the remembered ones]
bt pull --ref tickets/DM-1234
bt build

# Continuing a build after a code change (say, after a build failure)
# The -a flag tells build to recompute versions (otherwise it will refuse to build a "dirty" 
# tree -- "dirty" compared to the state just after pull)
bt build
... build fails ...
... chdir, edit, commit, push, etc ...
bt build -a

# Clone/pull given a build manifest. Build manifest is a text file of products,SHA1,version,dependencies tuples.
# This allows one to reproduce a build someone else made (with bt).
bt pull --manifest manifest.txt
bt build

 
# Push changes to all repositories (top-level + dependences)
bt push
 
# Create a branch in all repositories
bt branch release/8.1.0
 
# Tag all repositories (matching tag.pattern config parameter)
bt tag 8.1.0.0
 
# Increment version tag on all repositories matching tag.pattern. The goal is to allow simple increments of .W releases
# on release branches.
#
# For each repository, this will:
#   * check if there's an annotated tag on HEAD. If yes, continue to next repository.
#   * Find the closest tag in history. If the tag prefix matches the version part of the
#     branch name (e.g., in release/8.1.0, that would be 8.1.0), split off any remaining
#     integer <N> from the end (e.g., if the tag is 8.1.0.5, N=5). Otherwise, see if the
#     tag matches <prefix>-<version>.N (e.g., a tag 1.3.4-8.1.0.7 would match this
#     pattern, with N=7). Otherwise, assume <prefix>==tag, and N=-1
#   * Increment N by one (check for tag collisions, increment further if necessary).
#     Construct new tag using prefix, version, and N.
#
# The algorithm above is designed to tag LSST repositories with dotted-quad tags, while
# external packages with LSST patches will be tagged as <their_version>-<lsst_version>.
#
bt tag --increment
 
#
# Usage by the CI system
#
 
# Automated daily & triggered builds
bt pull && bt build										# build master
bt pull --ref tickets/DM-1234 && bt build				# build tickets/DM-1234, reverting to master if unavailable
bt pull --ref tickets/DM-1234 --ref next && bt build	# build tickets/DM-1234, reverting to next, reverting to master
 
# ... or, with some syntactic sugar:
bt build --ref tickets/DM-1234 --ref next
 
# Automated builds of release branches
# These are different in that we wish them to be tagged before they're built
bt pull --ref release/8.1.0
bt tag --increment							# the default increment prefixes will be taken from the current branch name
bt push
bt build -a


# ... or, with some syntactic sugar:
bt build --ref release/8.1.0 --tag-increment