THIS PAGE IS OUT OF DATE.  ALL GEN3 MIDDLEWARE CODE IS NOW ON MASTER, SO THERE ARE NO SPECIAL INSTRUCTIONS TO SET IT UP.  THE gen3-middleware BRANCH HAS BEEN RETIRED AND SHOULD NOT BE USED.


Prior to full integration with the rest of the codebase, Gen3 Middleware work is being done on different branches in different packages.  This page tracks which packages use which branches and which weekly versions are expected to be compatible.

In general, packages fall into four categories:

  • Gen3 work is merged to master, as no non-Gen3 version exists (daf_butlerpipe_supertask).
  • Gen3 work is merged to the gen3-middleware branch, which is rebased periodically on master to avoid diverging from the main codebase.
  • Gen3 work has been merged to master, as it is either considered stable or non-intrusive.
  • No Gen3 specializations exist; at least one weekly tag is guaranteed to be compatible with the Gen3 stack.

Packages

NameBranch/TagNotes
daf_butlermasterNo non-Gen3 code exists.
pipe_supertaskmasterNo non-Gen3 code exists.
skymapgen3-middleware
obs_subarugen3-middleware
ci_hscgen3-middleware
(all others)w.2018.22


Setting up the Gen3 Middleware Stack

As all Gen3-specific changes to date are pure-Python, ABI compatibilities with stack packages are not a concern.  The easiest way to obtain a Gen3 Middleware Stack is thus:

  1. Install or obtain a version of lsst_distrib with the appropriate weekly tag (i.e. eups distrib install lsst_distrib -t w_2018_XX).
  2. Use setup lsst_distrib -t w_2018_XX to set up a complete mainline stack with that tag.
  3. Use setup -j -r .  on a git checkout of each of the packages in the above table that do not use this tag.

Updating the Gen3 Middleware Stack to a new weekly tag

  1. Announce your intent to update the stack on the #dm-middleware Slack channel.
  2. Rebase all packages using the gen3-middleware branch on the desired tag, using the --preserve-merges option to keep merge commits with ticket numbers in the history.
  3. Build the gen3-middleware version of ci_hsc from the beginning (i.e. run scons -jN after deleting the .scons and DATA directories).
  4. Push (with --force) the rebased gen3-middleware branches.

This testing should also be possible with Jenkins, by pushing the rebased gen3-middleware branch to a temporary upstream branch.


  • No labels