It is a goal of the S18 cycle to re-deploy the existing PDAC components, and then continue their development, in a containerized fashion, using Kubernetes to manage the deployment, and then, based on the availability of Kubernetes, to deploy the JupyterLab/Notebook Aspect on the PDAC hardware in addition to the existing Database/API and SUIT/Portal Aspects.
Part of this work is to determine how, in detail, to apply Kubernetes to organize a containerized deployment of API and Portal Aspect components such as Qserv and Firefly.
To that end, what is required in the near term is not a fully-polished deployment environment, but a basic installation of Kubernetes that will allow the SLAC and IPAC teams to gain experience with using it to host their services. This will allow the investigation of the techniques appropriate to issues such as managing the deployment of the Qserv containers to the appropriate hardware, the establishment of multicast connections between the containerized Firefly servers (or the adoption of a different approach to managing the propagation of session state and cache between the servers), and so on.
We will also use this prototyping environment to explore techniques to use for configuration management of the deployed LSP containers and their management, i.e., the Kubernetes application configuration (yaml files), and learn how to coordinate this among the three Aspects and the underlying NCSA services.
The requirements for the initial use of the Notebook Aspect ("jellybean") on PDAC are to be able to make calls to the DAX APIs from Python processes, and for the PDAC services, including its instance of jellybean, to be able to access the NCSA AIM services.
As an initial step, we would like NCSA to add kubeadm , kubectl , and kubelet , from the Kubernetes 1.8 distribution, to the existing Puppet configuration for all the PDAC hardware hosts. The current OS seems to be acceptable for use with Kubernetes. Kubernetes depends on docker, of course, and the latest version of docker that is used in the Kubernetes developers' CI system is 17.03.2; this is apparently slightly older than the version currently installed on the PDAC nodes. Most likely that will not be an issue, but we should stay aware of it.
The full requirements for jellybean are available here: https://sqr-018.lsst.io/#infrastructure-resources but some of these (particularly resource requirements) are aimed at supporting moderate numbers of users, beyond what is immediately needed in the PDAC environment.