| Time | Item | Who | Notes |
|---|
| | Verification cluster status | Unknown User (jalt) | |
| | Integration cluster / PDAC hardware status | Unknown User (jalt) | - Setup was planned to be complete by today. Some delays due to additional security concerns.
- Will need to introduce an additional intermediate bastion host because of the access level required to the PDAC servers for installation of Qserv and Firefly services.
- Unknown User (jalt) will provide a script that will assist in bridging the multiple ssh hops required.
- It is hoped that this will accommodate the need for remote, tunneled access from IPAC development machines.
- Considering migrating to a different 2FA system.
- Additional time was required because of the recent SLAC group request for a previously unplanned RAID configuration on the Qserv machines.
|
| | File transfers from IN2P3 to NCSA, as well as intra-NCSA transfers of the Stripe82 data from the old servers | Igor Gaponenko | - Transfers of data from IN2P3 are slower than initially anticipated because the SDSS data is managed by iRODS and physically archived on tape as bundled files. This means that recalling individual files implies retrieving the whole bundle and unbundle it. There is also a limitation in the local storage of the transfer machine (1 TB) which reduces the number of recalling operations that can be simultaneously performed. In a private mail exchange prior to this meeting, Fabio Hernandez proposed some alternative solutions to Igor Gaponenko but they were perceived as adding unnecessary complexity. The time available to complete the data transfers is considered enough to continue using the current method.
- The data transfer process should take another two weeks to complete, but it is not a blocker to beginning to run services at NCSA once access to the PDAC cluster is complete.
|
| | 2017 planning | Unknown User (jalt) et al. | - Mario Juric visited NCSA this week and shared the vision for Level 3 capabilities built around a JupyterHub front end to user computing resources, previously discussed with SQuaRE and SUIT. NCSA group is enthusiastic about this because they can see it making it possible to meet a variety of requests associated with commissioning and other interactive use cases.
|
| | "lsst-dev" Firefly server | all | - This server is intended for use by DM Science Pipelines developers to visualize image and tabular data associated with their work, which will be stored on the shared filesystems available on lsst-dev7 and the verification cluster.
- Unknown User (jalt) asks: can these capabilities be merged with PDAC? Short answer (summarizing a lot of conversation): not in the next couple of years. They are rather distinct in their needs for access to data, mutability, non-LSST-project access, etc. However, we will look for all possible commonalities in their deployment architectures (containerization, etc.). By mid-2017 we will want Level 3-ish (JupyterHub) capabilities in both places.
- We need a separate host for the SciPi-facing server.
|