2018-10-26 Network Engineering Team Meeting notes

NET Action Items Status

Introductions

Update on ESnet agreement

LSST Network SC18 Demonstration:

Northbound use case

DECam Data set will be cached on the DTN.  Data set size is about 10TB. DTN has Fast disk (3 x 1.8 TB) and Slow disk ( 5 - 10 TB) are RAIDED. We will only do DTN Fast disk to Fast disk of maximum cacheable on fast disk, subset of data set ~5 TB. Matt and Jeronimo will get the necessary scripts for initiating data transfer.  JK will need read access to where the data is located.

At NCSA, will copy received files from DTN to GPFS, outer demo will access GPFS. Two windows will be open on the Jupyter notebook, one with network statistics, the other with watch app output. Matt will make arrangements for the use of the monitors in the NCSA booth. Goal is to demonstrate as close as possible to RT performance, small latency is not a problem.

As soon as DTNs are ready, start to test data transfers between DTNs. LS and Miami DTNs are installed. NCSA DTN is currently racked, missing 2 NVME drives. Can mount a GPFS drive as an intermediate step. (After meeting, Matt reported receiving the two remaining NVMe drives)

Mem to Mem tests will be tested first. Vlan 1960 is being used for testing. Goal: Achieve ping test between La Serena and NCSA DTNs asap.

Schedule

Oct 31 Achieve ping connectivity between all DTNs.

Nov 5 Want to be in a position to do as much testing as possible not being in Dallas.  This is the ideal goal. JK will have data set ready to copy to DTN by 11/2 and copied by 11/5.

Transfer protocol and script to be in place by then. Work with Pacific Research Platform Jim Chen to configure DTNs and define transfer protocol script and script - JB, MK

Decide on network monitoring points, statistics, and displays - JB, MK
Access to destination location (NCSA GPFS mount) - MK
Arrange for use of NCSA display monitors in Dallas - MK

NVME drives in NCSA DTN. - MK

Nov 10 Expect SCInet links to not be available until now, 2 days before the event. (JB)

Nov 12 and 13 for onsite dry runs.

Nov 14 morning is the demo.  

Southbound use case

Goal: Send a PB of data southbound to see how long it takes. Measure performance, such as Sustained a rate of X. Demonstrates overall throughput and capacity, FIFO management.
Secondary priority, not necessary to do at same time as NB. Get the northbound working well before worrying about the southbound use case.
Could do equivalent of last December demo, but in reverse direction, does not have to be disk to disk this time.

NRE template for Caltech
Matt believes the Caltech NRE template includes NCSA, which it should not.
Confirm Caltech NRE template is correct.

Brocade issues in LS
All issues have been resolved, we will have 100 Gbps for demo.