2021-01-14 Network Engineering Team Meeting notes

Philip DeMar

Reminder: Work on your action items: International Communications (aka Long-Haul Networks)

Network activities schedule (subject to COVID19 impacts):

January - Continued work on  Rubin Observatory Network Verification Plan (including JIRA). 

February - Completed draft  Rubin Observatory Network Verification Plan (including JIRA).

March - Scheduled Rubin Observatory Networks Pre-Verification Review for May 18 - 19

April - Prepare for Rubin Observatory Networks Pre-Verification Review.  Submit LSST Change Request to update LSE-78 Rubin Observatory Network Design (including Chilean, International, and US networks), and related milestones in PMCS.   Complete LDM-732 Rubin Observatory Networks Pre-verification Baseline.

May - Complete FY19, FY20 NET Reports and Technical Document

June - Conduct and respond to actions from Rubin Observatory Networks Pre-Verification Review.  Meet with GlobalNOC re VNOC support.

July  - Pre-Ops Rehearsal 2 testing.

August - Ops Rehearsal 2/Pre-verification testing.

September - Cisco ACI review, Huawei analysis 

October - Start Verification Testing, LS restart, LS - SCL 40G upgrade. (hopefully) - 

November - Update Rubin Observatory Operations & Management Plan, USDF Selection, CP Restart, Network Contracts and SLAs.

December - Updates on G-NOC service progress, Brazil path (Huawei) options, planning for US Data Facility at SLAC

January - Updates on G-NOC service progress, Brazil path (Huawei) options, planning for US Data Facility at SLAC

• Jeff K:  G-NOC proposal reviewed by Rubin Observatory operations & management.  Reaction has been positive, but such a service would need to be subject to an open solicitation process
• Jeff K:  The funding agencies (primarily NSF) have been briefed on the bypass options for the Huawei-based segment of the LHN in Brazil.  Expect to proceed with a supplemental request for option 2 (AmLight extension).  Need to determine AURA rate & overheads before proceeding.  Issue rests with Victor K right now.
• Mark F:  SLAC is in formative stage of determining DF requirements, such as floor space, power, cooling.  Internal network planning to follow.  They have been in discussions with ESnet on how their WAN connectivity should be modified to accommodate the LHN.
  • Jeff K will be soliciting SLAC input into development of the LHN SLAs

Paul Wefel

Jeronimo Bezerra

CONUS Networks, including ESnet:

• Paul W reported ESnet is solidifying plans for LHN support to the US DF, as well as support for data movement between the DF and IN2P3.  Paul raised the question of connecting the AmLight LHN service to SoX with the ESnet PoP at Peach Tree.  Jeromino B stated this won't be an issue.  Amlight will get to where ESnet is.  Currently this happens via an I2 backhaul.  The permanent AmLight LHN termination in ATL will be to Peach Tree, but this step has to wait for operations funding.  The current AmLight path via SoX could still serve as a backup path.  Jeff K asked if there's a commitment from ESnet to continue their peering with SoX.  Paul W agreed to take this an action item to find out.  Paul W also reported that concerns about potentially needing a repeater in El Paso to support one of the possible LHN paths to SLAC should be mitigated by deployment of new transponders.  Finally, there was some discussion on possibly using another ESnet PoP for the LHN besides just ATL.  Houston was briefly discussed. 
• Paul W responded about a question on an outage with the current (20Gb/s) LHN primary path within ESnet.  One link was indicating high loss, and was brought down for maintenance.

Cristián Silva

Albert Astudillo

Jeronimo Bezerra 

Developments in La Serena and end-to-end service to NCSA for upcoming Auxiliary Telescope Traffic and general (internet, mail, voice) traffic:

• The status of the backup link in Recinto is unchanged.  It is a facility issue.  The issue has been escalated to the Victor K level.
• Albert A:  The La Serena↔ Santiago backup path has been implemented and is working.  The missing link between Reuna and AmLight at Lumen is still in progress, but should get done this month.
• The question was raised about whether there is any auxiliary telescope traffic yet.  Cristian S would need to answer that.  Generally, while ops have started at the summit, there are still lots of restrictions/limitations.

Unknown User (kollross)

Richard Hughes-Jones

Stuart McLellan

NCSA - CC- IN2P3 network:

Tests have been run between IN2P3 and NCSA.  ~500TB was transferred.  While the circuit is capable of supporting 20Gb/s, transfer rates were limited to ~6Gb/s.  Current trans-Atlantic path is via GEANT.  IN2P3 & RENATER are ready to run tests with SLAC when they are in a position to do so.  Question was raised whether initial IN2P3 ↔ SLAC tests should just start with using ESnet trans-Atlantic links, since that traffic would fall under ESnet AUP.

By topic

Rubin Observatory Network Verification:

No update on this.  Verification testing rests with Cristian S.

Announcement on the SAACC & NET face-to-face meetings (both virtual) has gone out.  The dates will be April 13-14 for the SAACC and April 15-16 for the NET. 

Julio suggested that assembling a list of the pertinent LHN documents be assembled prior to the meeting.  Discussing/updating those documents could then be used as a framework for shaping the meeting agenda.  Jeff K said the LHN Confluence page should provide a stale list of the documents.  Julio took on an action item to create a catalog of the relevant documents.

Philip DeMar 

Wrap up and Next meetings:  

11 Feb 2021 

All meetings at 0800 PT/0900 MT/1000 CT/1100 ET/1300 CLT (subject to time changes)