New Network Architecture: Overview

Architecture

Gary as the network architect has proposed a new network architecture that is in-line with today's recommended network architecture. Basically this removes much of the so called "flat-earth" nature of the current network where subnets appear in multiple physical locations interconnected by the core and using VLANs (see for example Understanding the Spanning Tree Protocol) and layer 2 switches. The new architecture requires that each building/edge switch has its own set of subnets. These subnets in turn are divided by function, e.g. printers are in a different subnet to world-wide accessible hosts and hosts with a different function (e.g. training rooms, experimental equipment) are in different subnets. This enables improved security isolation and addresses a criticism by reviewers that the printers are not properly isolated. A downside is that hosts will need to be re-addressed to the appropriate new subnet when they are moved to the new architecture. Also, later whenever a host moves from one building/edge switch to another it will need to be re-addressed.

Relation to Network Upgrade

This architecture is currently tied in where possible with the network upgrade and the implementation is in progress. So far the central core network has been upgraded to 10Gbps, some buildings have also been upgraded with 10Gbps up links (Kavli, MCC) and we are well along in planning the upgrade of other building/edge switches. In addition we have been requested to move, by the end of this year, network equipment from the old rows to seismically retrofitted rows. The overall upgrade/move involves 20 racks of equipment in building 50 room 210, ~ 150 devices, hosts in about 60-70 buildings and about 4000 end nodes (hosts, controllers, printers etc.) Where possible we are striving to make the two changes (the upgrade with new links, switches etc. and the new subnet architecture) to be done together to reduce the need for multiple changes/upgrades consequent outages etc.

Pros

The upgrade and new architecture provide several important improvements including:

Architecture:
- Better Problem isolation
- Broadcast isolation (don't go through core)
- Potential to utilize redudant links (double capacity)
- Isolates things so may assist with security
Upgrade:
- Removing switches which are beyond end of life
- Enable auto-negotiation everywhere by replacing older switches
- Higher speeds to desktops (1Gbps with 10Gbps uplinks)

Cons

As mentioned above, a major downside is the need to re-address hosts for the new subnets both at the time of the re-architecting and in case of future moves. Without some automation support (DHCP) the re-addressing has to be done manually at both the switch and the host. Thus coordination is needed between the network and host administrators. In the long run we want to move to DHCP support but according to Gary, this requires a major overhaul of the functionality and accuracy of the CANDO database. DHCP would enable the automated re-addressing of hosts without manual intervention.

Current State

We have met with the OU admins and presented the case for the upgrade and re-architecting togehther with the impact and plans. We have also identified some buildings/switches (Building 050 floor 1, Test Lab) on which to start the process so we can gain experience and improve documentation/ procedures.

Since we do not believe it is possible to re-address all switches and hosts by the end of year deadline, we have developed an alternative that involves building/configuring two switch/routers (CORE3OLD and CORE4OLD) from spare parts. They will be located in seismically retrofitted racks (row 8). These switches will connect "non-address-moved" building/edge switches. Later these building/edge switches will be re-addressed and moved to the new core.