SLAC's
Network Principles
First version May 1996. Last Update: June 2008
SLAC: [ The Lab | Index |
Phone Dir. |
Network Group |
French tranlsation of this page ]
We have identified a set of key points (principles) that should be applied when
designing future network upgrades and making new purchases of network equipment:
- Focus on 10/100/1000/10000 Mbps Ethernet, only suppport other
technologies (e.g. Infiniband) if resources are made available.
- Structured wiring
- Use single mode fiber between buildings.
When pulling single mode fiber between buildings make sure
extra fibres (e.g. an extra 1/3) are pulled in case of failures or
added future needs.
The extra fibres can be left dark.
- Use CAT6 (or better) for desktops (rather than CAT5E),
since most of the cost
is pulling, testing etc. the cables, so the longer they last the
better. The rationale is that cables should last 10 years. Bandwidth
is doubling roughly every 18 months. Current applications running
at 1Gbps are pushing the limits of CAT5E. Newer applications like
streaming video, bigger files, network attached storage will
benefit from CAT6.
- Clusters typically have much shorter runs < 12
feet and they are pretty simple (no conduits, ceiling tile,
walls etc.) so maybe CAT5E is OK. This is especially true since
any upgrade in bandwidth for the cluster will probably require
new NICs, new hosts, a new cluster and new cabling.
In addition CAT5E is roughly 20% less than CAT6.
- We only install CAT3 cables where the IDF (phones) and BDF (data)
closets are in seperate locations. There is no way to move from
one location to the other. When the IDF and BDF are in the same
wiring closet, of course, we install CAT5E or CAT6 cables for the
phone lines.
- Support Wireless centrally
- Since the layoffs of 2008, Wireless networking is best effort (see
Policy and expectations for the SLAC Visitor and Wireless networks
).
- Recognize wireless is 50% harder ro support/manage than landlines.
- So we know what is on the network, are aware of rogue access points,
buy
reliable equipment, that can be monitored and managed.
- Provide monitoring (so need monitorbale devices)
- Wireless Access Ports (WAPs) for new installations are funded
from group/department budgets. After purchase, the WAPs are owned,
configured, supported and maintained centrally.
- Wireless Access Points are not physically secured. Measures have been
taken to mitigate this.
- Manage multicast centrally, do not deploy until cost benefits apparent.
- Continually upgrade the network, ensure
there is continuing funding support (e.g. SLAC GPP)
- Design the network to accomodate change.
- Aim to replace equipment that is beyond end of life
(note that network product cycles are only about 4-5 years).
- Keep track of age and state of equipment.
- Over-provision the network in order to avoid constant trouble shooting caused by
being close to the edge of the envelope
- Progressively provide switched 100Mbps Ethernet (or better)
to the desktop. In selected cases supply 1Gbps to the desktop.
- Increase the aggregate bandwidth available by a factor of 2 every 12-18 months
- Provide 1000 Mbps connections to high volume servers.
In general, we expect
servers requiring > 100 Mbps to be SLAC wide servers with
critical needs.
Thus, they are expected to be modern machines capable of fully
utilizing a 1000Mbps interface (e.g. at least a Pentium IV
or equivalent with >
1.5GHz Mhz and a 64-bit PCI and RAID). Also they are expected
to be backed up, for their Operating System to be kept current,
to have high availability which is monitored,
often supported by Uninterruptible Power Sources (UPS), with
automatic paging to someone on call 24 hours by 7 days
per week etc., be located in a
secure locked area, and usually located in the computer
center so they can be close to the
core networking.
- Provision the network core with 10Gbps Ethernet interconnects and
provide 1Gbps connections to high volume central Enterprise
servers.
- Plan to ensure net is not bottleneck.
- Centralize servers & storage to reduce costs.
- Ensure the trunks are higher speed than end connections, e.g. for
1Gbps to desktop switches, make sure the uplink trunk is 10Gbps.
- Monitor and track requirements for
backbone upgrades. Provide monitoring and management capabilities
for wired and wireless Ethernet.
- Reliability/robustness, minimize need for heroic efforts
- Design the network to facilitate ongoing maintenance. For
example provide redundancy so can take parts of the network out
of service with limted impact on the users.
- Provide redundancy for critical components.
- Place the critical core network components on UPS.
- Invest in network management and monitoring:
- To enable setting and maintenance of network performance expectations
- To leverage our proactive trouble shooting and planning resources
- To provide accurate documentation for trouble shooting
- Make sure that all new network devices support SNMP.
- Make sure that all new network devices of a certain class also support RMON.
- Provide appropriate tools for diagnosing problems down to the
wire level.
- Invest in tools to enable asset management,
in particular to keep track of network and
device configurations and topology, maintenance, purchase price & date,
end of life dates.
- Enable automatic notification of problems to appropriate personnel.
- Be prepared for failure
- For smaller switches rely on lifetime warranty and maintain spares.
- For larger switches put on Cisco maintenance (1-2 days replacement).
- A network containing shared hubs cannot be effectively managed. For
example they do not support VLANs and the Spanning Tree Protocol
and so can cause spanning tree loops; also traffic for individual nodes
cannot be measured by SNMP thus also raising security concerns.
Therefore we will endeavor to replace
existing granfathered shared hubs, and new connections of
shared hubs will not be allowed.
- Reduce complexity - simplify and standardize
- Minimize the number
of vendors providing
backbone/core networking components and maintenance,
and foster strategic relations with
those vendors. At the moment all backbone/core routers and
switches are manufactured by
Cisco.
- Take into account total cost of ownership, initial purchase with discount,
maintenance costs, carrying spares, training on devices, ability to monitor
and manage, part exchange value etc.
- Reduce the protocols carried in native form -
target only the IP protocol
family for the long term. Do not support AppleTalk,
bridging of DEC suite protocols (MOP, LAT etc.) in the core,
remove support for native DECnet phase IV.
Do not route IPX, actively
discourage use of IPX.
- Minimize the number of switch and line card models. This enable us to maintain spares,
carefully track end of life (sale) and service dates, ensure monitoring(e.g. SNMP MIBs)
are well understood, and the command line interface is well known.
- Where possible configure for auto-negotiation, especially when making changes.
- Security
- The SLAC Computer Security Program Plan states that
all unused taps are to be disabled until required.
- If a switch port is not seen in use for some period of time then
it is disabled. It is then available for reassignment (e.g. to
another room). This is a security requirement. The period of time
we will use is 3 months.
- When we install a new switch we disable all ports that have not been
requested to be be enabled.
- Except in special cases a subnet is restricted to one
building switch.
[ Feedback | Reporting Problems ]
Les Cottrell