RAL Site Visit - March 1998
LAN - Alan Flavell, John Gordon, Roland Brandword*
LAN - Alan Flavell, John Gordon, Roland Brandword
The Local Area Network is based on FDDI concentrator in CC (Computer Center). It is only lightly loaded, most traffic stays within department. Most of the large buildings have a router. The demark is in the router in each building. The department pays for board in routers and the CC manages the routers. There are 7 main site routers. SW (Structured Wiring) has been deployed so far in the CC. Funding for SW comes from depts. Budget 80 English pounds/point (point = 1/2 of junction box), will use SW for voice also. Putting in about 4400 points eventually. Will be UTP cat 5 to desktop (not fiber at the moment). Voice will also use SW UTP cat5. Will keep department routers in buildings with switches (3COM) with managed hubs (24 port hubs mainly) for workstations & routers. The SuperJanet (ATM <= 34 Mbps) Router (RAL's) is currently a 3COM, will be Cisco (better accounting and control facilities).
Connectivity to SJ (SuperJanet the Academic & Research network n the UK) is at 16Mbps, plus managed network points for various small sites, plus ISDN, plus CERN 64kbps leased line with compressors (6K English pounds initial cost, they pay for themselves quickly) for interactive use (have to login to CERN).
They are using a Cisco 4000 with 1 PRI with 10-15 users. Mainly for telecommuters, plus a small site. Use Cisco 760 in the client end. ISDN is costly, 400 English pounds to install and 80 pounds /quarter. Usage varies with time of day and for local calls varies from about 1p to 3.4p. Calls can be charged in both directions since the second channel may be brought up from the client end or the server end.
They are looking at DHCP. They are concerned with integration with DNS. Also removal of security by IP address for single machine, in particular the Datastore uses IP addresses for security. They recommend using DHCP for static addresses. This helps management since people do not have to enter configurations. At the moment some "villages" are deploying DHCP servers with dynamic pools. There are 2 test ones that will go into production with static addresses. This also allows portables to migrate over the site with one pre-allocated IP addresses per subnet. Expect to have one DHCP sever/village for increased availability (each one will serve the one or more subnets in its village). Token life will vary. When client comes up it sends a request for renewal of the address. If the server is up and the client is on the same subnet, then it will renew the address, else it will get a new address. The lease is recommended to be 2 days. They are using the WNT4 DHCP server, since the RAL policy is to move to NT. Since they use static addresses they get the user to tell them what the MAC address. Typically many users say this is my IP address and then Tim Evans looks at the MAC address associated with it and adds it to DHCP server. The main requirement is coming from the portables. Expect demand from conference rooms, but some conference rooms are not cabled.
No easy way available yet for DNS integration. There are 3rd party products, but not interoperable with other vendors, and fairly costly.
Issues with WNT 4 server. Tim does not recommend. Microsoft have a lot of room for improvement. WNT4 gives most of functionality they want. However, database is kept in 2 places. Part is in the registry and part in Jet, and need to reconcile, plus backup/restore is a challenge (e.g. restoring part of registry is challenging even Microsoft says don't touch registry). Tim has not had success with this in particular with the static mapping of IP to MAC address.
People typically are on their own. Admin ran CCMail, Daresbury ran MSMail, Unix ran Unix. The CC ran a POP server with Eudora client. Also running a POP compatible IMAP server. Particle Physics (PP) looked at IMAP servers and are considering moving. Then a year ago someone said need to look at a coordinated Email system. Quickly decided to focus on Microsoft Exchange (have 3-4 servers). Migration plan was to move MSmail first, followed by ccMail (both completed, ccMail closed down now). There were good migration tools. Then they want to migrate Eudora. Main problem of migrating from Eudora is the existing folders, nicknames etc. Want to close down POP server. They have a site wide mail redirector. Typically pass onto Microsoft Exchange server (which supports IMAP4). They have had problems with Pine (supports IMAP2, needs new Pine 4) not supporting fully IMAP4 (in particular in the area of folder support). PP are going to move from IMAP4 and moving to Exchange.
Exchange is claimed to support LDAP support. But have not explored yet. RAL does have as site directory. It used to be X.500 based. Information on RAL staff is kept in an Exchange directory, and Exchange is supposed to be LDAP capable. Not looked at ACAP yet.
Paul Kummer, Rob Bradshaw, Robin Tasker at Daresbury.
John MacAllister, Tim Evans, John Gordon, Allan Flavell, Paul Jeffries at RAL.
There is some overlap with monitoring already done by PPCNG, should they move to the ICFA PingER?
Networking mini agenda:
Concerns were raised about how to use the statistics. This meant more than just how to present but also how to use the measurements to understand and work with other identify and improve things.
Would like to have a pre-loaded NIMI. SLAC will look into pre-loading and shipping to RAL.
John Mac did some tests with FTP vs. Ping loss between sites in the US and the UK using tcpdump and there was a big correlation.
RAL would be willing to beta test. T.D.Evans@rl.ac.uk will be the contact person.
Rob is not fond of MRTG, nice student project. Good for providing a graphical output. Not easy to get at the database, e.g. if wish to run the data through another analysis program to provide collisions/frames. Idea of producing GIFs each minute that nobody looks at is questionable.
The UK PP folks are interested in characterizing the traffic, this is driven by new charging schemes. Would like to find a site with the physics department on a separate network then put something like Netramet on the subnet to see what the utilization is by particle physics site. Even then it would be for the one site only. They have done some work on the CERN-RAL TEN-34 link. Somebody (Jonathan) has done some work to identify the PP sites in the UK and has done a report to identify the bytes for each site pair.
John is porting TracePing on WNT and Perl. This will enable it to be system independent and portable. John hopes it will be ready by summer this year.
Roman Tirler's paper says the overall HEP traffic on the A&R nets is about 2%.
Allan Flavell & John Gordan will review.
Alan Flavell worked with CuSeeMe, but it is stuck with nv for MBONE use (the LBL folks have not made vic work with CuSeeMe), and if use nv does not work with the WhitePine extensions such as whiteboard. Alan thinks WhitePine are working on new releases including multicast for CuSeeme. Alan is therefore moving to vic and vat etc. But concerned about MBONE use and so want to migrate to reflectors. CERN's virtual room is based on RTPtrans. The HEPNRC are developing the MSB (Multi-Session-Bridge) which provides many features but had problems managing. The net result is that there are 2 tools that are migrating to one another but do not fully inter-work. HEPNRC claims to have developed a gateway between MSB and CERN's virtual rooms. He also said a Web page to configure the MSB will be available in the next couple of weeks. Gary Roedigger will be visiting CERN to talk about interworking. CERN (Christian Isnard) will be coming to RAL next week to discuss the CERN tools. CCLRC (RAL/Daresbury) puts in 1/3 FT for the PP community. There are others covering video conferencing
Daresbury LAN had structured wiring installed starting about 6 years ago. It was a major contributor to improved availability (stopped people messing with it). Have fiber installed on site. Expect single mode to be important due to size of site. Will connect hubs to switches. At moment switches are in data center, will be deploying switches in buildings. Using Alantec switches that have been very reliable (one port out of 3 switches failed in 3 years). Gbit Ethernet is the way forward. Looked at ATM on site a few years ago, but do not intend to expand. They expect link aggregation to be important which is going through standardization in the IEEE 802.3 WG (finish end of year). Daresbury is largely a class B network. On site routing is done on Alantecs. They only run RIP not OSPF. The SuperJanet routers will be all 3COM gear. They are not using VLANs since standard is only just finished. Two of the three Alantecs are tied together with redundancy (via spanning tree). Buying 3COM 3000 switches for building switches.
They have invested heavily in network management. Believe it helps a lot. Knowing where machines are is critical. Looking at DHCP. Not clear whether it will make things easier for complex networks.
Have monitor to look at traffic on the network. Also run HP OpenView, find it hard work, it is complicated with a huge learning curve, but worthwhile. They also find it hard to know how to size the system to run OpenView on. It is very hard to keep it up to date, they have not updated for 2-3 years. They also run Transcend from 3COM, they have but don't use NetMetrix, they use Powersite for Alantecs, and DEC's hub management package. Gather statistics for each segment, 30sec RMON statistics (can display as raw data or as 10 minute means and peaks), also create average hourly, daily etc. data for trends with all kinds of selections. Looking at how to characterize in terms of usability of threshold (e.g. thresholds for utilization and how often it goes over the threshold). One of the more useful packages appears to be NetHelp from Concord. The NMS is used by the network support folk (4 folks on rota to support network). Backend on NMS to ticketing system to record and send out email. They have an informal call-out arrangement. Pagers to the operator who check and may call someone from network support up to 10pm at night. Have consistency checking software to correlate DNS entries, property control, OpenView map, which segment something is on. They have also got software for displaying. The statistics are kept at:
http://netshp.dl.ac.uk/stats.html (raw data) & trends.html (for HP probes) For more information contact email@example.com.
Daresbury does not run DECnet. Native DECnet phased out. LocalTalk for a few Macs which can't upgrade.
Most security is reactive, want to be more proactive. Have reacted to denial of service ping attack. Running a large class B makes it worse than running a subnetted class B. Were seeing loads of 33Mbits on SuperJanet. DL does not have control over edge router so ACLs are put into Alantec.
Have a policy for SMTP to limited set of mail servers. They also block mail relay from outside to outside There is little central control at RAL. They had over 400 SMTP servers, now down to 50, by mid-April will block a lot more. Considering blocking IRC. No blocking for NFS or TFTP or spoofing. Glasgow & Oxford fixed spoofing.
They monitor for acceptable use, consist of 2 PCs with FDDI. They run Optimal Internet Monitor from Optimal networks, monitors FTP, HTTP, NNTP (looks inside packets, not just port) and produces a report with which sites are accessed. It is dumped each week, and run some analysis. Egregious inappropriateness can result in censuring of the individual (one was student who was banned from site, plus two others). There is a computing security officer for each site, but do not have much power (probably paid 2/3 time for security, and works 1/3 time on it). Security officer reports to Trevor Daniels (at RAL) the head of computing services. Experience of Grenoble was that the firewall was so onerous that people avoided it and placed themselves outside the firewall.
Daresbury, RAL & Oxford are moving to Microsoft Exchange. Glasgow looking at WebMail (a commercial product). They run a lot of POP into Microsoft Exchange. Oxford is happy with Microsoft Exchange.