SLAC CPE Software Engineering Group
Stanford Linear Accelerator Center
System Admin

March 2011 Downtime

 

SLAC Detailed
SLAC Computing
Software Home
Software Detailed
 

 

 


 

On March 8, 2011 at around 0600 the Accelerator will be shutdown.  The operators will have to wait an hour for the accelerator to cool before they can begin their checks.  Controls will coordinate with the EOIC and shutdown the Control system around 0830 to 0900.  Once the Computers are all shutdown, this should take between 30-45 minutes, Controls will inform the EOIC and they will coordinate with the electricians to shutdown power to DP6A1.

Immediately after MCC PDU shutdown the electricians will begin installing the 24 circuits into the PDU.  All preliminary work has been completed, so this should take about 8 hours.  Once these circuits and racks have been installed we can energize the PDU and start bringing up our servers.  If everything goes as planned we could have our servers back up on Thursday.

4 Racks will be removed and new ones installed during the downtime.  This will give us valuable rack space for future expansion.  Each Rack will have 4 new circuits added to provide power.   The new SLCNET chassis will be installed into the new racks.

During the downtime the electricians will be energizing a cut-over power switch in building 5 which will feed MCC Computer room, Networks, and A/C units.  This power switch will allow an electrician to safely switch the power source for the MCC Computer room from the primary power to the secondary power source coming from Building 7.  This will allow the MCC Servers, networks, and A/C unit to continue to run in the case of a power failure on our primary source.  At the end of the downtime we will shutdown the MCC Computer room and test the cut-over switch.

I expect 2-3 days of no power to our MCC Computer Room, but if there are problems this can be extended.  This means all production servers will be down and any development servers housed in MCC such as lcls-dev2 will be down also.  NLCTA servers will be affected also.

We will install and begin testing our NFS server during this downtime.  This is a fully redundant NFS server with high availability for MCC operations.  Integration will occur later.

 

    1. Install cut-over power switch (electricians) (2-3 days)
      • Provide power to MCC PDU
      • Provide power to MCC Computer room A/C
    2. Install Racks and outlets in MCC Computer room (Electricians/carpenter) (1-2  days)
      • 4 Racks       ( 2 days –but no impact to bringing up servers)
      • 24 Circuits  (~ 8 hours)
      • Install new SLCNET Gateways    (~4 hours)
    3. Update Server OS
    4. NFS server Installation and test
    5. Defrag VMS disks
    6. Upgrade Servers OS

 

 

Author: Ken Brobeck   10-Feb-2011

Modified: 07-Mar-2011