Test Stand Infrastructure for ATLAS RCE Development
General Information
A general introductory summary of ATCA infrastructure can be found on
Wikipedia.
For a typical test stand setup aimed at high bandwidth DAQ development, the following infrastructure components are needed:
- ATCA shelf (crate)
This has three components that are typically specified separately in purchases:
- Chassis: A key parameter for the chasiss is how many slots. With the typicaly processing power in each slot, the
simplest 2-slot shelf should be sufficient for most cases. Many full blown readout system can already be hosted in 5-6
slot shelves, while the full 14 slot shelves are only relevant for large systems. There are several advantages in
avoiding using the shelves with more slots than you need:
- More effective bandwidth on backplane fabric. There are several different types of backplane
network topology and ATLAS is in general
adopting the most versatile full mesh topology. The manufacturers typically aim for the 14 slot topology with 15
channel connectivity in each slot. When the shelf is reduced to e.g. 6-slots, these channels can be redistributed with
the link between each pair of slots using three channels. This forms the replicated-mesh so that for backplanes with
40Gb/s per channel capacity, this offers 3x40Gb/s between any pair of of the 6 slots simultaneously. A 2-slot shelf
can have 15x40Gb/s bandwidth between the two slots with the replicated mesh.
- Quieter fans. A large shelf with powerful loud fans can be a nuisance in a small lab.
- Lower cost.
The slot orientation can be horizontal or vertical which normally doesn't make much difference for test stands while for
large systems one needs to check the compatibility with rack cooling.
- Shelf manager: This is a controller board slotted at the bottom of the shelf for managing power. This
fully featured "smart" shelf manager is now necessary with the current version of RCE boards where power control through
IPMI is implemented.
- Power supply: ATCA shelves need 48V DC. This can be from a dedicated external distribution in a rack
for large systems while test stands typically just have an internal AC->48V converter plugged into wall power.
- Server computer
This can be just a typical desktop PC for office/lab. Having a bit more CPU power can be
helpful for compiling code during development. Some more disk space can be useful for some data taking activities. At
least one 1 Gb/s Ethernet port is necesssary and sufficient for many applications, while 10GE interface is needsed if one
would really like to explore the 10GE I/O.
- Network switch
It is recommended to have just a simple low end switch to isolate the test stand internal
network from the public network. This gives you some configuration freedom for the internal network while more importantly
to avoid acceidentally flooding the public network which the ATCA blade processors are fully capable of doing.
Test Stand ATCA Shelf Examples
There are several known vendors already used by various ATLAS upgrade R&D on ATCA. There are no significant funcationality
differences we know of. For entry level applications, the details are really not important but for those aiming for longer term
R&D for large systems, it is worth some additional investigation before following one the examples below for purchasing.
ASIS shelves used by SLAC
The ATCA shelves purchased by SLAC for RCE development and test stands are all from ASIS so far, with 40Gb/s compliant
full mesh backplanes. For test stands, one typically doesn't need to impmenent redundancy for shelf manager or P/S as in
real operational plants. The lead time for the shelf purchase is often severeal months. The relevant components and their part numbers are tabulated below. The pricing was a snapshoot for a particular batch of purchase at SLAC in 2013 while this is likely to evovlve for future purchase.
2-slot shelf
[Manual]:
| Item | Description | Part Number | Unit Price |
|---|
| Chassis | 2-slot 3U, horizontal | ATCA3U-010-4109 | $1949 |
| Shelf manager | Fully featured with Pigeon Point ShMM 500 | ATCAXXU-040-0023 | $1142 |
| Power supply | 1.5KW internal AC power | ATCAXXU-030-0004 | $531 |
6-slot shelf
[Manual]:
| Item | Description | Part Number | Unit Price |
|---|
| Chassis | 6-slot 6U, horizontal | ATCA6U-010-3308 | $3974 |
| Shelf manager | Fully featured with Pigeon Point ShMM 500 | ATCAXXU-040-0026 | $1142 |
| Power supply | 1.5KW internal AC power | ATCAXXU-030-0004 | $531 |
ELMA shelf used for IBL
The pixel IBL stave mounting test stand at Univ Geneva and CERN purchased test stand running RCEs used the
ELMA 3U 2-slot shelf.
For the custom configuration elections indicated in the datasheet,
the exact code was 11A02GJB38Y4HGBS, which implies:
- 2 slots
- 2xSM Radial IPMB
- Replicated mesh
- 2 xTray (plug in) cooling
- 3U height x 8U width x 400-499mm depth with rear I/O
- Horizontal card orientation
- 90-230V AC (plug-in)
- PSU output 1200-1299W
- 1 x Plug-in shelf manager
The purchasing experience so far had signifcaint higher prices for the ELMA 2-slot shelf compared to ASIS.
Other vendors
More vendors can be found on the manufacturer list on
this page .
Some useful links:
Mike Huffer
Su Dong
