At SSRL, we use VGM series VME single board computers by Synergy microsystems (featuring a G3/G4 PowerPC CPU) for the SPEAR3 electron storage ring control system.
On the software side, we use
EPICS (Experimental Physics and
Industrial Control System), running on the RTEMS (open-source) hard real-time OS.
Various software components of crucial importance are involved:
The RTEMS System
Development for RTEMS is done using the GNU tool chain.
One way for accessing remote files under RTEMS is by using TFTP. There is a TFTP filesystem available providing some abstraction from the TFTP protocol. Files on the TFTP filesystem are accessed simply by using the standard open(),read(),write() and close() routines.
Nevertheless, the TFTP filesystem has some restrictions which are discussed below. OTOH, the TFTP filesystem has the advantage that files can be served out of AFS.
This section describes how generic (i.e. non-EPICS) RTEMS is done. There exists a lot of information at the RTEMS/OAR website about the subject, hence this page is limited to a brief introduction and issues related to dynamic loading with CEXP.
NOTE: EPICS developpers may skip reading this section. The EPICS Makefile system transparently invokes the RTEMS/GNU tool chain. Generating EPICS applications is discussed below.
The relevant parts of RTEMS at SSRL are installed in AFS "package" space under
where the tools and rtems subdirectories are located (besides the prod directory, there is the src directory holding a checked-out tree of the RTEMS sources). As the names imply, the GNU toolchain is installed under tools (currently, the powerpc-rtems target is supported for linux-x86 and solaris-sparc hosts), RTEMS under rtems has been built for the mvme23xx and the svgm powerpc BSPs.
Traditionally, all parts of RTEMS were contained in a big static library which would be linked against the application code. (This has changed a little bit - as explained below.)
RTEMS comes with its own "Makefile system" which works as follows:
to your PATH, you can invoke the cross compiler as
hence you must set RTEMS_MAKEFILE_PATH to that directory. The SSRL customized Makefile.inc results in the SVGM BSP being used.
The SSRL template Makefile contains two different rules for the PGMS target:
where it will be found by the TFTP server for transmission to the target.
Note that there is also a
which should be used if your application depends on specific features of the VGM board.
A "traditional" application contains the application code and RTEMS itself - all symbol references have been resolved by the linker. Such an application is written (by the linker) into a 'xxx.exe' ELF file. The bootloader does not support loading ELF, however (the complete ELF file is also pretty big which would slow down loading). Therefore, the make process also generates a pure binary "xxx.bin" image of the application which is what must be presented to the bootloader. Also, a much smaller ELF file, "xxx.sym", is generated containing the application's symbol table information which can later be used by CEXP (CEXP is also capable of reading the symbols from the full-blown ELF file).
A loadable object OTOH is a normal ELF object containing lots of symbol and relocation information etc.Loadable objects only contain your application's code; they are not linked against RTEMS or libc (nor any of the libraries present in the "system" application). Note that loadable objects are loaded by CEXP (not the bootloader), which knows how to deal with it. Loadable objects get the "xxx.obj" suffix but any ordinary "yyy.o" file can as well be loaded. An ".obj" file is just a collection of ".o" files, linked together with the relocations preserved.
The /afs/slac/g/spear/rtemsApplications/system directory contains a relatively simple RTEMS application which merely fires up the system and starts CEXP. The "system" application is just "the" only generic application which is statically linked and booted. "Real" applications (e.g. EPICS applications) are built as loadable objects and added to the system at run-time. (GeSys is downloadable from outside of SLAC from this page.)
The "system" application consists of two configuration and one initialization source code files, a couple of linker scripts and a Makefile.
The configuration files take care of "system configuration" in a old-days UNIX or vxWorks sense, i.e. they define a bunch of static limits for various system resources (such as the maximal number of semaphores, tasks, networking buffers, file descriptors etc.). These limits are stored in so called 'configuration tables' which are consulted by the kernel to know how many resources to allocate for the system. Our configuration is pretty generous as the VGM board has plenty of memory and large EPICS applications might be loaded. However, it might turn out that some of the parameters need to be tuned in the future.
Then, there is completely separate aspect of "system configuration" that only is relevant when run-time loading is involved. A statically linked system always has all symbol references resolved by the linker which guarantees that all pieces of code needed by the application are pulled out of libraries, object files etc. and linked into the executable. A system with a dynamic linker, on the other hand, must be built in a different way: it is not enough to resolve all symbol references when putting the executable together as some symbols from the core libraries may be needed in the future (when loading objects into the running system). Assume, for example, that the "system application" does not use 'printf'. When linking against libc (last step of the 'system build' procedure), the static linker will thus not include the 'printf' routine into the executable and an attempt to loading a module which does call 'printf' at run-time will fail due to the unresolved reference to 'printf'.
Therefore, when building the "system application", the (static) linker must be forced to include vital parts of the core libraries into the system executable. What parts are considered 'vital' is the other aspect of "system configuration". This type of configuration is defined by a series of linker scripts found in the 'config' subdirectory of the "system application". These scripts simply list symbols to be considered 'undefined' by the linker who will then search the system libraries for resolving these forced references. The linker scripts are generated using the auxiliary 'Makefile.symlists' and must be manually edited/tailored by commenting out symbols which should not be included into the system build. Here's some more information.
The actual code executed by the "system application" is very basic: after performing some basic initialization, CEXP gains control, attempts to execute a couple of initialization/startup scripts and then waits for user input:
Note that failure to load a symbol file is fatal.
Booting RTEMS on a VGM board is done in four fundamental steps (FDIAG - SMON - netboot - RTEMS). In more detail, these are:
Note: when the interface between Netboot and the BSP changes (as it happened when support for DHCP/BOOTP vendor option 129 was added to RTEMS), Netboot needs to be upgraded to a version compatible with the BSP.
NOTE: The IP configuration of SMON, Netboot and vxWorks-bootrom are maintained in separate NVRAM regions and are hence completely orthogonal. SMON's IP configuration is only used/needed when reprogramming the FLASH (e.g. for installing or upgrading netboot).
A convenience script exists at
which can be loaded into VGM NVRAM by (xxx is to be substituted by the TFTP server path to the script)
loadEB "xxx/reflash.st" "reflash"
By simply executing the "reflash" script ('execEB "reflash" ') Netboot will be transparently upgraded to the latest version. (SMON's IP configuration has to be set up, though - use the 'config' command).
More detailed instructions about programming the VGM flash can be found here.
Working with netboot is pretty self-explaining. For changing the NVRAM configuration you have to hit a key (note that some keys invoke specific actions as listed/shown by netboot) before the timeout expires (this timeout is also NVRAM configurable). A good key to use is the space bar. Note that there are two countdowns/timeouts when booting a VGM. The first gives you a chance to abort the SMON startup script (not related to netboot), the second one is netboot's countdown.
By hitting the 'c' key, you can enter the 'edit/change' mode where you are prompted for all configurable parameters. There is command line history and full cursor control available. Some special <Ctrl> Key combinations are available for navigating through / committing / canceling your changes.
There are also keys available for temporarily changing netboots behavior:
These options are very practical for doing tests etc. when you don't want to touch the configuration but temporarily switch behavior.
The NVRAM configuration is now also accessible from a running RTEMS/GeSys systems using the 'nvramConfigShow()' and 'nvramConfig()' routines. Note that the actual, 'live' configuration of a board may differ from the configuration stored in NVRAM because the user might have used the 'm', 'd' or 'b' keys at boot-time. To obtain information about the current boot configuration, use 'bootConfigShow()'.
As already explained above, a user initialization script may be passed to the system application by providing 'netboot' with a 'command line parameter'. Examples:
Specify an absolute path on the 'server' (i.e. the same host where the boot file was loaded from):
Specify an absolute path on any host with a 'tftp' server:
Use a path relative to the directory where the system was loaded from.
Please note that vxWorks initialization scripts may need to be modified in order to be compliant with CEXP syntax.
Here's a dhcpd.conf file snippet...
back to top.