Soft IOC Release Procedure

LCLS Soft IOC Release Procedure

I. Releasing a new soft IOC to the production network

II. Adding a new soft IOC to an existing application

===============================================================================

I. Releasing a new soft IOC to the production network

These instructions tell you how to release a new soft IOC to the production network. Making a mistake at some point in this process could cause a lot of problems in the production control system. If you are new at this or if you don't fully understand all the steps, you should have somebody helping you to make sure you don't make any mistakes. Be very careful!

Major steps (details below):

1. Set up directories and create the soft IOC in the development environment.

Boot it and test it on the development server as much as possible.

2. Import the new IOC into CVS.

3. Go to the production network (lcls-builder). Create a new directory for your new IOC under iocTop. Check out your IOC from CVS here.

4. Create a data directory for your IOC.

5. Create a boot directory under iocCommon for your new IOC. Set it up to point to the new IOC you just set up under iocTop.

6. Add your new IOC to screeniocs file, the configuration file for iocConsole.

7. Use iocConsole to boot your IOC.

8. Set up your IOC to use the channel watcher if needed.

9. Use CVS to create an official tagged release of your IOC.

10. Check your tagged release of the IOC out of CVS into your IOC's directory under iocTop.

11. Change the symbolic link under iocCommon to point to your new tagged release.

12. Boot your IOC again using iocConsole and test.

13. Ask Jingchen to create scripts on the appropriate host (currently lcls-daemon2) to automatically start your IOC when the server boots.

14. Copy displays to $EDM and commit to CVS.

15. Ask Judy Rock to add PVs to the alarm handler as needed.

16. Ask Shashi Divekar to add PVs to the channel archiver as needed.

17. Ask Judy Rock to add your new IOC to the Network section of LCLSHome.

18. Send a message to controls-software-release to announce your new IOC.

Details:

1. Set up directories and create the IOC in the development environment. Boot it and test it as much as possible.

a. Log into development environment:

Log into lcls-dev2.

If your shell default is tcsh, then type bash to change to bash shell.

source /afs/slac/g/lcls/epics/setup/epicsReset.bash (or add this to your .bashrc)

b. Create application directories in the development environment:

* make a development directory for your application:

cd $EPICS_IOC_TOP (/afs/slac/g/lcls/epics/iocTop)

mkdir <moduleName> (for example Alarms)

cd <moduleName>

mkdir Development

cd Development

(to work on an existing application, cd <moduleName> and cvs checkout the desired release)

c. Create the IOC using the makeBaseApp.pl script. Use the "lcls", "ioc", or "example" template. For example:

makeBaseApp.pl -t lcls -a linux-x86 <appname>

makeBaseApp.pl -i -t lcls -a linux-x86 <appname>

d. Get a good copy of configure/RELEASE from one of the development IOCs and modify it so that it includes any modules that your IOC needs.

Make sure EPICS_SITE_TOP is set to the development directory: /afs/slac/g/lcls/epics

d. Modify configure/CONFIG so that your IOC application will be built only for linux.

e. Modify <appname>App/src/Makefile to load libraries and dbd files from the modules your IOC needs.

f. Rename the directory iocBoot/ioc<appname> to iocBoot/sioc-<area>-<sys>##, following the naming convention

g. Create databases and displays and modify st.cmd as needed.

h. Make displays. These can be optionally saved with the IOC under <appname>App/srcDisplay.

i. Build the IOC.

j. Now create boot and data directories in the development environment:

* make a boot directory:

cd $IOC

mkdir <dev iocnodename> (for example sioc-sys0-al0d)

cd <dev iocnodename>

create bin and st.cmd links (again, see examples in production $IOC)

create a startup.cmd file (see example in $IOC on production)

use Dev versions of pre- and post-st.cmd scripts:

# Execute dev soft_pre_st.cmd script

< /afs/slac/g/lcls/epics/iocCommon/All/Dev/soft_pre_st.cmd

...

# Execute dev soft_post_st.cmd script

< /afs/slac/g/lcls/epics/iocCommon/All/Dev/soft_post_st.cmd

* create data and iocInfo directories for your ioc

cd $LCLS_DATA/epics/ioc/data

if there is no data directory for your ioc already, create one

mkdir <iocnodename>

cd <iocnodename>

mkdir iocInfo

k. Add your ioc to the development section of the $IOC/screeniocs file so that it can be booted from iocConsole.

l. Now you're ready to test away!

2. Import the new IOC into CVS.

a. Choose a module name for the IOC. Module names are stored in $CVSROOT/CVSROOT/modules - see the names there for examples.

b. Check out the modules file from CVS into a temporary working directory: cvs co -d <temporarydirectoryname> CVSROOT/modules

c. Edit your local copy of modules file and add your new module. The location of your IOC in CVS will be epics/ioc/<modulename>.

d. Commit the modules file back to CVS: cvs commit -m "add new ioc <modulename>" modules

e. Import your IOC into CVS. From the top directory of your IOC application, type:

make clean uninstall

cvs import epics/ioc/<modulename> CD_SOFT R1_0

f. Check that your import was successful. In a working directory, type: cvs co -d <anything> <modulename>. Then cd <anything> and build. Run the IOC to test.

3. Go to the production network (lcls-builder). Create a new directory for your new IOC under iocTop. Check out your IOC from CVS here.

a. From mcclogin, ssh iocegr@lcls-builder.

b. Set the environment variable CVSROOT: export CVSROOT=:ext:<username>@mcclogin:/afs/slac/g/lcls/cvs

c. Have Jingchen or Ernest create a directory for your project under $EPICS_IOC_TOP (/usr/local/lcls/epics/iocTop)

d. cd <modulename>

e. Check out your IOC from cvs: cvs co -d Devl <modulename>

f. cd Devl

g. modify configure/RELEASE so that EPICS_SITE_TOP=/usr/local/lcls/epics

h. make

i. Boot the IOC and test. Your are running your IOC on the production network. Be careful to avoid PV name collisions!

4. Create a data directory for your IOC.

a. Have Jingchen or Ernest add a data directory for your IOC. They will do steps a1, a2.

a1. cd /u1/lcls/epics/ioc/data

a2. Create a directory with your IOC's nodename. This nodename must match the name of the subdirectory under iocBoot in your IOC: mkdir <nodename>

b. cd <nodename>

c. If it was not already done, create iocInfo, Channel Watcher, and autosave subdirectories:
mkdir iocInfo
mkdir autosave
mkdir autosave-req
mkdir restore

5. Create a boot directory under iocCommon for your new IOC. Set it up to point to the new IOC you just set up under iocTop.

5. Ask Jingchen or Ernest to create a boot directory under iocCommon for your new IOC. They will do steps a1, a2.

a1. cd /usr/local/lcls/epics/iocCommon

a2. Create a boot directory for your IOC. The name of this directory must following the IOC naming convention, and it must match the subdirectory under iocBoot. Use sioc-sys0-nw01 as a template: cp -r sioc-sys0-nw01 sioc-<area>-<sys>##.

b. Go to your new directory: cd sioc-<area>-<sys>##

c. Delete the bin/ symbolic link: rm bin.

d. Create new symbolic links:

bin directory:

ln -s /usr/local/lcls/epics/iocTop/<modulename>/Devl/bin/linux-x86 bin

st.cmd

ln -s startup.cmd st.cmd

e. Modify the startup.cmd file. Replace the date and username with your own. Modify the line beginning with cd /usr/local/lcls ..... so that it goes to your IOC's boot directory.

Modify #!bin/IOCManager and replace IOCManager with your IOC executable.

f. Execute st.cmd to verify that your IOC boots: ./st.cmd

g. Verify that your IOC's info is being written to the data directory: ls /u1/lcls/epics/ioc/data/<nodename>/iocInfo You should see several files with names starting with IOC.

6. Add your new IOC to screeniocs file, the configuration file for iocConsole.

a. Have Jingchen or Ernest edit screeniocs to add your new IOC to iocConsole.

7. Use iocConsole to boot your IOC.

a. Use iocConsole to boot your IOC: iocConsole <nodename>

8. Set up your IOC to use the channel watcher if needed.

a. <details to be added later>

9. Use CVS to create an official tagged release of your IOC.

a. Go back to the top directory of your IOC: cd /usr/local/lcls/epics/iocTop/<modulename>/Devl

b. Commit any changes you have made to cvs: cvs commit -m "prepare for initial release"

c. Create a cvs tag for your IOC: cvs tag <modulename>-R1-0-0

10. Check your tagged release of the IOC out of CVS into your IOC's directory under iocTop.

a. Go up one level, to your <modulename> directory: cd ..

b: Check out your new tagged release from CVS: cvs co -d <modulename>-R1-0-0 -r <modulename>-R1-0-0 <modulename>

c. Go to the top of the new tagged release of your IOC: cd <modulename>-R1-0-0

d. Build: make

e. Test: cd iocBoot/<nodename> then ./st.cmd

11. Change the symbolic link under iocCommon to point to your new tagged release.

a. cd /usr/local/lcls/epics/iocCommon/<nodename>

b. Remove the symbolic link bin: rm bin

c. Create a new symbolic link pointing to your tagged release: ln -s /usr/local/lcls/epics/iocTop/<modulename>/<modulename>-R1-0-0/bin/linux-x86 bin

12. Boot your IOC again using iocConsole and test.

a. iocConsole <nodename>

b. If you are happy with your IOC and want to start running it on the production server immediately, run iocConsole with the -stayup option: iocConsole -stayup <nodename>. To exit iocConsole and leave the IOC running, type ctrl-a then ctrl-d. This is the way you must always exit iocConsole for a production IOC, whether it is a soft IOC or a VME IOC. You must never type exit or ctrl-c or ctrl-y in a production soft IOC, because this will kill the IOC and cause it to reboot. Watch out! This can be very disruptive to operations!

13. Ask Jingchen to create scripts on lcls-daemon1 or lcls-daemon2 to automatically start your IOC when the server boots.

a. Send Jingchen an email with the nodename of your new soft IOC. Ask him to add scripts to boot your IOC on lcls-daemon1 or lcls-daemon2.

b. After Jingchen has done this, you will no longer use iocConsole -stayup to boot your soft IOC. Instead you will log into lcls-daemon1 or lcls-daemon2 with the group account "laci". From mcclogin, ssh laci@lcls-daemon1 or ssh laci@lcls-daemon2. To start or stop your IOC on the daemon machines, type /etc/init.d/st.<nodename> start or /etc/init.d/st.<nodename> stop or /etc/init.d/st.<nodename> restart. Another way to restart your IOC, as noted above, is by typing exit inside of iocConsole. This is a perfectly fine way to reboot an IOC.

14. Copy displays to $EDM and commit to CVS.

a. All production EDM displays are stored in one of the subdirectories under $EDM. The subdirectory $EDM/lcls is reserved for files related to the top level lclshome display, so don't put your files there. Instead find the appropriate subdirectory for your subsystem and copy the files there: cp *.edl $EDM/<subdirectory>.

b. Go to the subdirectory and add the files to CVS and commit: cd $EDM/<subdirectory>; cvs add <filename>; cvs commit -m "initial import (or other reason)" <filename>

15. Add PVs to the alarm handler as needed.

16. Ask Murali Shankar to add PVs to the channel archiver as needed.

If you want to add PVs to the archiver, send Bob an email with a list of PVs and the name of your subsystem. You also need to specify the method that the alarm handler should use to watch these PVs; typically "monitor" is fine for all PVs. You will also need to specify the rate at which the archiver might expect to see the data change; typically 1 second is fine.

17. Ask Judy Rock to add your new IOC to the Network section of LCLS Home.

Send Judy an email with the name of your new soft IOC. She will add it to the Network pages under lclshome, and to her soft IOC that monitors all the production IOCs.

18. Send a message to controls-software-release to announce your new IOC.

Your release message will automatically be written to the software section of the operations elog.

----------------------------------

You're done!

----------------------------------

===============================================================================

II. Adding a new IOC to an existing application

in Development:

1. Check out the application in dev, and add the IOC directory

a. on lcls-dev2, checkout the default version of the app (e.g. Alarms)

cvs co -d Development <app name>

b. cd Development/iocBoot

c. mkdir <new ioc nodename> (e.g. sioc-sys0-al01)

d. Copy Makefile from an existing IOC into the new iocBoot dir

e. Create (or clone) st.cmd for the new IOC in the new iocBoot dir

2. in $IOC create boot directory and files and symbolic links for the new IOC - see section I above

3. in $IOC_DATA create IOC data directory for the new IOC - see section I above

4. boot the new IOC and make sure that it boots, and that iocInfo files are created in the right place - see section I above

5. Add the new IOC to CVS:

a. cd Development/iocBoot

cvs add <ioc nodename>

cvs commit

cd <ioc nodename>

cvs add *

cvs commit

in Prod:

6. cvs checkout on prod

7. create boot directory and files and symbolic links for the new IOC

8. create data directory for the new IOC

9. boot the new IOC to make sure it works

10. add new IOC to screeniocs, and cvs commit this file

11. create ChannelWatcher config file for the new IOC, and edit Master.XX Channel Watcher master file to include the new IOC's config file and data dirs.

12. Make a new tagged version on prod with the new IOC and point boot links there - see section I above for details

13. Ask Jingchen to create a new st.<ioc nodename> for the new IOC.

===================================================================

last modified by S. Hoobler 4/6/2012 16:23 PM