Managing UNCMPHST disks on VMS

LOGICALS

File to modify: SLAC$COMMON:[MON_DISK_SPACE]MON_DISK_SPACE_START.COM

• Logical: FREE_LIM_NOCHKLIST

"FREE_LIM_NOCHKLIST" [super,clusterwide] = "/UNCMPHST4/UNCMPHST5/UNCMPHST6/UNCMPHST7/UNCMPHST
8/UNCMPHST9/HISTRY03/HISTRY04/HISTRY08/HISTRY10/HISTRY11/SWAPNPAGE1/SWAPNPAGE2/SWAPNPAGE6/PHYSI
CSA/PHYSICSB/PHYSICSC/PHYSICSD/"

 

File to modify: SLCCOM:ASSIGNDIR.COM

• Logical: HST_UNCMP_DATA_VMS

HST_UNCMP_DATA_VMS  disk$uncmphst12:[PRODUCTION.HISTORY],-    ! this first disk gets new files weekly

                                                disk$uncmphst11:[PRODUCTION.HISTORY],-

                                                disk$uncmphst10:[PRODUCTION.HISTORY],-

                                                disk$uncmphst9:[PRODUCTION.HISTORY],-

                                                disk$uncmphst8:[PRODUCTION.HISTORY],-

                                                disk$uncmphst7:[PRODUCTION.HISTORY],-

                                                disk$uncmphst6:[PRODUCTION.HISTORY]

 

When you remove a disk from the HST_UNCMP_DATA_VMS list you have to add the corresponding disk directory to the HST_UNCMP_DATA_NFS list of directories. Otherwise that data can no longer be seen by the SCP.

• Logical: HST_UNCMP_DATA_NFS

   

   

  HST_UNCMP_DATA_NFS  DISK$NFS_SLCDATA:[PRODUCTION.HST5],-

                                                DISK$NFS_SLCDATA:[PRODUCTION.HST4],-

                                                DISK$NFS_SLCDATA:[PRODUCTION.HST3],-

                                                DISK$NFS_SLCDATA:[PRODUCTION.HST2],-

                                                DISK$NFS_SLCDATA:[PRODUCTION.HST1]

Files to modify:

CLU$COMMON:[SYSMGR]DEFINE_PREFER_HSC.COM

CLU$COMMON:[SYSMGR]SYLOGICALS.COM

CLU$COMMON:[SYSMGR]SYMOUNT_DISKS.COM

 

Sparcified Data:  (PROD$HIST) & HIST_ARCHIVE

The specific history data at issue is summarized below --

 

Sparsified archive files on HIST_ARCHIVE disks:

disk                 blocks       years included

----                 ------       --------------

disk$histry16   35500K  Dec 2015 -01-Jun-2021

disk$histry15   8500K    Sep 2014-Nov 2015

disk$histry14    8500K    Apr 2013-Aug 2014

disk$histry13    8093K    2011-Mar 2013

disk$histry12   16300K  2009-2011

disk$histry11   32000K  2004-2009   Moved to DISK$NFS_SLCDATA: [prodhist.histry11]

disk$histry10    8500K  1998-2004   Moved to DISK$NFS_SLCDATA: [prodhist.histry10]

disk$histry08   35500K  1997-2003    Moved to DISK$NFS_SLCDATA: [prodhist.histry8]

disk$histry04    9100K  1993-1997  -Moved to DISK$NFS_SLCDATA: [prodhist.histry4]

disk$histry03    8300K  1994-1997  -Moved to DISK$NFS_SLCDATA: [prodhist.histry3]

 

As of 10-Apr-2023 MCC::BROBECK>sho log hist_archive "HIST_ARCHIVE" = "DISK$HISTRY16:[PRODUCTION.]" (LNM$GROUP_000044) = "DISK$HISTRY15:[PRODUCTION.]" = "DISK$HISTRY14:[PRODUCTION.]" = "DISK$HISTRY13:[PRODUCTION.]" = "DISK$HISTRY12:[PRODUCTION.]" = "DISK$HISTRY11:[PRODUCTION.]"

 

 

MCC::BROBECK>dir nfs1:[000000.prodhist]

Directory NFS1:[000000.PRODHIST]

HISTRY10.DIR;1 HISTRY11.DIR;1 HISTRY3.DIR;1 HISTRY4.DIR;1
HISTRY8.DIR;1

Total of 5 files.

 

-Moved Data from oldest disk (disk$histry03) to DISK$NFS_SLCDATA: [prodhist]

 

Do NOT delete or copy     *_x.hstdat     *_c.hstdat   *_w.hstdat ---These are files used to collect the data -then move to files *_$*.hstdat

 

 

 

-Then recycled disk$histry03 to disk$histry13 ( set volume/label=histry13 $255$dua46 )

-Modified symount_disks.com and assigndir.com (hist_archive logical)

-Copy over old data from prod$hist (dua5) to new disk 

EX:

• cd $255$dua5:[production.history]
• backup/log/before=01-jan-2012 *_$*.hstdat; $244$dua46:[production.history]
• delete  /log/before=01-jan-2012 *_$*.hstdat;

MCC::BROBECK>sho log hist_archive
"HIST_ARCHIVE" = "DISK$HISTRY15:[PRODUCTION.]" (LNM$GROUP_000044)
= "DISK$HISTRY14:[PRODUCTION.]"
= "DISK$HISTRY13:[PRODUCTION.]"
= "DISK$HISTRY12:[PRODUCTION.]"
= "DISK$HISTRY11:[PRODUCTION.]"
= "DISK$HISTRY08:[PRODUCTION.]"

 

 

and, sparsified archives file still on production disk:

PROD$HIST       45300K  2011-2019

 

the usage for one year might be typical of furture usage?:

PROD$HIST        6000K  2018 only

 

The incremental approach would be to delete the oldest data from disk$histry* disks and use that space to

offload the oldest files from PROD$HIST.  For example, the disK$histry03 and disK$histry04 space would be

enough to get about 3 years of future use (if future use is similar to 2018 use).

 

General Info

Re:  Details of the NFS reorganization of "uncompressed" history files

 

  I have copied most of the VMS uncompressed history files from VMS to the NFS disk.  I

  decided to transfer files from each VMS disk to its own unique directory on the NFS disk.

  Thus, for n = 1 to 7 (and part of n = 8) I have copied

 

    DISK$UNCMPHSTn:[PRODUCTION.HISTORY]  to  DISK$NFS_SLCDATA:[PRODUCTION.HSTn]

 

  I think we should maintain this pattern in the future (for n = 8,9,10...).

 

  For history access to uncompressed files from MCC, the logicals are set up so as to get the

  more recent files from VMS, the older files from NFS.  Currently this is set up with

  n = 1,2 from NFS, n = 3,4,...8 from VMS.  However that could be changed any time to

  get up to n = 7 from NFS (and the remainder from VMS).

 

  The VMS logical names that control this file access are defined in CMS_COM_GEN

  ASSIGNDIR.COM as follows:

 

$

$         OUR_DEFINE   HST_UNCMP_DATA_VMS  disk$uncmphst12:[PRODUCTION.HISTORY],-    ! this first disk gets new files weekly

 

                                           disk$uncmphst11:[PRODUCTION.HISTORY],-

                                           disk$uncmphst10:[PRODUCTION.HISTORY],-

                                           disk$uncmphst9:[PRODUCTION.HISTORY],-

                                           disk$uncmphst8:[PRODUCTION.HISTORY],-

                                           disk$uncmphst7:[PRODUCTION.HISTORY],-

                                           disk$uncmphst6:[PRODUCTION.HISTORY]

$!!!                                    !! disk$uncmphst5:[PRODUCTION.HISTORY]

$!!!                                    !! disk$uncmphst4:[PRODUCTION.HISTORY]

$!!!                                    !! disk$uncmphst3:[PRODUCTION.HISTORY]

$!!!                                    !! disk$uncmphst2:[PRODUCTION.HISTORY]

$!!!                                    !! disk$uncmphst1:[PRODUCTION.HISTORY]

$

 

When you remove a disk from the HST_UNCMP_DATA_VMS list you have to add the corresponding disk directory to the HST_UNCMP_DATA_NFS list of directories. Otherwise that data can no longer be seen by the SCP.

 

$         OUR_DEFINE   HST_UNCMP_DATA_NFS  DISK$NFS_SLCDATA:[PRODUCTION.HST5],-

                                           DISK$NFS_SLCDATA:[PRODUCTION.HST4],-

                                           DISK$NFS_SLCDATA:[PRODUCTION.HST3],-

                                           DISK$NFS_SLCDATA:[PRODUCTION.HST2],-

                                           DISK$NFS_SLCDATA:[PRODUCTION.HST1]

 

$!

$!  NOTE:  logical HST_UNCMP_DATA will/will-not include HST_UNCMP_DATA_NFS: when the NFS

$!         disk is/is-not available, as controlled by SLCCOM:MON_NFS_SLCDATA.SUBMIT

$!         Here, if HST_UNCMP_DATA is already defined, we leave it untouched;

$!         but if it is NOT defined (typically, at boot time), we define it WITHOUT

$!         HST_UNCMP_DATA_NFS:, and leave it up to SLCCOM:MON_NFS_SLCDATA.SUBMIT

$!         to add that if and when appropriate.

$!

$         if  f$trnlnm ("HST_UNCMP_DATA",lnmtable_name) .eqs. ""

$         then

$           OUR_DEFINE   HST_UNCMP_DATA   HST_UNCMP_DATA_VMS:

$!!!        OUR_DEFINE   HST_UNCMP_DATA   HST_UNCMP_DATA_VMS:,HST_UMCMP_DATA_NFS:  ! done in MON_NFS_SLCDATA.SUBMIT

$         endif

$

 

  Not too long from now (a month or so?) DISK$UNCMPHST8 will be full.  If you have enough  free disk space, you could create a new disk  DISK$UNCMPHST9  and add it to the top  of the list in HST_UNCMP_DATA_VMS in ASSIGNDIR.COM.  Alternatively, you could make sure the DISK$UNCMPHST1 files are backed up, delete the old files, then change its volume label from UNCMPHST1 to UNCMPHST9, and then use that as the new uncompressed history disk.

  (To change the volume label I think you need to DISMOUNT cluster-wide, MOUNT again privately,   then use the SET VOLUME/LABEL= command (see HELP), dismount, then mount cluster-wide with the new label.)

  Note that a watchdog runs weekly (on Tues) to estimate when DISK$UNCMPHST8 will fill up.

 

Since we started using your new "unsparsified" history disk DISK$UNCMPHST9, I have now

copied all of the DISK$UNCMPHST8 files to the NFS disk.  Presumbably we will be doing this

again in about 8 months, and at similar intervals afterwards.

 

I've created three SLCCOM: procedures to make these future copying of files to NFS

easy to do:

 

        SUBMIT_NFS_COPY_HSTDAT.COM -- procedure to submit a batch copy job

        NFS_COPY_HSTDAT.COM        -- the batch procedure submitted by the above

        NFS_COMPARE_HSTDAT.COM     -- a procedure to verify that all files were copied

 

The batch procedures handles some details so we don't have to think about them:

        running the job under SLCSHR (which is only username with NFS WRITE priv)

        running the job on MCCDEV (which is only node that has the disk mounted for WRITE access)

        Assuring that any files previously copied are NOT re-copied

 

I've added reminders about the existence of the above procedures both in ASSIGNDIR.COM and also

in the watchdog that warns you when the current DISK$UNCMPHSTn disk is nearly full.