Stop the Tape Habit

By Steve Hammond

Inside VESOFT covers tips and techniques you can use with VESOFT’s products, especially MPEX.

I’ve been in this business long enough to remember the days when disc was so expensive, you jammed as much as you could into a few bytes. When Y2K was the hot topic, I had to remind people that the reason we allocated only two digits for storage of the year in a date, was because we didn’t want to waste precious disc space on hundreds of thousands of occurrences of ’19.’ We just didn’t do it. Our mantra was “Tape is cheap, disc is expensive.”

For years I supported a classic batch-processing system that started with a job to store several databases to tape along with other essential data files. Then followed three more jobs, each with a STORE or DBSTORE at the beginning because the databases were updated in each job. This was before the concept of dynamic expansion of datasets. One of the jobs would add new records to a dataset, then delete some old records. So it was possible (and often happened) that the capacity of a dataset would be exceeded in the middle of the run, but when it completed, it would be below the original capacity. Unfortunately, in between, the job would have abended with a DBERROR 16, a restore of the database, an Adager step to increase the capacity and a restart of the job, which again began with the STORE or DBSTORE to tape. Of course, all this would happen somewhere in the wee hours of the morning, depriving me of one of my favorite activities — sleep.

As we got to those bigger, more powerful HP 3000s, disc got cheaper — much cheaper. We slapped in new disc drives the way George Steinbrenner buys relief pitchers, in multiples! But we were still doing the STORE/DBSTORE dance. I said enough is enough and turned to MPEX and its related JCWs to stop the tape habit and let me avoid midnight phone calls.

The first job still did a store to tape — we wanted to get one off-line store of the data for recovery reasons, but even there MPEX would make a difference. We did a complex STORE of files — databases and flat files, across account boundaries (running with SM capabilities). The STORE statement specifically excluded UDC, PHD and ODBCLOG files, all of which would be open and were not crucial to the process. But any other files in the fileset that could not be stored would call for us to redo the job. Therefore, I used MPEX to do some preliminary checking before the STORE:

!run MPEX.PUB.VESOFT;parm=1
%LISTF @.@.prod-udc.pub.prod-phd.pub.prod-odbclog.pub.prod, ACCESS
% exit
!showvar MPEXNUMSUCCEEDED
!if MPEXNUMSUCCEEDED > 0
! tellop *****
! tellop job terminating — files are open
! tellop *****
! eoj
!endif
!continue

So what’s that all about? Simple, I took the same set of files in the upcoming STORE (excluding the specific files, as I would in the STORE) and did a LISTF, ACCESS on them. The MPEXNUMSUCCEEDED is just that — the number of files that MPEX successfully found in the prior command. If that value is anything more than 0, then someone was accessing a file in that fileset and we don’t want to even proceed to the STORE.

In the subsequent jobs, I would MPEX COPY the databases to a separate group corresponding to the job (the luxury of plentiful, cheap disc space). At the end of the sequence of four jobs, I would have four versions of the databases on disc (one in production — PUB — and three in the separate backup groups) and one on tape. In the old days, more than one copy of a database online was downright wasteful. Now it’s just the smarter way to operate.

The subsequent jobs did get a bit more problematic because of some security issues. For various reasons, the jobs ran under a non-AM user. Therefore, it could not MPEX COPY the databases because one needs AM capability to do that. I solved that issue with an MPEX command file that I created in PUB of the account. No one other than programmers had a colon prompt into PUB, so we felt that was a low enough risk to do it. The command file, which I called ‘copydb’ was:

parm parm1,parm2
withcaps “AM”, copy !parm1@(code=‘PRIV’),=.! parm2;yes;
creator=‘MANAGER’

The two parameters passed are the name of the database root file and the group to copy the database into. The command WITHCAPS declares that the reset of the command in this line may be done with the cited, in this case AM, capability. So parm1 declares the database name, and it is plugged into the COPY command selecting just PRIV file codes. The database is copied using the same name (“=”), in the group selected the second parameter and the creator is changed to MANAGER. There is one more very important facet of this I will discuss at the end of the column.

The command would look like COPYDB PRODDB, RUN1BKUP which meant to copy all of PRODDB database into RUN1BKUP group.

Each of the three subsequent jobs would copy two to four databases into its corresponding backup group. I counted the number of datasets to be copied to backup in each run and made note of the number. The jobs, which were running in the PUB group, then included:

!run MPEX.PUB.VESOFT
setvar success 0
setvar fail
copydb DBA,JOBXBKUP
setvar success=mpexnumsucceeded + success
setvar fail=mpexsumfailed + fail
showvar mpexnumsucceeded, mpexnumfailed, success, fail
...
exit
!if success <>nn or fail > 0
! tellop *****
! tellop copy failed, databases not stored
! tellop job terminating
! tellop *****
! eoj
!endif

The ‘fail’ value is obvious — if any COPY failed, then one cannot proceed. The ‘success’ fail is more interesting. The value of ‘nn’ is the precalculated (by me) number of files (root files plus datasets) that should be copied. If the value of ‘success’ is fewer than ‘nn’, then it’s an obvious failure.

But why must it be exactly that value — would greater be okay? Well, no. If a database is open, the DBGB would be stored, obviously an event we don’t want to happen (we didn’t do the preliminary LISTF, ACCESS as we did in job 1). Also on more than one occasion, a dataset was added to the database, causing this failure. Not a crucial circumstance, but still we wanted to keep things tidy. Remember, in “Jurassic Park” their first failure was to see if there might be more than the expected number of dinosaurs, so they never realized they were breeding.

Now, finally, what about that WITHCAPS command. You just can’t slap a WITHCAPS into a command and expect it work — otherwise, you’d have programmers throwing WITHCAPS “SM” into everything just to get system manager abilities. You need to include the name of the command file in the file STREAMX.DATA.VESOFT (so you do need to own SECURITY/3000 along with MPEX to do this). The file would include:

$WITHCAPS-PERMIT user.account COPYDB.PUB.ACCOUNT “AM”

You can use wildcards for both users and accounts, but there should be an entry in this file for every user you expect to utilize the command file.

Steve Hammond, who works for a professional association in Washington, DC, will only state his age in hexadecimal.