Lund Performance Solutions
240 2nd Avenue SW
Albany, OR 97321
Tel: 541.926.3800
Fax: 541.926.7723
E-mail: info@lund.com
Web: www.lund.com

Shadow D/R includes the software required for both systems to enable you to shadow to a second machine. The NS services that it makes use of are included with the base operating system, so no further software is required.

In a less-than-crowded marketplace for data replication products, Lund has revived Shadow from its acquisition of the Carolian products a few years ago. They’ve blown off the dust, fixed the bugs, added some enhancements and released it as Shadow D/R. The concept behind Shadow is that it will perfectly copy data in an HP 3000 IMAGE/SQL database between two or more HP 3000s.

Shadow only works with IMAGE/SQL in this release — no Allbase, KSAM or MPE files can be shadowed. For some people that’s a problem — for many others it’s not. By shadowing your data you can buy a number of things very easily: offsite, realtime backup of your critical data, and distributed processing that allows you to load balance between machines. Imagine a batch-only machine where reports could duke it out without affecting online users.

How does it work?

Shadow D/R is implemented by taking advantage primarily of IMAGE logging. This feature comes with IMAGE, and will maintain a transaction log separate from the database of each transaction. You can control this yourself to a certain degree by using DBBEGIN..DBEND pairs in your code, but it’s not necessary. Shadow makes use of NS services to transport the data to the shadow machine, and rolls the transactions out of the log file and into the database. From this perspective Shadow doesn’t have much to do, but it’s more complex than that.

There are monitor processes on both systems to keep track of the ‘heartbeat’ between the two systems making sure that communication is always there. If you lose your connection for some reason, like a system going down, then Shadow will alert the still-active machine. There is a high-water mark that you can set for how many transactions can queue on the master machine before Shadow will start archiving to tape. This is based on the assumption that both machines could go down at any time once the first machine goes down. This is an interesting feature, and a good idea for disaster recovery purposes.

There is typically one background job for each account/group in which you want to shadow databases, which also means there is a config file for each. Take a look at Figure 1 for an example config file. The format is rather similar to a schema file, but here is where you specify how and where the remote system is located, the database to shadow, the name of the log file that will contain the information, and how the job should log on.

BEGIN << Sample SHADOW configuration file >>

LOCAL=!JOB shadow,mgr/nimbus.smga,pub;hipri;outclass=lp,1,1;pri=bs:

COMMANDS = shadowp1.pub.smga: << Pipe to CP >>
RESPONSES = shadowp2.pub.smga: << Pipe from CP >>
SAVEFILE = shadowsv.pub.smga: << Saves environment >>

USE logproc WITH dumb.pub.smga/FUGAZI:

MODE = 1: << mode=1/locking >>
WAIT = 6000: << ten seconds >>
MAXRECS = 1000: << before requesting tape >>
DSLINE = OMEGA: << 1 LAN Lines >>

REMOTE=hello shadowsp,mgr.smga;pri=bs;hipri:

LOG = logfl001.pub.smga:

TAPEDEV = tape: << Tape device class >>

END. << of configuration file >>

Features

Shadow has a number of handy features for use in a production shop. There is the BACKUP command, which will suspend updates on the shadow system to allow you to back it up. All changes will queue on the master system until a RESUME command is issued. This can give you theoretical 24x7 access to your production database if you aren’t concerned about the records that queue up during the backup period.

As mentioned earlier, Shadow will start archiving to tape if the logs hit their high-water marks. The RECOVER command will roll data into the database from the tape. Their are a couple of options to tell Shadow whether to drop incomplete transactions or not.

There are a number of other commands to do things such as start and stop Shadow configurations, see what is running in Shadow, or verify a config file. Nothing very glamorous, but it gets the job done. One thing that Shadow would benefit from in this regard is a unified config and maintenance program, as opposed to the loose config files laying around.

Installation and Documentation

Installation is very easy. Your standard RESTORE and job stream can configure the accounting structure. The manual is very well written and easy to follow. It’s slightly over 120 pages, but this includes a table of contents, an index, and a number of appendices with things like the error messages and frequently asked questions.

The manual is arranged to give you first an overview of what shadowing is, why and how you would use it, and how it works. It then moves logically forward to how to configure the system, how to use it, and a description of all commands. Overall, it’s a very easy read, and quite friendly. However, it could really use a demo guide that walks you through an example of the whole process. The only real problem is that it still refers to the SHADOW account, and not the LPS account the product is distributed in. The default config file contains the same errors.

Test Drive

I built a new database from scratch so I would have a fresh environment. There are two main tricks to getting going. One is getting the IMAGE logging enabled, which isn’t straightforward for most people. The other is making sure you have the right node information in the config file. Here again, a very clear short, canned tutorial would do wonders to speed the process up.

As part of the price of the product, Lund will send a tech out to help you get set up and going. While this is very generous, it isn’t strictly necessary, because the product isn’t that hard. I spent about 20 minutes on the phone with them clearing up my NS connection because I was using the name from HOSTS.NET.SYS and not the name from NMMGR of the remote machine. Once that was set up, away we went.

A nice feature of Shadow is that you don’t have to set up anything on the shadow machine — other than making sure you have the database over there that you want to shadow to. Once the job fires off, just start adding, changing and deleting records from your master database. You can easily query the shadow machine. The tests I ran had the data showing up just as quickly as I could check. I didn’t do any CPU second counts, but it was instant as far as I could verify manually.

I tried a number of things to test the fault tolerance like severing the NS connection, and aborting the shadow job on the master and on the slave. Shadow handled everything correctly. When severing the connection, it would start writing out to tape when required, and transactions would queue up and re-synchronize very easily. I was very happy with how fast and easy it was to use.

Comparisons

The obvious comparison here is to Netbase from Quest. I have years of experience using Netbase, and it is a very complete product. From the perspective of straight IMAGE shadowing, Shadow D/R is much easier than Netbase, and seems to have less process overhead.

From the perspective of a complete solution for every type of file, and porting different processes access around the network to different machines depending on the access type, Netbase is definitely in the lead. I don’t know that a full comparison between the two products is entirely fair. Once you are outside of IMAGE, Shadow can’t do anything, and Netbase still has tons of options left. You pay for these options, of course, but if you need them, it can be worth the extra cost.

Conclusions

It’s nice to have some choices in this arena. Lund has done a nice job with Shadow — it’s fast, it’s easy to set up and maintain, it works, and it’s fairly priced. The overhead on your system is pretty light as well. I am anxious to see support of non-IMAGE files, as I think this would really round the product out. The manual needs to be better in its explanation of setting up the IMAGE log files. This can be confusing for many people, and there’s no step-by-step tutorial.