NewsWire TestDrive: Netbase/Shadowing

Click here for RAC Software sponsor message

Netbase/Shadowing:
Who knows? The Shadow

Ensuring the data is always available
is what makes Netbase/Shareplex shine

Review by John Burke

Managing IS, and in particular planning for Disaster Recovery, is much more complicated today than in the days of batch processing, when all you needed were your backup tapes and a similarly configured machine somewhere. Today, with online, distributed, 7x24x365 access increasingly the norm, many organizations are creating their own disaster recovery sites, and even integrating them into normal processing for peak load balancing, batch reporting and to ensure true 7x24x365 access. This is where Netbase/Shadowing from Quest Software comes into play.

Quest Software is certainly best known for its data replication product, Netbase/Shadowing (also marketed and supported by Hewlett-Packard as Shareplex/iX). And Netbase/Shadowing is what this Test Drive will be all about. However, the Netbase family also includes these modular products: Network File Access (NFA), Netbase Client middleware, Netbase Spooling (NBSpool) spooler/printer management, performance statistics, and remote program access via AutoRPM. (Please see “Other Netbase Modules” below.)

This TestDrive is different than most in that we not only actually use the product, but have used it for quite some time – and it is critical to our company’s operation.

What and Why

Netbase/Shadowing is all about data availability. Making data available 24 hours a day, seven days a week, 365 days a year. Whether the pressure on availability comes from the need to create a backup without disrupting production or whether a catastrophic failure has occurred on a production machine or anything in between, Netbase/Shadowing ensures the availability of your critical data.

Netbase/Shadowing creates and maintains – in real time – exact copies of MPE files and databases on one or more secondary “Shadow” machines (see Figure 1 ). Typically, the Shadow machine is located at some distance from the Master machine. In our case, it is only about ten miles, but I know of situations where the Master and Shadow machines are separated by thousands of miles. All updates to files or databases set up for shadowing are performed on the Master system and then transported and posted to the Shadow systems. The Netbase/Shadowing system guarantees they are exact duplicates. If anything goes wrong, the Shadow copy is marked as out of sync.

With Netbase/Shadowing, you can have complete data and environment replication. What this means is that in a disaster, failover to the Shadow machine can be accomplished in minutes, not hours or even days. In the event that the Master machine becomes unavailable for any reason, and assuming appropriate network configuration, users can simply reconnect to the secondary machine and resume normal processing. The way we accomplish this for PC users is to make sure they have a second icon on their desktops which points to the Shadow machine. For terminal users, since we use OpenView DTC Manager, we have all our DTCs configured for multiple host access; it is just a matter of doing a CTRL-K and connecting to a different host. Non-disaster failovers are also possible in order to do planned or unplanned maintenance on the Master machine.

Also with Netbase/Shadowing, since the Shadow machine has exact copies of data, backups can be done on the Shadow machine, freeing the Master machine for true 7x24x365 access. While the backup is in progress, transactions are still received and queued on the Shadow system to maintain protection against failure of the Master system. Thus, even if the Master system were to fail, the Shadow system, after posting the queued transactions, will contain an exact copy of the data as it existed when the Master system failed. While online backup products exist that, in theory, are non-disruptive of processing, they can still use significant resources. Moving backup to the Shadow machine virtually eliminates any impact on production.

Finally, since the shadowing process itself generally uses only a small percentage of the Shadow system’s resources, you can off-load certain processing to the Shadow machine. For example, batch reporting is a good candidate. Also, using the Shadow machine for non-mission-critical EIS systems is a common scenario.

Additional features of Netbase/Shadowing include: selective shadowing, bi directional shadowing, multicast shadowing, consolidated shadowing, and user-exit shadowing.

Supported file types include MPE flat files, message files, KSAM (both CM and NM) files, TurboIMAGE databases, IMAGE/SQL databases and, optionally, Allbase/SQL databases. Note that Netbase/Shadowing does NOT currently support shadowing of the HFS/Posix file space. [Note: Netbase/Shadowing does support the shadowing of IMAGE databases with Jumbo datasets even though they exist in the Posix filespace. This is because shadowing is accomplished through the IMAGE intrinsics, thus hiding the underlying Posix structure.]

Configuration Requirements

Netbase/Shadowing requires, at a minimum, two HP 3000s networked via TCP/IP. One is designated the Master system, while others are the secondary, or Shadow, systems. The primary, or Master, copies of the files and databases reside on the Master system while the Shadow systems hold the duplicate copies of the files and databases. In our particular case, both the Master and Shadow machines are multiprocessor 9x9s located about 10 miles apart. They are on different segments of our corporate LAN, connected by a T1 line. Netbase/Shadowing shares this circuit with all the other traffic (e-mail, VT, file and printer sharing) generated at the Shadow site.

How it works

The Netbase Shell (see Figure 2 ) intercepts all opens of databases and files and checks the Netbase directory to see if the file should be shadowed. When an update on the Master system that is flagged for shadowing is completed successfully, it is written to the NBM.IPC message file.

NBEXPORT (a process under the Master system’s Netbase Job) reads from the NBM.IPC file and transports the transactions to the Shadow system. If NBEXPORT cannot communicate with the Shadow system, the transactions are written to Export Queue files. (When NBEXPORT can again communicate with the Shadow system, the transactions in the Export Queue files will be sent to the Shadow system, and then the transactions from the NBM file will be sent.)

NBIMPORT (a process under the Shadow system’s Netbase Job) takes transactions sent from the Master system and writes them to Import Queue files.

NBPOST (also a process under the Shadow system’s Netbase Job) reads the transactions from the Import Queue files and applies the transactions to the shadowed copy of the file/database. If NBPOST encounters a non-fatal open error when trying to open the Shadow copy, it will write the transactions for that file into a Post Queue file. (These transactions can be applied later once the open error has been resolved.)

A lot is happening here; however, the impact on the Master system is barely recordable. Under normal production circumstances, we see Netbase/Shadowing using only 1-2 percent of resources on the Master machine.

Installation

Installation is an easy four-step process on each system in the Netbase/Shadowing network that takes as little as 30 minutes to complete. The same is true for updates to an existing Netbase/Shadowing network. However, since Netbase modules are installed into the system libraries SL.PUB.SYS, RTMSL.PUB.SYS and XL.PUB.SYS, each system must be restarted for the install/update to take effect. This also means that if you are applying a PowerPatch or IMAGE patch to a system in your Netbase/Shadowing network you will have two extra reboots, the first to remove the Netbase modules before applying the patches and the second to re-install the Netbase modules after applying the patches.

The actual configuration of the shadowing environment takes some planning though it is easy to do using the NBDIR utility program. Entries are made on both the Master and Shadow systems. I would recommend that most new users contract for initial installation support from Quest. Not because it is hard to configure, but to ensure that it is done right and that all contingencies are covered.

Documentation, Training and Support

An excellent 350-plus page manual that was published in conjunction with release 9.7 in 1997 provides the Netbase reference documentation. It covers everything from configuration issues to trouble shooting. You’ll want to pay particular attention to the section on Disaster Recovery Procedures (Chapter 12).

It is probably possible to learn everything you need to know to set up a Netbase/Shadowing environment from the documentation; however, I strongly recommend the class conducted by Quest Software. Netbase/Shadowing is not difficult to use, but considering the importance of the job it is doing, it is just good business to learn as much as possible.

Quest provides general telephone technical support during normal business hours and emergency support after normal hours. The technical support personnel I’ve talked with are well versed in normal operational issues. I’ve had occasion to need technical support several times on the weekend and the response has been excellent.

Other Netbase Modules

Netbase NFA lets you easily spread applications across multiple machines. It gives you direct access to files on remote machines without the need for remote sessions. Netbase Client provides access to HP 3000 MPE files and TurboIMAGE databases from Unix, DOS, Windows 3.x, Windows 95 and Windows NT. Netbase Spooling automates the management of spool files, including the organization, presentation, archiving and transferring among machines. Netbase Statistics records and reports network file access, process overhead, and response times. It can uncover faulty locking strategies and application inefficiencies. Netbase AutoRPM gives you direct access to any application on any machine in the network.

Conclusion

Shadow Lite, which includes Shadowing and NBCopy, costs from $6,000 to $50,000 depending upon CPU tier. The Shadow Bundle, which includes Shadowing, AutoRPM, Netbase/Spooling, Vista and NBCopy, costs from $8,000 to $59,000. Finally, the Full Bundle, which includes Shadowing, NFA, Statistics, AutoRPM, Netbase/Spooling, Vista and NBCopy, is priced from $14,000 to $100,000, again depending upon CPU tier.

Netbase/Shadowing is neither for the faint of heart, nor those with a skinny purse; but you have to ask yourself how important data availability is to your operation. Along with death and taxes, you can add unplanned downtime to the list of certainties in life. Netbase/Shadowing provides a very robust HP 3000 solution for any organization that cannot tolerate downtime or lost data.

Netbase/Shadowing

Quest Software
610 Newport Center Dr.
Suite 1400
Newport Beach, CA 92660-6565

Tel: 714.720.1434
Fax: 714.720.0426
E-mail: info@quests.com
WWW: www.quests.com

Netbase/Shadowing provides an HP 3000-based software solution to the problems of continuous data availability and Disaster Recovery by “shadowing” or replicating all changes in data in real time to a secondary HP 3000.

Netbase/Shadowing software is priced from $6,000 to $100,000 depending upon options and CPU tier.



Click here for RAC Software sponsor message
	Netbase/Shadowing: Who knows? The Shadow
		Ensuring the data is always available is what makes Netbase/Shareplex shine

		Review by John Burke Managing IS, and in particular planning for Disaster Recovery, is much more complicated today than in the days of batch processing, when all you needed were your backup tapes and a similarly configured machine somewhere. Today, with online, distributed, 7x24x365 access increasingly the norm, many organizations are creating their own disaster recovery sites, and even integrating them into normal processing for peak load balancing, batch reporting and to ensure true 7x24x365 access. This is where Netbase/Shadowing from Quest Software comes into play. Quest Software is certainly best known for its data replication product, Netbase/Shadowing (also marketed and supported by Hewlett-Packard as Shareplex/iX). And Netbase/Shadowing is what this Test Drive will be all about. However, the Netbase family also includes these modular products: Network File Access (NFA), Netbase Client middleware, Netbase Spooling (NBSpool) spooler/printer management, performance statistics, and remote program access via AutoRPM. (Please see “Other Netbase Modules” below.) This TestDrive is different than most in that we not only actually use the product, but have used it for quite some time – and it is critical to our company’s operation. What and Why Netbase/Shadowing is all about data availability. Making data available 24 hours a day, seven days a week, 365 days a year. Whether the pressure on availability comes from the need to create a backup without disrupting production or whether a catastrophic failure has occurred on a production machine or anything in between, Netbase/Shadowing ensures the availability of your critical data. Netbase/Shadowing creates and maintains – in real time – exact copies of MPE files and databases on one or more secondary “Shadow” machines (see Figure 1 ). Typically, the Shadow machine is located at some distance from the Master machine. In our case, it is only about ten miles, but I know of situations where the Master and Shadow machines are separated by thousands of miles. All updates to files or databases set up for shadowing are performed on the Master system and then transported and posted to the Shadow systems. The Netbase/Shadowing system guarantees they are exact duplicates. If anything goes wrong, the Shadow copy is marked as out of sync. With Netbase/Shadowing, you can have complete data and environment replication. What this means is that in a disaster, failover to the Shadow machine can be accomplished in minutes, not hours or even days. In the event that the Master machine becomes unavailable for any reason, and assuming appropriate network configuration, users can simply reconnect to the secondary machine and resume normal processing. The way we accomplish this for PC users is to make sure they have a second icon on their desktops which points to the Shadow machine. For terminal users, since we use OpenView DTC Manager, we have all our DTCs configured for multiple host access; it is just a matter of doing a CTRL-K and connecting to a different host. Non-disaster failovers are also possible in order to do planned or unplanned maintenance on the Master machine. Also with Netbase/Shadowing, since the Shadow machine has exact copies of data, backups can be done on the Shadow machine, freeing the Master machine for true 7x24x365 access. While the backup is in progress, transactions are still received and queued on the Shadow system to maintain protection against failure of the Master system. Thus, even if the Master system were to fail, the Shadow system, after posting the queued transactions, will contain an exact copy of the data as it existed when the Master system failed. While online backup products exist that, in theory, are non-disruptive of processing, they can still use significant resources. Moving backup to the Shadow machine virtually eliminates any impact on production. Finally, since the shadowing process itself generally uses only a small percentage of the Shadow system’s resources, you can off-load certain processing to the Shadow machine. For example, batch reporting is a good candidate. Also, using the Shadow machine for non-mission-critical EIS systems is a common scenario. Additional features of Netbase/Shadowing include: selective shadowing, bi directional shadowing, multicast shadowing, consolidated shadowing, and user-exit shadowing. Supported file types include MPE flat files, message files, KSAM (both CM and NM) files, TurboIMAGE databases, IMAGE/SQL databases and, optionally, Allbase/SQL databases. Note that Netbase/Shadowing does NOT currently support shadowing of the HFS/Posix file space. [Note: Netbase/Shadowing does support the shadowing of IMAGE databases with Jumbo datasets even though they exist in the Posix filespace. This is because shadowing is accomplished through the IMAGE intrinsics, thus hiding the underlying Posix structure.] Configuration Requirements Netbase/Shadowing requires, at a minimum, two HP 3000s networked via TCP/IP. One is designated the Master system, while others are the secondary, or Shadow, systems. The primary, or Master, copies of the files and databases reside on the Master system while the Shadow systems hold the duplicate copies of the files and databases. In our particular case, both the Master and Shadow machines are multiprocessor 9x9s located about 10 miles apart. They are on different segments of our corporate LAN, connected by a T1 line. Netbase/Shadowing shares this circuit with all the other traffic (e-mail, VT, file and printer sharing) generated at the Shadow site. How it works The Netbase Shell (see Figure 2 ) intercepts all opens of databases and files and checks the Netbase directory to see if the file should be shadowed. When an update on the Master system that is flagged for shadowing is completed successfully, it is written to the NBM.IPC message file. NBEXPORT (a process under the Master system’s Netbase Job) reads from the NBM.IPC file and transports the transactions to the Shadow system. If NBEXPORT cannot communicate with the Shadow system, the transactions are written to Export Queue files. (When NBEXPORT can again communicate with the Shadow system, the transactions in the Export Queue files will be sent to the Shadow system, and then the transactions from the NBM file will be sent.) NBIMPORT (a process under the Shadow system’s Netbase Job) takes transactions sent from the Master system and writes them to Import Queue files. NBPOST (also a process under the Shadow system’s Netbase Job) reads the transactions from the Import Queue files and applies the transactions to the shadowed copy of the file/database. If NBPOST encounters a non-fatal open error when trying to open the Shadow copy, it will write the transactions for that file into a Post Queue file. (These transactions can be applied later once the open error has been resolved.) A lot is happening here; however, the impact on the Master system is barely recordable. Under normal production circumstances, we see Netbase/Shadowing using only 1-2 percent of resources on the Master machine. Installation Installation is an easy four-step process on each system in the Netbase/Shadowing network that takes as little as 30 minutes to complete. The same is true for updates to an existing Netbase/Shadowing network. However, since Netbase modules are installed into the system libraries SL.PUB.SYS, RTMSL.PUB.SYS and XL.PUB.SYS, each system must be restarted for the install/update to take effect. This also means that if you are applying a PowerPatch or IMAGE patch to a system in your Netbase/Shadowing network you will have two extra reboots, the first to remove the Netbase modules before applying the patches and the second to re-install the Netbase modules after applying the patches. The actual configuration of the shadowing environment takes some planning though it is easy to do using the NBDIR utility program. Entries are made on both the Master and Shadow systems. I would recommend that most new users contract for initial installation support from Quest. Not because it is hard to configure, but to ensure that it is done right and that all contingencies are covered. Documentation, Training and Support An excellent 350-plus page manual that was published in conjunction with release 9.7 in 1997 provides the Netbase reference documentation. It covers everything from configuration issues to trouble shooting. You’ll want to pay particular attention to the section on Disaster Recovery Procedures (Chapter 12). It is probably possible to learn everything you need to know to set up a Netbase/Shadowing environment from the documentation; however, I strongly recommend the class conducted by Quest Software. Netbase/Shadowing is not difficult to use, but considering the importance of the job it is doing, it is just good business to learn as much as possible. Quest provides general telephone technical support during normal business hours and emergency support after normal hours. The technical support personnel I’ve talked with are well versed in normal operational issues. I’ve had occasion to need technical support several times on the weekend and the response has been excellent. Other Netbase Modules Netbase NFA lets you easily spread applications across multiple machines. It gives you direct access to files on remote machines without the need for remote sessions. Netbase Client provides access to HP 3000 MPE files and TurboIMAGE databases from Unix, DOS, Windows 3.x, Windows 95 and Windows NT. Netbase Spooling automates the management of spool files, including the organization, presentation, archiving and transferring among machines. Netbase Statistics records and reports network file access, process overhead, and response times. It can uncover faulty locking strategies and application inefficiencies. Netbase AutoRPM gives you direct access to any application on any machine in the network. Conclusion Shadow Lite, which includes Shadowing and NBCopy, costs from $6,000 to $50,000 depending upon CPU tier. The Shadow Bundle, which includes Shadowing, AutoRPM, Netbase/Spooling, Vista and NBCopy, costs from $8,000 to $59,000. Finally, the Full Bundle, which includes Shadowing, NFA, Statistics, AutoRPM, Netbase/Spooling, Vista and NBCopy, is priced from $14,000 to $100,000, again depending upon CPU tier. Netbase/Shadowing is neither for the faint of heart, nor those with a skinny purse; but you have to ask yourself how important data availability is to your operation. Along with death and taxes, you can add unplanned downtime to the list of certainties in life. Netbase/Shadowing provides a very robust HP 3000 solution for any organization that cannot tolerate downtime or lost data.
Netbase/Shadowing Quest Software 610 Newport Center Dr. Suite 1400 Newport Beach, CA 92660-6565 Tel: 714.720.1434 Fax: 714.720.0426 E-mail: info@quests.com WWW: www.quests.com Netbase/Shadowing provides an HP 3000-based software solution to the problems of continuous data availability and Disaster Recovery by “shadowing” or replicating all changes in data in real time to a secondary HP 3000. Netbase/Shadowing software is priced from $6,000 to $100,000 depending upon options and CPU tier.