If De-Frag/X cannot do it, it can’t be done

Swiss Army knife of disk space management gets dozens of new features

Review by John Burke

Lund Performance Solutions’ (LPS) De-Frag/X is nothing less than the Swiss Army knife of disk space management. With at least 24 new commands added to De-Frag/X since I last reviewed the product, it can probably be said that if you do not see what you want, chances are it cannot be done. De-Frag/X gives the HP e3000 system manager the ability to analyze, condense, defragment, balance, clone, erase and fragment disk space, usually online, and surprisingly fast, with minimal impact on the user. Most De-Frag/X functions are tied to MPE/iX’s Transaction Manager (XM), guaranteeing data integrity and protecting against data loss.

Why should you care about a product like this? After all, IO bottlenecks are a thing of the past, right? It is all just a matter of having enough memory, and then any IO issues on MPE/iX disappear? Theoretically, perhaps, but in practice there are still potential IO bottlenecks out there just waiting to rob your system of performance. As MPE systems have grown, the amount of data storage consumed has grown even faster. In fact, the ratio between disk storage used and main memory has increased significantly over the years at many sites. Fragmentation in its many forms is the chief cause of IO performance problems on MPE/iX systems.

There are four types of fragmentation found on HP e3000 MPE/iX systems: file fragmentation, disk fragmentation, system fragmentation and database internal fragmentation. De-Frag/X addresses the first three.

File fragmentation occurs when a file’s set of extents becomes physically discontinuous on disk. This can particularly affect serial IO performance. Disk fragmentation occurs when the free space on a disk drive is spread throughout the disk in many small pieces. This can have performance as well as management implications. System fragmentation views the system as a whole and the interrelationships between files, disks, volume sets and application mix. Under certain circumstances, file fragmentation can improve application performance. Also, balancing the data across multiple spindles can improve system performance. De-Frag/X can address both these issues effectively and efficiently.

Mismanaged disk space robs your system of performance and hurts system availability in a variety of ways:

• Insufficient contiguous disk space on LDEV 1 will prevent OS updates, or, at the very least, make them problematic;

• Severe disk fragmentation wastes disk space and could lead to unnecessary equipment upgrades;

• Severe disk fragmentation will prevent fork() from working in some situations, causing aborts;

• Severe file fragmentation will negatively impact serial IO;

• Unbalanced volume sets will negatively impact all IO;

• Unbalanced files will negatively impact random IO (e.g. IMAGE datasets); and,

• Wasted disk space beyond file EOFs could lead to unnecessary equipment upgrades.

The CONTIGVOL command of VOLUTIL, introduced with MPE/iX 5.0 provides limited mitigation for the first problem. However, there does not exist any tool in MPE/iX that takes on the other disk space management issues. De-Frag/X addresses all these issues and more.

Features

Key basic features include:

• Report the current level of of disk fragmentation, or file fragmentation for any fileset

• Condense or create contiguous space any drives

• Defragment drives or whole volume sets

• Defragment (or combine) fragmented files to improve serial IO

• Balance the distribution of files on the multiple physical drives of a volume set

• Balance the placement of file extents across multiple physical drives to improve the IO performance for files accessed predominantly in a random access pattern

• Trim unused, wasted space from the end of files

• Performs most functions on-line while the system is in use without significantly degrading performance

Additional features/commands include:

• ALTFILE: Can be used to inspect or change certain file characteristics such as file code.

• CLONEDISK: Allows you to perform a bit-by-bit copy of a failing disk drive to another disk drive of the same or greater size. It can be used to take action against the loss of valuable data that may be located on a failing drive.

• ERASEDISK: Scratches a specified LDEV. It can be used to write zeros over the entire disk drive, effectively erasing all user data.

• FINDBIGGEST: Finds the largest free space on the specified disk or all disks.

• FSCHECK: Displays a short description of errors reported by fscheck.mpexl.telesup.

• ISSYS: Checks the specified filename and reports if it is a “system file”, De-Frag/X will refrain from moving certain “system files” at all and will refrain from moving certain other files off LDEV 1.

• SCOREDISK: Computes a fragmentation score for a disk.

• WHOUSE: Reports disk usage grouped by account, or top-level HFS directory, for the top 50 disk users.

Installation

Installation is a snap. Simply restore files with the CREATE option and execute an install script to complete the installation. Many system managers like to install third party software on user volume sets, keeping only HP software on the system volume set. LPS makes it very easy to install De-Frag/X onto a user volume set.

Documentation

The User’s Guide for De-Frag/X, at a mere 32 pages, is a bit skimpy for my taste. However, the User’s Guide does contain a good basic tutorial on fragmentation and, combined with the copious online help covering De-Frag/X commands, is adequate for most users. For those interested in delving into fragmentation in a big way, there is De-Frag/X author Stan Sieler’s white paper “Analysis and Correction of Fragmentation” (www.allegro.com/papers/fragmentation.html).

Let’s take it out for a spin

LPS strongly recommends you run FSCHECK first and fix any problems discovered (with Response Center consultation) before running De-Frag/X. In fact, to “encourage” this behavior, De-Frag/X programmatically restricts data intrusive functions if FSCHECK was not run first. This can, of course, be overridden. [Note: FSCHECK is described in “MPE/iX System Utilities Reference Manual, 32650-90081.] And, of course, if you are planning any data-intrusive tasks, create a full backup before starting.

Figure 1 is just a list of commands available in De-Frag/X. Note that the data intrusive commands require SM capability. Figure 2 shows first the results of a “MAP ALL” command on a small test system. The second part of Figure 2 analyzes the target system looking for any files greater than 30 megabytes that could benefit from trimming unused, wasted space from the end of the file.

Figure 3 is a snapshot of a disk condense operation in progress. It closely resembles the display from a PC disk defragmentation tool. The time it takes to do a CONDENSE depends, for example, upon the amount of fragmentation, the speed of the drive and IO channel and whether it is contending for resources with other products. One drive was only slightly fragmented and it took less than a minute to run a CONDENSE. Another drive was much more fragmented and took slightly over 10 minutes to run a CONDENSE. The more frequently you do a CONDENSE, the less time it will take. Note that De-Frag/X can be run from a batch job so you can schedule resource-intensive tasks to run regularly during off-hours. A good choice might be right after a full backup.

Conclusion

If you want to take control of your disk space instead of having it control you, then check out De-Frag/X from Lund Performance Solutions. Mismanaged disk space can be robbing you of performance and could force you to add disk storage prematurely. A well-tuned storage farm will pay out in performance dividends.

John Burke is editor of the NewsWire’s net.digest and Hidden Value columns and has managed HP 3000 systems for more than 20 years.