NewsWire Test Drive: The Express Train to Data Mining and Extraction

Click here for TSG sponsor message

DataExpress:
The Express Train
to Data Mining and Extraction

Review by John Burke

First of two parts

DataExpress empowers even the casual, unsophisticated user with a tool to rapidly select and extract data from HP 3000 data structures, manipulate, calculate, summarize and reformat the data and then download it to a PC or Mac. You can also report the selected data out to a file or printer. For the power user, the possibilities of DataExpress are almost limitless.

With the product’s optional ODBCLink interface – a Level 2-compliant implementation of Microsoft’s Open Database Connectivity (ODBC) protocol – you can use Windows-based applications and tools to access TurboIMAGE (natively, without relying on any Allbase/SQL overhead), IMAGE/SQL, Oracle, and Allbase/SQL databases on the HP 3000. It offers all this in a client-server environment secured and controlled at the server level by DataExpress. [As part of an agreement with HP, a stripped down version of ODBCLink called ODBCLink/SE, without the control/security features and without the ability to access TurboIMAGE directly, is distributed with MPE/iX as of MPE/iX 5.5 Express 3.]

In use on over 1,000 systems, DataExpress has been on the market since the mid 1980s. It was initially developed by the same company that produced the Rapid products (acquired by HP), the best known of which is Transact. M.B. Foster Associates acquired DataExpress in 1989 and continues to develop (witness ODBCLink) and market it as an end-user data access tool. In Part I of this Test Drive, I am going to look at the core DataExpress product. In Part II, I will concentrate on the ODBCLink option and how it differs from ODBCLink/SE, the middleware now included with Express 4 MPE/iX 5.5.

How it works,
what it will do

While originating on the Classic MPE/V systems, DataExpress was completely re engineered to take advantage of MPE/iX and PA-RISC features to achieve maximum performance. The main module of DataExpress, and the only module most users will directly interact with, is called DataExpressCenter (DXC). Figure 1 shows the main menu. Within DataExpressCenter, the procedure is the basic unit of work. You create procedures, catalog and maintain procedures, and run procedures.

A procedure can access all major types of MPE files: TurboIMAGE and IMAGE/SQL, KSAM, MPE sequential files, MPE Self-Describing (SD) files, and DataExpress subfiles (more on these later), singly or in any combination. In the case of multiple sets and multiple files, the order in which sets are retrieved can be specified along with the fields used to link sets.

DataExpress can create output in a variety of formats (see Figure 2) that interface directly with almost any PC application or HP 3000 application. Note that in particular it uses PRN (comma delimited, quoted string), SD (various self-describing file types, SYLK (for MS Excel), ASCII, KSAM and SD KSAM (this last is very useful when interfacing with HP 3000 applications.

DataExpress can also produce standard, formatted report listings. Its strength is not, however, as a report writer, though some customers use it as their only end-user report writing tool. DataExpress’s strength is in interfacing existing application data on the HP 3000 with other applications, usually on the PC. Classically, this has been through the extraction/download approach, and DataExpress does this at least as well and probably better than the other products on the market. When you employ the included ODBCLink module, the data is directly available to the PC application, but unlike standard ODBC, both the view of the data and the security of the data are controlled by the DataExpress server process.

The TestDrive

In the case of formats destined for the PC, users can choose to let DataExpress automatically download the output file to the PC. To test the general ease of use I ran DXC, picked an existing database and created a procedure to extract a subset of information from a dataset (about 1,000 records and five fields from a dataset of 30,000 records with over 20 fields per record). I then had DXC download it to my PC, where I loaded it into Excel. I was able to do all of this in a matter of minutes without once cracking open the manual. With DataExpress, your users can become immediately productive.

Procedures are created using an easy-to-navigate character-mode menu interface (see the figures for examples). Before you wrinkle up your nose in disdain, this DXC interface is fast, efficient, robust and very user-friendly. The system remembers the last procedure worked on and makes extensive use of both function keys (so if you must use your mouse, you can usually click on a function key to navigate) and shorthand keystrokes.

For example, you can usually back up a prompt by either entering ”/“ or hitting the function key labeled “Escape.” DataExpress has three types of help available at any prompt, two of which are context sensitive: “?” yields a one- or two-sentence description; “??” yields a more extensive screen of help; and, “??” followed by F4 puts you into an indexed help subsystem that lets you get help on any prompt. Options are also available for the user to customize menu appearance.

A procedure can use a single TurboIMAGE dataset or MPE file, or it can access complex linked structures as though they were one single file. The linkage between datasets/files can be fully defined in the procedure, or it can be pre-arranged in a DataView (see DataCatalog below). Your procedures can be stored for re-use in a catalog file. Utilities are provided for maintaining catalog files and copying procedures from one catalog to another, and procedures can be given a title and a password for security.

When creating a procedure, basic TurboIMAGE item definitions can be obtained directly from the root file (Figures 3 and 4) or, like KSAM and MPE file definitions, may be entered dynamically as part of the procedure. Additionally, DataExpress comes with the DXFDGEN utility that can be used to define fields in TurboIMAGE databases, KSAM files and flat files creating a DataExpress data dictionary for use by any procedure. Definitions can include, for example, edit masks for decimal and date formats. Optional interfaces also exist to Powerhouse Dictionary and HP Dictionary.

Computed fields, both numeric and string, can be defined and even used as sort keys within procedures. If data is already sorted, a pre-sort flag can be set to suppress sorting, while still processing summaries at level breaks. Selection values and output formats can be hard coded into a procedure or prompted for at run time. Selection criteria can be the common comparison to a range of values or even the comparison of two fields.

DataExpress has excellent date handling capabilities, and supports all popular storage formats, including POWERHOUSE dates and MM3000 dates, and can reformat dates to fit almost any requirement. Figure 5 shows example date output where the internal format was YYYYMMDD. These features mean that DataExpress can play an important role in Year 2000 projects.

DataExpress supports named and unnamed temporary files for multipass or drill down processing. The DataExpress subfile is a self-describing (SD) file with user labels describing fields and their attributes such as decimal precision edit mask and menu name. An extraction “file” can be re-used, eliminating the need to redo the data collection phase of the reporting process. For example, you can preview the data in report list form (scrolling forward or backward through the list) before creating the extract file. It’s especially useful because you can do all this without having to re-run the potentially lengthy data extraction.

Options

Optional DataExpress modules include:

• An ODBCLink for client-server access to data structures on the HP 3000 (which I’ll discuss next month in Part II)

• DataCatalog and ManagerPlus for customizing the user environment and control of system resources

• ReportDesigner and GraphicDesigner for enhanced output

• POWERHOUSE and HP Dictionary interfaces

• Omnidex, Allbase/SQL and Oracle interfaces

• PrintLocal for managing and retrieving print spoolfiles

• French and Spanish language message catalogs

DataCatalog and ManagerPlus deserve special consideration. DataCatalog is a “must” for any serious user of DataExpress. It is a fast, powerful and easy-to-use way to create table views of complex data structures with pre-defined naming conventions (item, prompt, header), links between sets/files, edit masks, etc. Dataviews shield the end-user from the underlying messy detail of complex links, obscure name references and unknown data formats.

You can further refine a dataview with secured fields, restricting access to data based upon the pre-defined value of a specific field. For example, you could create different “logical” views of sales data based upon the value of sales territory.

ManagerPlus enables the DataExpress System Administrator to optimize the use of available resources and further secure the DataExpress environment. For example, some users could be designated runtime only.

Limitations

DataExpress does not currently recognize Superdex keys, though this is planned for a future release (note that the ODBCLink option does support Superdex indexing now). Also not available yet, but scheduled for a future release, is support for Image B trees.

Installation and documentation

Installation follows the standard “restore a job, which when streamed sets up the accounting structure and finishes the restore” procedure. The whole process takes maybe ten minutes. A UDC file contains all the shorthand commands necessary to begin immediately using DataExpress.

M.B. Foster Associates takes justified pride in its documentation and detailed online help. The manual is a somewhat intimidating, hernia-inducing monster in a 2-inch binder (the Administrator’s Guide adds another inch). Not to worry. The online help is so good and complete and the user interface so intuitive that most users will need the manual only as an occasional reference for doing trickier stuff. Aiding the new user is a series of online tutorials created using a feature available to all DataExpress customers: keystrokes can be saved to a file, edited – adding comments – and used as input to DataExpress, creating customized tutorials.

Conclusion

DataExpress is an easy to learn and easy to use tool that gives even the casual, unsophisticated user powerful, sophisticated data access manipulation and extraction capabilities. The DataExpress train will deliver whatever you need.

DataExpress

M.B. Foster Associates Ltd.
82 Main Street South
P.O. Box 580
Chesterville, ON Canada
K0C 1H0

Tel: 613.448.2333
800.ANSWERS (267.9377)
Fax: 613.448.2588
E-mail: sales@mbfoster.com

An easy-to-learn, easy-to-use tool that provides even the casual, unsophisticated user with powerful, sophisticated data access, manipulation and extraction capabilities. The optional ODBCLink interface directly links the world of PC applications with data on the HP 3000.

DataExpress runs on all supported versions of MPE/iX. Pricing is by module across seven CPU classes. Prices range from $2,000 (US) for the DataExpress Core module on a Class 1 platform to $32,000 (US) for a typical installation on a Class 7 platform. A trial version is available upon request.




Click here for TSG sponsor message
	DataExpress: The Express Train to Data Mining and Extraction

		Review by John Burke First of two parts DataExpress empowers even the casual, unsophisticated user with a tool to rapidly select and extract data from HP 3000 data structures, manipulate, calculate, summarize and reformat the data and then download it to a PC or Mac. You can also report the selected data out to a file or printer. For the power user, the possibilities of DataExpress are almost limitless. With the product’s optional ODBCLink interface – a Level 2-compliant implementation of Microsoft’s Open Database Connectivity (ODBC) protocol – you can use Windows-based applications and tools to access TurboIMAGE (natively, without relying on any Allbase/SQL overhead), IMAGE/SQL, Oracle, and Allbase/SQL databases on the HP 3000. It offers all this in a client-server environment secured and controlled at the server level by DataExpress. [As part of an agreement with HP, a stripped down version of ODBCLink called ODBCLink/SE, without the control/security features and without the ability to access TurboIMAGE directly, is distributed with MPE/iX as of MPE/iX 5.5 Express 3.] In use on over 1,000 systems, DataExpress has been on the market since the mid 1980s. It was initially developed by the same company that produced the Rapid products (acquired by HP), the best known of which is Transact. M.B. Foster Associates acquired DataExpress in 1989 and continues to develop (witness ODBCLink) and market it as an end-user data access tool. In Part I of this Test Drive, I am going to look at the core DataExpress product. In Part II, I will concentrate on the ODBCLink option and how it differs from ODBCLink/SE, the middleware now included with Express 4 MPE/iX 5.5. How it works, what it will do While originating on the Classic MPE/V systems, DataExpress was completely re engineered to take advantage of MPE/iX and PA-RISC features to achieve maximum performance. The main module of DataExpress, and the only module most users will directly interact with, is called DataExpressCenter (DXC). Figure 1 shows the main menu. Within DataExpressCenter, the procedure is the basic unit of work. You create procedures, catalog and maintain procedures, and run procedures. A procedure can access all major types of MPE files: TurboIMAGE and IMAGE/SQL, KSAM, MPE sequential files, MPE Self-Describing (SD) files, and DataExpress subfiles (more on these later), singly or in any combination. In the case of multiple sets and multiple files, the order in which sets are retrieved can be specified along with the fields used to link sets. DataExpress can create output in a variety of formats (see Figure 2) that interface directly with almost any PC application or HP 3000 application. Note that in particular it uses PRN (comma delimited, quoted string), SD (various self-describing file types, SYLK (for MS Excel), ASCII, KSAM and SD KSAM (this last is very useful when interfacing with HP 3000 applications. DataExpress can also produce standard, formatted report listings. Its strength is not, however, as a report writer, though some customers use it as their only end-user report writing tool. DataExpress’s strength is in interfacing existing application data on the HP 3000 with other applications, usually on the PC. Classically, this has been through the extraction/download approach, and DataExpress does this at least as well and probably better than the other products on the market. When you employ the included ODBCLink module, the data is directly available to the PC application, but unlike standard ODBC, both the view of the data and the security of the data are controlled by the DataExpress server process. The TestDrive In the case of formats destined for the PC, users can choose to let DataExpress automatically download the output file to the PC. To test the general ease of use I ran DXC, picked an existing database and created a procedure to extract a subset of information from a dataset (about 1,000 records and five fields from a dataset of 30,000 records with over 20 fields per record). I then had DXC download it to my PC, where I loaded it into Excel. I was able to do all of this in a matter of minutes without once cracking open the manual. With DataExpress, your users can become immediately productive. Procedures are created using an easy-to-navigate character-mode menu interface (see the figures for examples). Before you wrinkle up your nose in disdain, this DXC interface is fast, efficient, robust and very user-friendly. The system remembers the last procedure worked on and makes extensive use of both function keys (so if you must use your mouse, you can usually click on a function key to navigate) and shorthand keystrokes. For example, you can usually back up a prompt by either entering ”/“ or hitting the function key labeled “Escape.” DataExpress has three types of help available at any prompt, two of which are context sensitive: “?” yields a one- or two-sentence description; “??” yields a more extensive screen of help; and, “??” followed by F4 puts you into an indexed help subsystem that lets you get help on any prompt. Options are also available for the user to customize menu appearance. A procedure can use a single TurboIMAGE dataset or MPE file, or it can access complex linked structures as though they were one single file. The linkage between datasets/files can be fully defined in the procedure, or it can be pre-arranged in a DataView (see DataCatalog below). Your procedures can be stored for re-use in a catalog file. Utilities are provided for maintaining catalog files and copying procedures from one catalog to another, and procedures can be given a title and a password for security. When creating a procedure, basic TurboIMAGE item definitions can be obtained directly from the root file (Figures 3 and 4) or, like KSAM and MPE file definitions, may be entered dynamically as part of the procedure. Additionally, DataExpress comes with the DXFDGEN utility that can be used to define fields in TurboIMAGE databases, KSAM files and flat files creating a DataExpress data dictionary for use by any procedure. Definitions can include, for example, edit masks for decimal and date formats. Optional interfaces also exist to Powerhouse Dictionary and HP Dictionary. Computed fields, both numeric and string, can be defined and even used as sort keys within procedures. If data is already sorted, a pre-sort flag can be set to suppress sorting, while still processing summaries at level breaks. Selection values and output formats can be hard coded into a procedure or prompted for at run time. Selection criteria can be the common comparison to a range of values or even the comparison of two fields. DataExpress has excellent date handling capabilities, and supports all popular storage formats, including POWERHOUSE dates and MM3000 dates, and can reformat dates to fit almost any requirement. Figure 5 shows example date output where the internal format was YYYYMMDD. These features mean that DataExpress can play an important role in Year 2000 projects. DataExpress supports named and unnamed temporary files for multipass or drill down processing. The DataExpress subfile is a self-describing (SD) file with user labels describing fields and their attributes such as decimal precision edit mask and menu name. An extraction “file” can be re-used, eliminating the need to redo the data collection phase of the reporting process. For example, you can preview the data in report list form (scrolling forward or backward through the list) before creating the extract file. It’s especially useful because you can do all this without having to re-run the potentially lengthy data extraction. Options Optional DataExpress modules include: • An ODBCLink for client-server access to data structures on the HP 3000 (which I’ll discuss next month in Part II) • DataCatalog and ManagerPlus for customizing the user environment and control of system resources • ReportDesigner and GraphicDesigner for enhanced output • POWERHOUSE and HP Dictionary interfaces • Omnidex, Allbase/SQL and Oracle interfaces • PrintLocal for managing and retrieving print spoolfiles • French and Spanish language message catalogs DataCatalog and ManagerPlus deserve special consideration. DataCatalog is a “must” for any serious user of DataExpress. It is a fast, powerful and easy-to-use way to create table views of complex data structures with pre-defined naming conventions (item, prompt, header), links between sets/files, edit masks, etc. Dataviews shield the end-user from the underlying messy detail of complex links, obscure name references and unknown data formats. You can further refine a dataview with secured fields, restricting access to data based upon the pre-defined value of a specific field. For example, you could create different “logical” views of sales data based upon the value of sales territory. ManagerPlus enables the DataExpress System Administrator to optimize the use of available resources and further secure the DataExpress environment. For example, some users could be designated runtime only. Limitations DataExpress does not currently recognize Superdex keys, though this is planned for a future release (note that the ODBCLink option does support Superdex indexing now). Also not available yet, but scheduled for a future release, is support for Image B trees. Installation and documentation Installation follows the standard “restore a job, which when streamed sets up the accounting structure and finishes the restore” procedure. The whole process takes maybe ten minutes. A UDC file contains all the shorthand commands necessary to begin immediately using DataExpress. M.B. Foster Associates takes justified pride in its documentation and detailed online help. The manual is a somewhat intimidating, hernia-inducing monster in a 2-inch binder (the Administrator’s Guide adds another inch). Not to worry. The online help is so good and complete and the user interface so intuitive that most users will need the manual only as an occasional reference for doing trickier stuff. Aiding the new user is a series of online tutorials created using a feature available to all DataExpress customers: keystrokes can be saved to a file, edited – adding comments – and used as input to DataExpress, creating customized tutorials. Conclusion DataExpress is an easy to learn and easy to use tool that gives even the casual, unsophisticated user powerful, sophisticated data access manipulation and extraction capabilities. The DataExpress train will deliver whatever you need.
DataExpress M.B. Foster Associates Ltd. 82 Main Street South P.O. Box 580 Chesterville, ON Canada K0C 1H0 Tel: 613.448.2333 800.ANSWERS (267.9377) Fax: 613.448.2588 E-mail: sales@mbfoster.com An easy-to-learn, easy-to-use tool that provides even the casual, unsophisticated user with powerful, sophisticated data access, manipulation and extraction capabilities. The optional ODBCLink interface directly links the world of PC applications with data on the HP 3000. DataExpress runs on all supported versions of MPE/iX. Pricing is by module across seven CPU classes. Prices range from $2,000 (US) for the DataExpress Core module on a Class 1 platform to $32,000 (US) for a typical installation on a Class 7 platform. A trial version is available upon request.