Click here for Bradmark sponsor message

Terabytes become commonplace in upcoming versions of database

CSY engineers outlined changes to MPE that would allow file sizes of 1 terabyte, while the limit to the files would be 2 gigarecords. Plans include annual performance growth of 20 percent per year. HP’s briefings also included updates on locking improvements, QUERY and DBChange enhancements, third-party index performance boosts and a 3000 data warehousing initiative from CSY.

HP called its project to scale IMAGE the dataset size limitation mitigation. It’s another way to say that IMAGE datasets are going to get a lot bigger. Today’s maximum dataset size is about 40Gb, created by stitching together 4Gb files. HP’s efforts to scale the database and dataset sizes are in response to growth in customer solutions.

IPROF attendees were shown three possible methods to increase IMAGE capacity. First, and the least R&D effort and with the smallest impact on existing applications, was using the sign bit of the record name. This phase of the work would double the maximum size of the datasets to 80Gb.

A second option would be to enlarge the IMAGE block size up to 32K. This would yield a maximum dataset size of 272Gb, and HP said the phase would be a small R&D effort but would require customers to migrate their databases.

Finally, the most expansive phase would be the use of EntryByNumber instead of EntryByName, increasing the maximum size to 10 terabytes (Tb). HP characterized this as a large R&D effort, one that also would require customers to perform a database migration.

HP chose a two-stage solution to expand IMAGE. CSY will first implement using the sign bit of EntryByName first, providing near-term relief for large customers who may be getting close to the current 40Gb limit. Customers in healthcare and direct mail operations are already nearing the current limit. It will be implemented using a new $CONTROL option in DBSCHEMA such as UNSIGNEDBLOCKS and NOUNSIGNEDBLOCKS, with NOUNSIGNEDBLOCKS being the default. Work on this enhancement is in process.

While that work proceeds, HP will implement the EntryByNumber option to allow IMAGE internal limits of individual datasets to get as big as 10Tb. HP plans to do this with another DBSCHEMA option, one that will allow coexistence of both EntryByName and EntryByNumber databases on a single system. HP noted that increasing the BLOCKMAX beyond 5120 bytes might also be considered.

BLOCKMAX expansion could also help HP get ready for two IMAGE enhancements ranked high on the current SIGIMAGE enhancement ballot: Binary Large Object (BLOB) support and increasing the paths per dataset beyond 16.

Locking improvements for IMAGE/SQL will provide for entry-level locks on both Manual MASTERS and DETAILs. TurboIMAGE allows entry-level locks for both Manual MASTERS and DETAILs. Meanwhile, SQL via IMAGE/SQL supports entry-level locks on DETAILs for isolation level RR,RC, but requires a set-level lock on Manual MASTERS. This enhancement will also resolve the “DBG lock area full” problem.

Announced Third Party Indexing (TPI) performance enhancements, available in an upcoming C.07.12 version of TurboIMAGE, will speed up DBFINDs and DBGETs for datasets without TPI. The enhancements will also result in faster serial DBGETs for datasets with TPI and faster chained DBGETs for non-TPI IMAGE DBFINDs on datasets with TPI. Bharati Desai of the CSY database lab said the performance improvements will be accomplished by bypassing TPI procedure calls when they are not needed. Backward compatibility of TurboIMAGE, SUPERDEX, and OMNIDEX will be maintained.

Attendees also noted during the discussion that the dynamic detail dataset expansion (DDX) doesn’t support jumbo datasets in the current version of IMAGE/SQL. HP said it would make DDX work with the new larger file sizes when they are available.

HP did announce that QUERY’s support for the new IMAGE b-trees and third-party indexing features are available now on GR patch QUEKX48A / B for MPE/iX 5.0/5.5. However, the much-requested FIND FIELDA = FIELDB enhancement for QUERY is currently on hold. Additional third-party indexing improvements are under investigation, and HP said that QUERY will support IMAGE Master Dataset Dynamic Expansion (MDX) when MDX comes out later this year.

“SIGIMAGE will be working with HP to refine the requirements and acceptable limitations for the FIND FIELDA = FIELDB enhancement,” said SIGIMAGE chair Ken Sletten, “to see if we can pull it out of hold status in the reasonably near future.”

HP also noted that it own DBChange database utility can now detect and correct the problem that could be caused by the DDX bug (See our story on page one of this issue).

Finally, HP introduced a first look at an all-3000 data warehouse solution, using third-party software bundled with dedicated HP 3000 systems. Jennie Hou of CSY outlined a first draft of a solution at the conference, one that needs no Unix or NT components.

“It’s unique because it is a native HP 3000 solution,” Hou said at the IPROF briefing. “This is very different from what we’ve been doing.”

The solution, which HP said was still in the planning stages, deploys NetBase or SharePlex/iX in its Bridgeware configuration to move data from the production HP 3000 into the data warehouse HP 3000. Bridgeware combines NetBase or SharePlex with Taurus Software’s Warehouse, which filters, transforms and replicates changes that SharePlex detects in 3000 databases. Bridgeware can also ship these changes into an NT or Unix-based SQL database.

On the target data warehouse HP 3000, Omnidex’s search engine will optimize queries using a star schema and multidimensional and aggregation indexes. HP envisions three levels of systems to act as the warehouse, ranging from a 939/020 to a 969/220 to a 979/400 at the top end. Price brackets discussed at the meeting ran from $150,000 for a solution including the 939, $300,000 including the Series 969 and $500,000 with the Series 979.

“Right now we’re still developing all the prototypes for this,” Hou said, “and we’re looking at rolling out the product in this calendar year,” she said.

Larry Boyd, one of the leaders of the New Opportunities Solution Team in CSY, said CSY hasn’t “finished ‘getting all our ducks in a row,’ so there could be changes in what the final product looks like. Our primary goal was to see what the interest in the idea of a 3000 data warehouse was for our existing customers.” Prospects interviewed were interested in building warehouses of 50Gb or more. For more information on the solution’s current state or to specify your needs with CSY, contact Hou at 408.447.5971 or jennie_hou@hp.com.