Cooking with Python on MPE

March 2002

Cooking with Python on MPE

Whip up tasty dishes with ample applications using this scripting language

By Curtis Larsen

Something I’m sure I share with other HP 3000 folks is a love for technology that solves problems in some straightforward yet elegant manner. After all, this is why we all continue to use our HP 3000 systems (and probably why we chose them in the first place).

Especially exciting, though, are the new technologies that further extend our systems and solve even more problems. I can remember eagerly reading each new “Communicator” for the enhancements list, and my personal favorites – upgrades to the Command Interpreter. I’d be excited to try out each new function and new “FINFO” parm. (When the JOBINFO function hit the streets I was practically singing.)

As much fun as I have with the CI though, I recognize that every programming language has its limitations. So HP 3000 programmers have used various different languages available such as COBOL, Business Basic, or VESoft scripting – or a combination of them — to solve larger tasks. Each language brings its own flavor and abilities to the programming table, and when POSIX scripting was added to the ingredients, a wonderful curry resulted. Like a great curry, you can still see, smell and taste each individual ingredient, but they all contribute to a sum greater than the parts.

Some of the new flavors the POSIX world gives us are the Bourne, BASH, C, and Korn, shells, as well as public-domain C, FORTRAN, Assembly, and Basic compilers. It also gives us the newer interpreted scripting languages such as Perl, Java, and Python.

Now chances are good you’ve heard about Perl and Java, but how much do you know about Python? Ahhh… let’s start opening some of those spice jars, shall we?

Python is a fast, extensible, object-oriented scripting language created by Guido Van Rossum and named after the ’70’s British comedy troupe and television show “Monty Python’s Flying Circus”. (It’s too good to make up folks. If that doesn’t begin to spell “curry” to you, I don’t know what does.)

Python and Perl are very similar in capabilities —what can be done in one, can also be done in the other – it’s just that their approaches to those capabilities differ. Perl leans towards terse syntax, and many small “add-in” modules, while Python (if not verbose) leans more towards self-describing syntax and fewer, larger, “add-in” modules. Both can be written in Ye Olde Top-Downe style, or by using objects and methods in the newer Object-Oriented style of programming — pick the sauce that suits you. Both languages have been ported to almost every OS imaginable (and a few that weren’t). Python also uses OOP syntax similar to Java with it’s “Object<dot>Method”. This similarity to Java comes in very handy, but more on that later.

One quirk that Python people have to get used to though is the indenting. The beginning of a text line is significant in Python, and indicates dependence. Now, now – I heard that agonized cry, like you had just eaten too much pepper. Trust me on this one – like a pepper you may find it hard to swallow at first (especially in one large bite), but if you come to it slowly, you’ll learn to appreciate the subtleties and grow just as addicted to it. And let’s be reasonable – if you can start a significant line of COBOL at a certain column for twenty-odd years, you can handle a little reasonable indenting.

Rather than explain Python’s syntax in detail though, I’ll point you to the Python web site where you can really chew on it. The main plate is at www.python.org, and many other links can be found from there for general consumption.

Ok, so let’s say you buy into it for now – you’re at the MPE table checking out the various Python dishes for appearance and aroma before biting into one. A wise course of action, but while some of the offerings can be a bit exotic, I’ve found nothing unattractive. You can find an incredible list of Python applications at “The Vaults of Parnassus” (www.vex.net/parnassus) — here are a few that might be of interest to you in combination with an HP 3000:

• “ZOPE”: An extremely powerful Web-application creation utility — and then some!

• “Gadfly”: A powerful Python-based SQL engine – see its Web page for details.

• “Peercat”: Web-based aggregate data collector. (Usually news items, but…)

• “Scanerrlog”: A script to meaningfully parse Apache’s “error_log” file to HTML, etc.

• “txt2html.py”: Converts various ASCII formats to HTML.

• “inpim.py”: Calendar/To-Do app w/cross-system sync. Budgeting subsystem, too.

• “PyBook”: Searches, downloads, and displays Project Gutenberg books.

• “Roundup”: Issue tracking system with web,CLI, and e-mail interfaces.

• “RoutePlanner”: A highway trip planner.

• “Pyrite”: Allows Palm Pilot synchronization.

• “PyPsion”: Similar application for Psion PDAs.

• “web2ldap.py”: Full-featured web-based LDAPv2+ client.

• “flad”: Create, read, and write INI-like config files.

• “ODBC Socket Server”: Access Windows ODBC sources via XML TCP/IP interface.

• “Python-DSV”: Parses CSV, TSV, etc. files, guesses string encoding, etc.

And more CGI scripts than you can shake a kebob stick at.

This list just scratches the surface, since there are far more HP-useful Python applications than can be shown here, and due to its ease of use and power, more people are sitting down to the “Python picnic” each day. There are many Python applications that could have a strong impact on your regular MPE recipes, and more are continually arriving. Look over the list above – ever want to directly use your HP data with a Palm Pilot? Send and receive data from an ODBC source directly from a batch job? Map the shortest routes to clients? Check on the weather before shipping a product there? Many more possibilities exist!

Python includes very good support for XML (including XML-RPC and SOAP), LDAP, and offers an interesting native object persistence (or “object pickling”) to a file, so that you can save an object and its properties, freezing it for cooking another day.

Although the “original” Python was written in C, a newer version of Python named “Jython” has been written using Java. In addition to running native Python scripts, this version also permits use of the native Java objects and features from within Python. Looking (and reading) very similar to Java, you can actually code something very quickly using Python for later export to Java, etc., treating Python as a sort of RAD language. So whether you think of Python as a “C food” entree, or as a side dish to chew on with your morning “cup of Java,” you’ll find it both digestible and flavorful. You can use a little or a lot, spread it thinly or plaster it on thickly – whatever suits your taste and mood.

I’ve only used the C-based Python on my HP system, but you could give Jython a whirl. The C version is a bit dated but runs quickly, while the Java version is very current, but starts a tad slower. It just depends on what you want to use Python for, and whether or not you’ll need some of the newer functions and modules.

Probably the thing that really excites me personally about Python is its “dictionaries,” or what Perl calls “hashes.” Folks who use these types of variables almost always get hooked on them. For example, you can create a dictionary named “D”, and store a value in it using a string key named “info” which might look like this: D[“info”] = “555-1212”. Now whenever you say ‘print D[“info”]’ you’ll get “555-1212”. It’s like using an array, but without the limitations of an array — you don’t need to preset a dictionary, worry about internal structure, use only numbers, or know how many “elements” it will have. You can even dynamically store a dictionary within another dictionary (and so on) to get some incredible depths of sophistication. Suddenly a “simple scripting language” allows fast use of tables and three-dimensional databases. (And it pulls up those values just as quickly!)

For an HP 3000-specific example of using Python dictionaries, I combined a VESoft Security/3000 report with a quick Python hack – er – script to show me the last date any terminal in a given date range was used, then print those results sorted by terminal. Sure, I could have coded it all some other way, but using the report as input to a filtering script was simplest for me, and gave me the results I wanted very quickly. Figure 1 below shows the script (be gentle).

Figure 1

import fileinput, string # These modules provide some # nice extra methods for file # and string handling, etc. x = " " # Inits: Not needed, but. . . hist = {} totals = {} for line in fileinput.input(): # Loop thru each report file r = string.strip(line) # line (given as the parm) if r[0:5] == "Logon": # Detail record? Process it. rdate = "0" + string.strip(r[07:14]) # Load our work variables rdate = rdate[-7:] rdev = int(string.strip(r[23:27])) rlogon = string.strip(r[27:62]) rname = string.strip(r[63:94]) hist[rdev] = (rdate, rlogon, rname) # Load the ‘history’ dictionary # using the LDEV # as the key if rdate <> x: # Print each new date as we go x = rdate print rdate # Initialize the ‘totals’ dictionary logon count at first use if not totals.has_key(rdev): totals[rdev] = 0 totals[rdev] = totals[rdev] + 1 # Add one to the logon count keylist = hist.keys() # ‘keylist’ contains seen LDEVs keylist.sort() # BAM! Now it’s a sorted list for ldev in keylist: # for each LDEV we saw. . . rdate, rlogon, rname = hist[ldev] # load some work vars, and: print "%04u %5u %7s %s" % (ldev, totals[ldev], rdate, rlogon)

This script assumes that a chronologically-ordered list of LDEV logons will show the most recent logon for every terminal (with any luck, hey?), and records every logon for each LDEV. Each successive assignment overwrites the previous value used by that LDEV key, so the last logon for each LDEV key is the last value kept. Additionally, the script counts each time that LDEV key was seen, so that you have an idea as to how often the terminal is used. This little script could easily be adapted to record other information, like the first logon, or whatever else you might need.

I hope this helps whet your appetite for the Python language, and that you will give it a try – again, there’s far more to the language than can be described here. Many thanks to Joseph Koshy at HP Bangalore for bringing Python to MPE/iX, and to the HP3000-L list for assistance. Like to try making some dishes of your own? You can download the “official” Python/iX v1.5.1 from Jazz at jazz.external.hp.com/src/pythonix .. You can also see what Koshy is doing in his spare time to port Python v2.x by visiting the Python/iX web site on Sourceforge: pythonix.sourceforge.net. For a copy of Jython, head on over to www.jython.org and download a copy.

Go ahead — join the Python/iX groundswell now, and get your piece of the Python.

Curtis Larsen has been working with HP 3000s for 11 years, and believes that, given enough time, any application can be written using the CI. He currently works for Covance Laboratories, in Madison, Wis.