Analysis by John Burke
Put a fork() in it, it's done --
well, maybe.
It appeared at first to be a problem with that poor maligned product Netware/iX: "Has anyone seen the following error message from Netware/iX? What do I do?"
NETWARE Spawn NCPENGINE: fork ->NCPENGIN -- failed!!This gave Jeff Kell (HP3000-L list master) the opportunity to again
discuss one of his pet
peeves: fork() and MPE/iX.
errno=11 NETWARE Spawn NCPENGINE: fork ->NCPENGIN -- failed!!
Resource busy, try again
"I just love plugging away at this error that still is not fixed. Check the volumes in your mpexl_system_volume_set for the largest contiguous area that is available to transient space. Repeat for all volumes in the set.In an example of what makes HP3000-L so unique and useful, Goetz Neumann, of the HP Worldwide Technology Expert Center MPE/iX, replied:"If the minimum of these values is below ~1024 that's your problem. Defragment the volume set if you have one of the third-party utilities to do this. Use VOLUTIL's CONTIGVOL if you don't."
Why is this important? Many POSIX compliant programs, especially server programs, use fork(). In particular, the Web server programs use fork(). If any member of mpexl_system_volume_set is severely fragmented or very low on free space, you will have a problem.
"For MPE/iX 5.0, there is the General Release patch MPEHXT8A (Ed. note: now superseded by MPEJXA7A). For MPE/iX 5.5 a Beta Test patch MPEHXT8B now exists, which will be incorporated in the next major release.Hooray. But then Goetz adds this from the patch text: "This patch returns to the older algorithm.""Excerpt from the patch text: ...fork() failed to create a child process when disk space is fragmented. Currently, when fork() is called to create a child process, the data from the parent's HEAP object needs to be copied over to the child's HEAP object. Due to the generally large amount of data from the HEAP that needs to be copied, the conventional copy mechanism proved to be too slow and insufficient. Therefore, a VSM routine, copy_non_virgin_pages_of_object, was design solely for this purpose. In order to get maximum performance, this VSM routine has a lot of restrictions and uses a lot of assumptions. One of the restrictions is that the extents of the parent HEAP object have to be exactly 'cloned' for the child's HEAP.
"Now, if the disk space is low or fragmented, the system will have a hard time finding a large enough chunk of disk space to allocate to the child's extents if the parent's extents happen to be created in a larger size. In this case, fork() will fail with 'out of system resources, try again'."
Which prompted both Stan Sieler and Jeff Kell to ask: "This 'patch' doesn't back out the other fork() enhancements, does it?" No word yet from HP, although Stan suggested a means to test a patched system. If we get an answer, we'll note it in a future issue of the NewsWire.
What does CONTIGVOL do? Really?
In the section about "fork()", the CONTIGVOL command of VOLUTIL is mentioned. As it happens, the question of what exactly CONTIGVOL does and how it compares to commercial defragmentation tools came up in another thread. There still appears to be some confusion over the capabilities of this command that became part of the VOLUTIL tool with MPE/.iX 5.0.
CONTIGVOL was added specifically to aid the OS software update process. At a certain stage you need to have n sectors of contiguous free space on LDEV 1 so that a work file can be created with just one extent. Prior to CONTIGVOL, HP did not provide any means to manage free space. Some people actually had to reload to create contiguous free space on LDEV 1 before they could update the OS.
VOLUTIL's CONTIGVOL command will attempt to consolidate free space into larger pieces. It does not do any file extent consolidation. If you need more than this, there are several excellent third-party tools that provide true defragmentation.
OK, so POSIX is important --
Now how do I learn about it, and get it to work?
It is evident from the various POSIX threads on HP3000-L that many of us, including yours truly, are just now starting to venture into POSIX land. With web servers, Perl, gcc, Samba and the like showing up on the HP3000, we now have a reason to leave the familiar territory of the MPE name space for the relative uncertainty of the HFS.
Several people, most notably Jeff Kell, have been sharing their experiences with POSIX. They also shared information sources for POSIX and MPE/iX.
For the Hierarchical File System (HFS): "New Features of MPE/iX: Using the Hierarchical File System", HP part # 32650-9035 (also on LaserROM)
The computer based training (cbt) course that comes with MPE/iX 5.0 (xeq posixcbt.lsn.sys).
For the POSIX shell: "MPE/iX Shell and Utilities User's Guide", HP part # 36431-90002 (unfortunately still not on LaserROM)
And, don't forget the UDCs in HPPXUDC.PUB.SYS (distributed with MPE/iX 5.0).
Once you read up a little on the shell, you may want to try it out. If you are on MPE/iX 5.5 and are trying to access the shell from a plain vanilla user, you are out of luck. It seems that the file permissions, a concept you will have to become intimately familiar with if you want to work with POSIX on MPE/iX, were screwed up on the MPE/iX 5.5 release.
The following, executed from within the shell as manager.sys or some other
user with SM
capabilit, should at least allow a user with default capabilities to enter
the shell without
error:
cd /usr/libHowever, you will discover that, for example, man pages (the shell's
version of online
help) are still unavailable. What follows was posted by Lars Appel of HP
Germany. Appel
is responsible for the easiest to set up version of the freeware web server
and is currently
working on porting Samba to MPE/iX. He noted that the procedure is from the
SR for the
bad permissions.
chmod -R o+r terminfo
chmod o+r /etc/profile
chmod o+r /etc/motd
1. Restore hfsfiles.hp36431.support from the FOS tape. (This file will be used just to get a list of the files which need to be modified).
2. :tobyte.hpbin.sys "hfsfiles.hp36431.support /tmp/tarfile"
3.
:SH.HPBIN.SYS -LNOTE: The quote that precedes tar and follows 57- in the above command is the "backward" quote usually found as the top-left key on a keyboard and those surrounding the ^d are normal single quotes.shell/iX> for file in `tar tvf /tmp/tarfile 2>/dev/null |
> grep -v '^d' | cut -c 57- `
> do
> chmod 444 $file
> done
shell/iX> exit
4. :purge /tmp/tarfile
:purge hfsfiles.hp36431.support
I tried this on my system (fingers trembling as I attempted to copy everything exactly) and it seemed to work. And did not screw up anything else. By the way, if you understand everything in step 3 without referring to a pile of manuals, then you qualify as a POSIX shell expert. At least in my book.
STORE/iX or TurboSTORE/iX II?
Or, an unexpected present
The following exchange of posts got me thinking, dangerous as that may be:
"We tried to RESTORE a partial database when the following error appeared:Then:
File NOT STORED: FILE IS PART OF AN IMAGE DATABASE AND ROOT IS NOT SPECIFIED. A quick lookup in HELP and we 'forgot' to put the PARTIALDB option on the end of our RESTORE command line. Seems the new version of RESTORE will, by default, restore only full databases. Where in the world did that one come from?"
"On 5.5, you have FOS STORE/RESTORE, TurboSTORE and TurboSTORE 7x24. Check the product number in the heading. I know all the TurboSTORE stuff is new code. TS is B5151AA and TS7x24 is B5152AA. PARTIALDB was added in the Turbo varieties, but I didn't think the FOS one was affected (but you could have just proven that to be false). Note that it won't STORE one either unless it can find and access all the files (unless you specify PARTDB). And this includes any attached DBE-files and TPI files."And finally:
"Did I hear correctly that HP ships a demo copy of TurboSTORE with 5.5 whether you want it or not for the first 30 days or so after you install? Maybe this would explain what you are seeing."
When I wrote a series of articles on MPE/iX 5.5, I touched briefly on the enhancements to STORE and TurboSTORE. I also noted my experience with the output from STORE (undocumented in the MPE/iX 5.5 Communicator): all database STOREd are specifically listed at the end of the SHOW listing along with the number of files making up the database.
It was my understanding, per the MPE/iX 5.5 Communicator, that PARTIALDB, FULLDB, STOREDIRECTORY, NOSTOREDIRECTORY and STATISTICS were new features of the FOS STORE. While I will not discuss this here, you should look into the documentation on STOREDIRECTORY. It could prove to be a useful feature for some. Also, HP did ship a trial copy of TurboSTORE II with MPE/iX 5.0 documented in the MPE/iX 5.0 Communicator, it was my understanding that this was not being done with MPE/iX 5.5.
While there appears to be some ambiguity in the MPE/iX 5.5 Communicator, MPE/iX 5.5 HELP clearly states that PARTIALDB, STOREDIRECTORY, NOSTOREDIRECTORY and STATISTICS are part of the FOS STORE/RESTORE. Hmmm. HELP says nothing about FULLDB. So, I slap a tape into the old trusty DDS drive and try
:STORE mydbrootfile;*tape;FULLDB;SHOW;STATISTICSPretty interesting stuff. So, HELP lied. FULLDB does exist as an option for STORE. And FOS STORE has all these neat new options.
>>TURBO-STORE/RESTORE VERSION C.55.07 B5152AA <<STORE PCCTS;*TAPE;SHOW;FULLDB;STATISTICS
TUE, JAN 21, 1997, 9:34 AM
(file listing snipped)
BACKUP STATISTICSFILE DATA MISC DATA TOTAL ON REEL MEDIA 1 | 460288 | 5184 | 465472--------- +-------------+-------------+------------- BACKUP TOTAL THROUGHPUT 460288 5184 465472
MEDIA START TIME MEDIA END TIME THROUGHPUT (K/S) RETRIES 9:34 AM 9:35 AM 75 K/S 0
TOTAL ELAPSED TIME: 0 hours, 0 minutes, 6 seconds DATABASE INFORMATION: TURBOIMAGE DATABASE: PCCTS.PUB.CCC FILES STORED :4 FILES STORED :4 TOTAL MEDIA WRITTEN :1
Wait a minute. Take a look at the STORE banner in my test above: "TURBO STORE/RESTORE VERSION C.55.07 B5152AA". This is the full TurboSTORE/iX 7x24 True-Online Backup product. But we do not own any version of TurboSTORE/iX.
Did HP ship a trial version with MPE/iX 5.5 and just did not document it? It does not say anything about being a trial version. I know the trial version that shipped with MPE/iX 5.0 specifically stated it was a trial version in the header. And, besides, it has been over 90 days since I installed MPE/iX 5.5. I checked.
So, what have I demonstrated here? Precious little for sure. HELP fails to document the FULLDB option of STORE/RESTORE. We still do not know for sure if PARTIALDB, FULLDB, STOREDIRECTORY, NOSTOREDIRECTORY and STATISTICS are part of the FOS STORE/RESTORE. And, if you are on MPE/iX 5.5, you should check and see what version of STORE/RESTORE you have. You too may have gotten an unintentional Christmas present from HP.