RSX utility for reading VMS backup tapes

   Back in the day, as the sun was setting on the RSX world, VAX/VMS grew strong in the land. As many people and sites migrated from PDP-11s to VAXen, there was a steady demand for a VMS utility that could read BRU format tapes (BRU is the RSX Backup and Restore Utility, used to backup up files and disks). Several pretty good solutions were coded up by the user community, and they all worked pretty well.

  Back in September, I had occasion to restore some old VMS files from VMS Backup TK50 and TK70 cassettes.  As it happens, at the moment, I don't have any VAXen that have TK tape drives installed. I do, however, have a couple of RSX systems that have TK drives - so I needed the opposite of the assorted VMS BRUREAD utilities. I did a little looking around, and couldn't find any RSX tools that could read a VMS backup tape.   I guess that doesn't really come as a surprise - the IT world almost always operates by Italian racing rules (what's behind you is not important).

  So, I needed a utility to restore files from a VMS Backup save set to an RSX system. I did a little looking around, and didn't find anything that would do the job. There was a Linux utility that can read VMS Backup tapes (vmsbackup), but it couldn't really deal with getting the file attributes right on the RSX system. I could have modified it to do the job, with a lot of work removing the Linuxisms and inserting RSXisms, but, I'd rather write my own solution in Macro-11 on a clean sheet of paper than spend a lot of time modifying a C program. There was a DECUS RT11 based project to read VMS Backup tapes on RT11, but, it didn't actually ever get completed - and RT11 is a whole lot different from RSX, so it wouldn't have been that helpful. 

  There's not a whole lot of info available about the format and data structures of a VMS Backup save set. I probably have the sources on microfiche - but, I haven't had access to a microfiche reader since I retired from MegaBig Engineering Corp. Most of the information I needed was gleaned by viewing the output of the still undocumented VMS BACKUP/LIST/ANALYZE command on some example save sets on a VMS system. I found some good info about the breakdown and offsets of the data in a Backup set on a VMS Freeware collection (https://www.digiater.nl/openvms/freeware/v30/logging-virtdisk/), as part of an interesting project by Glenn Everhart that provided automatic logging of writes to a device. More info was found in the file system MACRO definitions on both RSX and VMS systems.

  In a nutshell, Backup save sets consist of big blocks, each of which consists of a block header and typically contains one or more  "records". The possible type of records are

Null - null records are used at the end of a block to fill it to length

Summary - Each backup save set volume has a summary record, containing info about the block, including the command that created the save set.

Volume - contains summary information for each volume in the save set.

File - each file saved in  the backup set has a file record, which contains info about the file, including its name and attributes.

VBN - VBN records contain the virtual blocks of a file. These records come after a file record.

Physical - physical record - prolly from backup/physical - I didn't research this, since I am not interested in BACKUP/PHYSICAL save sets. You occasionally find one in the middle of a save set - not sure what that's all about.

LBN - not sure - probably used with physical backups - didn't see any in my research

FID - These appear to contain a list of all the file IDs of the files that are included in an image backup. 

  So, to make a long story short, in a nutshell, to summarize, prècis style, as bullet points, generally speaking, to synopsize, from 10,000 feet, glossing over a heckuva lot of details,  I whomped up a program that reads tape blocks via QIO, finds records in the blocks, and acts on each record. For each File record, I open a file of the indicated name and of the specified size and extent, with the indicated attributes. For each following VBN record after that, I write it sequentially to the file. When I get a new File record, I close the existing file and start a new one. Until we're done. Sounds pretty straightforward, nicht wahr? Yeah, it took a couple months to get it working just so. All of the IO was done via QIOs, including ACP QIOs, to avoid the space that including FCS would have cost us. Address space was an issue (as Cutler used to say, "Size matters") since Backup save set often used very large tape blocks. 

 This code doesn't change mode or muck about with the disk or OS internals using non-supported methods, but, like all my work, it's use at your own risk....

  So, here's the code

BTR.MAC

To build...

mac btr=btr

tkb btr/id=btr

  It has to be a /ID task, because I and D space is required to allow sufficient address space for the large tape input buffer. Sorry, RSX11M and IAS users (they don't have I&D space available).

  To use...

Mount the tape as foreign

>mou mu0: /for

then...

>run btr

  You have to "run btr" instead of installing it and doing BTR commands, since BTR doesn't have GCML and CSI command support - they were omitted to conserve address space.

  It will prompt as BTR>. Enter the name of the tape drive. If that's all you enter, you get a listing of the backup saveset on the tape.

>BTR>mu0:

[GLEASON.ACCOUNT]SETACCOUNT.MAR;31

[GLEASON.ACCOUNTING]Z.MAR;2

[GLEASON.ACP]ATTMOD.MAR;3

[GLEASON.ACP]CHECKUIC.MAR;1

[GLEASON.ACP]CHGCRE.MAR;3

[GLEASON.ACP]CHGDATE.MAR;5

[GLEASON.ACP]DATE.MAR;38

[GLEASON.ACP]GETFILNAM.MAR;1

 And like that.

  If you want to restore all the files, specify an output directory

>BTR>mu0:/out=[somedir]

  If you want to restore or list just some files, use the FIL switch, with wildcards

>BTR>mu0:/fil=*att*.mar  (to list selected files)

>BTR>mu0:/out=[somedir]/fil=*uic*.mar   (to restore selected files)

  If you want to see some of the inner workings while listing or restoring, you can add the /DBG switch, with one of the following values

a.dbg  =   1                    ;print a whole bunch of debug info

a.wld  =   2                    ;print wild card processing

a.iosb =   4                    ;print tape dsw and iosb

a.labl =  10                    ;print label related stuff

a.file =   20                    ;print file activities

a.summ =  40                ;print save set and volume summary info

a.rec  =    100                ;print record processing

a.blk  =    200                 ;print block header info

These are octal values. You can get more than one at a time by adding them together - ie, wild card processing and record processing  together, use 102. Note - some dbg options have bugs in them - /dbg:1 can fail in mid tape, for instance, and /dbg:200 occasionally fails as well. They worked well enough to debug the thing, so fixing them isn't a high priority.

  Since VMS supports 39.39 format filenames (39 chars for filename, 39 chars for extensions) and also has hierarchical directories, a lot of filename wrangling has to get done when restoring files. Filenames are truncated down to the 9.3 format used by RSX. No attempt was made to write them to more than one directory on restore - the output directory you specify is it. Also, VMS supports $,_ and - in filenames. RSX...doesn't. These characters are replaced with X, Y and Z. BTR will happily restore the .DIR files in the save set...but RSX will not use them as directories - they'll just be one more file in the output directory. All this means that you'll have to pay attention to what files you're getting, and what directories they were originally in.

   BTR also restores the .DIR and .SYS files from the backup, so it would be a pretty bad idea to specify [0,0] as the directory to restore to....

 Since a lot of VMS savesets spanned more than one tape, I have included primitive multi-tape support in BTR. When you get to the end of a tape of a multivolume save set, BTR will prompt for the next tape. Hang the next tape and press return to proceed. There's no checking to make sure that you hung the right tape, so...be careful...I also didn't bother adding any support for multi-savesets on a single tape. I mean, who does that? I never did , back when I managed VMS systems.

If you spot any problems using BTR - let me know...I'm working on Version 2, which will have a larger tape buffer, so I'll be fixing any bugs I hear about.

RD32 - Nothing Lasts Forever

 Lately, I've been using an old Micro PDP-11/73 for some development work. It's compact, fits under the desk, and is plenty fast enough for the project. But, it only had a single RD32 disk drive in it, which left me a little cramped for disk space. There's not enough room to install DECnet, for example, so that means I have to sneakernet things on and off of it using RX50s.

  So, I figured I'd add another RD32, which would double the disk space available. I installed a 6 button control panel (not strictly required, but it makes adding a second drive to a BA23 system easier). Time to install the disk... I spent quite a little bit of time on this - the cables I had aren't keyed, so I had to span the vector space of possible orientations looking for how to plug them in. No soap - the thing still wouldn't come on line. I tried some diags, but, since I didn't know how these particular diags acted when everything was working, I couldn't really conclude anything from their results. 

  It finally started making some odd noises, so I tried another disk, which worked fine. I pulled the one that wouldn't work apart, to see if I could tell what the problem was. Here's a picture of what I found....

  It's a picture of the disk arm for the top platter, with a broken wire trailing off from where the read/write head used to be attached. So, no surprise it wasn't working.

  Here's a picture of me reflected in the top platter - nothing to do with the problem, it's just how I do a selfie...

TU58 emulator for RSX11

  So every once in a while I try to retrieve some space here in the War Room (my computer room - we call it that because, during a spirited discussion about how messy it was, I jokingly told my wife "we can't fight in here - this is the War Room") by assembling some of the parts into systems. I have an old BA11-VA  QBUS box I got in trade a long time ago.I figured it could consume a few QBUS cards and maybe even do something useful (well, OK, maybe not useful, but at least amusing). 

I had a look at the specs for it. The power supply isn't all that powerful in that box, so the choice of cards is a little limited. It could just barely power an 11/23, an MXV11A and an RQDX3 in there. And that would have been a relatively useful system. - I could control a floppy and an old RD disk with it, and the MXV would provide memory and two serial ports.

  But, I didn't have a PDP11-23 (I got plenty of 11/23+s, but no plain 11/23s). An 11/23+, MXV11A and  RQDX3  together  draw too much power for that plan. And, I was in no mood to make the cables required to use the RQDX3 without the breakout board it gets used with in BA23 system boxes. And ST506 hard drives are loud and getting rarer every day. So scratch that idea - even though it would have allowed me to use RSX on it. 

  I decided to use an 11/2 processor (KD11-HA), and an MXV11A. The MXV11A has memory and boot roms on it, and it has two serial ports. One gets used for the console. The other one can be used to connect to a TU58 drive unit, which provides (slow and annoying) mass storage to the system. This won't run RSX (well, it would run unmapped RSX, but that's not my idea of a good time), but it could run RT11 and stand alone programs.

  A few articles back, I joked about how richly hated  TK50 drives were. Another DEC tape device that was not well liked was DECtape II - the  TU58. TU58s were little tape cartridge drives, that were used like a disk. Like their previous generation's namesake, DECtape, they were block addressable like disks and you could rewrite a block in the middle of the tape. TU58s connect via RS-232, and use good ole async serial ASCII to communicate with the system. TU58s were tiny (only held 512 blocks, 256K) and slow - very slow. They usually connected at 9600 baud, and seek times were...long.

  Since they were  controlled like disks, that meant that they were managed by a disk ACP on RSX . Disk ACPs are single threaded for all the disks that they control. If you mounted a TU58 using the same ACP as, say, the system disk, you would stall all disk activity on the system disk  whenever the TU58 needed a few minutes to seek to the end of the tape looking for a block...this is bound to make the phones ring...

  TU58s showed up as load media on assorted DEC systems. They were why VAX-11/730s and HSC50s booted slowly. A TU58 was used as part of the boot process on my old 11/725, and it took quite awhile. I managed to cut the amount of time booting required in half  by careful arrangement of the needed files on the tape so that rewinds were avoided.

  Real TU58 drives, however, are scarce these days, and almost all of the rubber capstans in them have deteriorated into gooey messes. Not a problem, though - lots of folks have written TU58 emulators, that connect to the serial ports and serve blocks back and forth just like a real TU58 unit does. Here's a couple of  good projects that illustrate use of them

http://www.pdp-11.nl/ba11-boxes/ba11va/ba11va-info.html 

http://www.diane-neisius.de/pdp11/index_E.html

   On the implementation side, Will Kranz has done a lot of work on TU58 emulation in general - lots of good info on his page at

http://www.willsworks.net/pdp11/tu58-emu

especially about the new version of the RSP protocol, MSRP.

  Don North has also done a lot of work in the area, both on emulators and on tools to manipulate TU58 images. See some of his work at

https://github.com/AK6DN

  So, there's lot's of TU58 emulators available. But, (as far as I could discover), they all run on Windows, Linux, and a few run on stand alone microcontroller gizmos. I'm sure I've mentioned it before, but let me repeat again here, that I hate Windows and Unix/Linux with the incandescent, coruscating, ravening white hot soul searing  intensity of a billion exploding supernova antimatter hypergalaxies. Now that I think about it, I don't like Alphas, Itaniums or VMS-X86-64, either, but all that's another story... anyway, I will be thrice damned before I use Windows or Linux to support a DEC hardware project. I decided to write my own emulator, using my favorite platform - RSX11M+.  I'm a lot more likely to have RSX running here when I need to emulate a TU58 than I am Windows or Linux...

  To develop and test this program, I used a Micro-11/73 I already had. By luck, this old system came with a DLV-11 already installed - this was fortunate, since that is what RSX uses to communicate with TU58s. I connected a cable from it to my HP 4951C logic analyser, and from it, back to a DHV11 TT port back on the same 11/73 system. This way, RSX would be looking to see a TU58 on the DLV, and  my emulator program would strive to look like one on the DHV port. 

  The HP 4951C is a piece of gear I recently got on Ebay for $100, and it's invaluable for this sort of thing. It lets you see what's really on the wire, instead of having to add debug print statements and guessing which side wrote what, when. It cost $3500 back when it was new - but, there's not much demand for them anymore, since ASCII communications on RS232 style connections are pretty rare these days. If you have to do any work like this, I highly recommend acquiring one. It's from the golden age of HP test equipment.

  So, basically, a program to emulate a TU58 boils down to doing a bunch of reads and writes.  Writes are easy - Just assemble a protocol message and send it out in one piece, via a QIOW.  Reads are another story. You can't really predict what or how much you're gonna get ahead of time, so you can't just do a read and wait for some number of characters.  The easiest way to code something like this would be to do one character reads and react to them as they come in, as the protocol requires. I didn't even try that strategy - doing that many QIOWs at 9600 baud or higher would be a surefire recipe for failure.

  Years and years back, Jamie Hanrahan (RIP) posted somewhere a description of his strategy for dealing with IO for a protocol on a serial line. I spent a couple of days looking for it -  couldn't find it. It may have been part of the comments for the DECUS UUCP project he worked on - I'm still looking for it. Anyway, as well as I can remember, it did a one character, non-blocking QIO read. When that read completed, it activated an AST that determined how many characters are in the typeahead buffer, then did a QIOW that actually read them. Those two steps repeat (getting the count, and reading that many characters) until the typeahead count returns 0). When the typeahead buffer is empty, the one character QIO is reissued, thus arming it for the next burst of incoming data. I inserted the incoming characters into a circular queue, and the code running at non-AST level de-queues them out one at a time as needed.

  The above strategy drastically reduces the number of QIOs vs the single character method - in this program, it does around 3 or 4 QIOs/data packet, vs. the 134 a full data packet would require, reading a character at a time. And, it doesn't get behind and choke itself to death, which is the whole point of doing it this way.

  Anyway, getting packets and then acting on them was middlin' straightforward. There were a couple of stumbles along the way, caused by the fact that every piece of DEC gear that accesses TU58s has its own set of quirks.

  For instance, RSP protocol has a boot protocol defined - but nothing I tested with used that - they all just did a standard read to block 0 when they wanted to boot. There is an enhanced protocol called MRSP that some TU58s support, with additional handshaking. TU58 hosts are allowed to query the TU58 if they support this protocol. XXDP and RT11 never queried about it. RSX queried about it - every other command  packet. It seems like DDDRV (the RSX TU58 driver) doesn't record the fact that we replied "none for me" to the MSRP query. Also,RSX occasionally "stutters" - emits two bytes of a packet, then starts over and sends the whole packet. RT11 likes to occasionally write FF into the unit and switch fields of a command packet for no reason.

  Anyway, it got done. I've tested this using RSX-11m+, RT11 and XXDP... Each one was a little different in behavior - for example, RSX never does a partial block read or write, while RT11 and XXDP do them both during boot and during normal operations. Here's a few screen shots of it in action...

  Running the program with raw command packet trace debug

  Here's RT11 booted from ETU58

  Here's XXDP booted from ETU58

So, here it is...

etu58.mac

  Here's how to build it...

>mac etu58=etu58

>tkb

etu58/pr:0=etu58

/

suplib=fcsfsl:sv (or libr=fcsres:ro)

//

 

  To use, hook up the intended client device to a free terminal line and...

>run etu58

ETU>tt5:=xxxx.dsk,yyyy.dsk

 Where tt5: in the example is the  terminal line the client is connected to

xxxx.dsk and yyyy.dsk are example tu58 disk images.To use only one image,

ttx:=xxxx

  For only unit 0

ttx:=,xxxx

  For only unit 1

  There's one switch, /DBG, that will print out some debug info. It accepts a bitmask that determines what info to print.

BIt 0 - 1  prints out decoded incoming command packets

BIt 1 - 2  prints out raw incoming command packets

Bit 2 - 4  prints out comms related info

Bit 4 - 8  prints out queuing related info.

If you want more than one at a time, add up the values and enter as a decimal value

So, all four would be

  You get the picture...I like using /DBG:2

ETU>tt5:=disk0,disk1/dbg:15

 I like to run this with /DBG:2  - the raw command packets don't take up a lot of screen room, and you can easily see what unit, blocks and lengths are being requested. It was amusing to run this /DBG:2 while using disk1 as a shadow unit to disk 0 on RSX, and watch the load sharing occur (said load sharing was totally ineffectual, performance wise, since the requests are single threaded and not overlapped - still fun to watch). It's also fun to set the TU58 devices cached on RSX, and notice the reduction in IOs that get issued, as requests for blocks get served out of the cache.

  Any problems, let me know...I'm planning on improving this and testing it on more platforms (VAX is next), so I'd appreciate any feedback.

An interesting way to use the Program Counter in MACRO-11

  If you're like me, you enjoy reading well written pieces of MACRO-11 code, in order to grow your skill with it (if you're like me you also enjoy 60s muscle cars, tequila, and the music of the Grateful Dead, but those are subjects for a different post).

  I was reading the code for a boot rom the other day and noticed a clever use of the PC. This rom had a subroutine  that  needed to execute another subroutine twice, then return. If I needed to do that, I'd likely have done something like this...

A:       jsr  PC,B
          (finish
          booting
          here)

B:      (do
         other
         stuff)
         jsr  PC,C
         jsr  PC,C
         rts  PC

C:     (do
        some
        processing)
        rts PC

  But this rom took advantage of the fact that C is adjacent to the end of B. That allowed it to save a few bytes of code. It did...

A:      jsr  PC,B
          (finish
          booting
          here)

B:      (do
         other
         stuff)
         jsr   PC,(PC)

C:   (do
      some
      processing)
      rts    PC

  
  So, have a look at the rom's version of B. It takes advantage of the fact that while the JSR instruction is executing, the PC is already updated to point to the next instruction, which, due to the adjacent location, is at C, and calls it as (PC). That saves a word over calling it as jsr   PC,C. 

       004717                         A:      jsr     PC,(PC)    
vs
       004767  000000           A:      jsr     PC,C
  .
  It save a word, since (PC) is addressing mode 1 (register deferred) - the register contains the address. JSR  PC,C uses mode 6 (indexed, called relative when the PC is the register) - the address is the register value+an offset contained in a word following the instruction.

  Additionally, you  see that there is only one call to C, and no RTS to A in the rom's version of routine B. That's because, when C executes its RTS instruction, it returns to the address following its call in B - which is the beginning of C. C then executes again, and when it hits its rts instruction this time, it causes a return to routine A. That saves several more bytes.

  So, it's not something I'll be making use of everyday, and it can be argued it's a less supportable way to code this, since it will take the next programmer a little thought to figure out what's going on - but it's a handy couple of tricks to know if you're ever tight on space.

The PDP-11 DIV instruction and how to avoid it (sometimes)

  In a MACRO-11 program I've been working on, I have occasion to emulate tabbed text when composing a line for output. Part of that task involves figuring out how many spaces to insert after a text item, to get to the next "virtual" tab stop.

  Easy enough - if I divide the length of the line, including the new item, by 8, then take the remainder from the division and subtract it from 8, that tells me how many spaces I have to append to it to take me to the next tab stop in the output line. The PDP-11 instruction DIV provides the remainder from the division in the higher of the two registers used, so it seems like an obvious choice here.

  Easy enough to describe, but, distasteful for several reasons. The DIV instruction on PDP-11s is one of my least favorite instructions. Actually, all of the EIS instructions (MUL, DIV, ASH, ASHC and XOR) are pains in the sitz-platz. MUL, DIV and ASHC all use two registers, and it's hard for me  to remember which ones hold the high and low words, for arguments and results. When you're trying to make all of your code reentrant and PIC, wasting registers is a real problem. XOR and ASH aren't  too bad - they have single word arguments, in the logical order, but they  aren't completely orthogonal - ASH only works on registers, and you can't XOR two memory locations, for instance. Additionally, EIS instructions aren't available on all PDP11 processors - so as soon as you manage to learn how to use them, you'll find yourself working on an 11/05, and you'll have to fall back on repeated adds, subtracts, and shifts, to get the job done.

  OK, so you get the picture - I'd rather eat a spoonful of dirt than use DIV. But, I have a job to get done here, and it's always easier to dig out the PDP11 handbook and look up how it works again than it is to stop and come up with a good alternative. And that's what I did this time - again. But, I came back a couple days later and realized that, since the divisor in this case is a power of two, I can get the remainder I need practically by inspection.

  To get the remainder of a division by 8, all you have to do is clear  bits 3 and above in the argument (it's bit 3, because 2 to the 3rd power is 8). Logically enough, since by doing that, you clear away everything in the number but the bits that make up a number less than 8 - and that's the definition of the remainder. Have a look at this example...



A:     .word   37.

         bic      #177770,A         ;clear bits 3 to 15

  37 decimal is 100101 binary. Clearing all but the bits 0, 1 and 2 (that is clearing bits 3 and up) leaves us with 101, which is 5 decimal - and that's equal to the remainder of 37/8.

  In like wise, for other powers of two, mask from the appropriate bit up to bit 15, For remainders of 16, for example, bit 4 and up (bit 4, since 2 to the 4th power = 16).

A:     .word   41.

        bic       #177760,A     ;clear bits 4 to 15

  41 decimal is 101001. Clearing all bits except for 0, 1, 2 and 3 leaves us with 1001, which leaves a remainder of 9 in word A - and that's the remainder of 41/16.

  OK, it's not the most profound bit of MACRO-11  math, but, it did save use of an extra register, which means it also most likely saved a push and a pop to save the register before I used it.... And I didn't have to use DIV...