Saturday, December 5, 2020

RSX utility for reading VMS backup tapes

   Back in the day, as the sun was setting on the RSX world, VAX/VMS grew strong in the land. As many people and sites migrated from PDP-11s to VAXen, there was a steady demand for a VMS utility that could read BRU format tapes (BRU is the RSX Backup and Restore Utility, used to backup up files and disks). Several pretty good solutions were coded up by the user community, and they all worked pretty well.

  Back in September, I had occasion to restore some old VMS files from VMS Backup TK50 and TK70 cassettes.  As it happens, at the moment, I don't have any VAXen that have TK tape drives installed. I do, however, have a couple of RSX systems that have TK drives - so I needed the opposite of the assorted VMS BRUREAD utilities. I did a little looking around, and couldn't find any RSX tools that could read a VMS backup tape.   I guess that doesn't really come as a surprise - the IT world almost always operates by Italian racing rules (what's behind you is not important).

  So, I needed a utility to restore files from a VMS Backup save set to an RSX system. I did a little looking around, and didn't find anything that would do the job. There was a Linux utility that can read VMS Backup tapes (vmsbackup), but it couldn't really deal with getting the file attributes right on the RSX system. I could have modified it to do the job, with a lot of work removing the Linuxisms and inserting RSXisms, but, I'd rather write my own solution in Macro-11 on a clean sheet of paper than spend a lot of time modifying a C program. There was a DECUS RT11 based project to read VMS Backup tapes on RT11, but, it didn't actually ever get completed - and RT11 is a whole lot different from RSX, so it wouldn't have been that helpful. 

  There's not a whole lot of info available about the format and data structures of a VMS Backup save set. I probably have the sources on microfiche - but, I haven't had access to a microfiche reader since I retired from MegaBig Engineering Corp. Most of the information I needed was gleaned by viewing the output of the still undocumented VMS BACKUP/LIST/ANALYZE command on some example save sets on a VMS system. I found some good info about the breakdown and offsets of the data in a Backup set on a VMS Freeware collection (https://www.digiater.nl/openvms/freeware/v30/logging-virtdisk/), as part of an interesting project by Glenn Everhart that provided automatic logging of writes to a device. More info was found in the file system MACRO definitions on both RSX and VMS systems.

  In a nutshell, Backup save sets consist of big blocks, each of which consists of a block header and typically contains one or more  "records". The possible type of records are

Null - null records are used at the end of a block to fill it to length

Summary - Each backup save set volume has a summary record, containing info about the block, including the command that created the save set.

Volume - contains summary information for each volume in the save set.

File - each file saved in  the backup set has a file record, which contains info about the file, including its name and attributes.

VBN - VBN records contain the virtual blocks of a file. These records come after a file record.

Physical - physical record - prolly from backup/physical - I didn't research this, since I am not interested in BACKUP/PHYSICAL save sets. You occasionally find one in the middle of a save set - not sure what that's all about.

LBN - not sure - probably used with physical backups - didn't see any in my research

FID - These appear to contain a list of all the file IDs of the files that are included in an image backup. 


  So, to make a long story short, in a nutshell, to summarize, prècis style, as bullet points, generally speaking, to synopsize, from 10,000 feet, glossing over a heckuva lot of details,  I whomped up a program that reads tape blocks via QIO, finds records in the blocks, and acts on each record. For each File record, I open a file of the indicated name and of the specified size and extent, with the indicated attributes. For each following VBN record after that, I write it sequentially to the file. When I get a new File record, I close the existing file and start a new one. Until we're done. Sounds pretty straightforward, nicht wahr? Yeah, it took a couple months to get it working just so. All of the IO was done via QIOs, including ACP QIOs, to avoid the space that including FCS would have cost us. Address space was an issue (as Cutler used to say, "Size matters") since Backup save set often used very large tape blocks. 

 This code doesn't change mode or muck about with the disk or OS internals using non-supported methods, but, like all my work, it's use at your own risk....

  So, here's the code

BTR.MAC

To build...

mac btr=btr

tkb btr/id=btr

  It has to be a /ID task, because I and D space is required to allow sufficient address space for the large tape input buffer. Sorry, RSX11M and IAS users (they don't have I&D space available).

  To use...

Mount the tape as foreign

>mou mu0: /for

then...

>run btr

  You have to "run btr" instead of installing it and doing BTR commands, since BTR doesn't have GCML and CSI command support - they were omitted to conserve address space.

  It will prompt as BTR>. Enter the name of the tape drive. If that's all you enter, you get a listing of the backup saveset on the tape.

>BTR>mu0:

[GLEASON.ACCOUNT]SETACCOUNT.MAR;31

[GLEASON.ACCOUNTING]Z.MAR;2

[GLEASON.ACP]ATTMOD.MAR;3

[GLEASON.ACP]CHECKUIC.MAR;1

[GLEASON.ACP]CHGCRE.MAR;3

[GLEASON.ACP]CHGDATE.MAR;5

[GLEASON.ACP]DATE.MAR;38

[GLEASON.ACP]GETFILNAM.MAR;1

 And like that.


  If you want to restore all the files, specify an output directory

>BTR>mu0:/out=[somedir]

  If you want to restore or list just some files, use the FIL switch, with wildcards

>BTR>mu0:/fil=*att*.mar  (to list selected files)

>BTR>mu0:/out=[somedir]/fil=*uic*.mar   (to restore selected files)

  If you want to see some of the inner workings while listing or restoring, you can add the /DBG switch, with one of the following values

a.dbg  =   1                    ;print a whole bunch of debug info

a.wld  =   2                    ;print wild card processing

a.iosb =   4                    ;print tape dsw and iosb

a.labl =  10                    ;print label related stuff

a.file =   20                    ;print file activities

a.summ =  40                ;print save set and volume summary info

a.rec  =    100                ;print record processing

a.blk  =    200                 ;print block header info

These are octal values. You can get more than one at a time by adding them together - ie, wild card processing and record processing  together, use 102. Note - some dbg options have bugs in them - /dbg:1 can fail in mid tape, for instance, and /dbg:200 occasionally fails as well. They worked well enough to debug the thing, so fixing them isn't a high priority.

  Since VMS supports 39.39 format filenames (39 chars for filename, 39 chars for extensions) and also has hierarchical directories, a lot of filename wrangling has to get done when restoring files. Filenames are truncated down to the 9.3 format used by RSX. No attempt was made to write them to more than one directory on restore - the output directory you specify is it. Also, VMS supports $,_ and - in filenames. RSX...doesn't. These characters are replaced with X, Y and Z. BTR will happily restore the .DIR files in the save set...but RSX will not use them as directories - they'll just be one more file in the output directory. All this means that you'll have to pay attention to what files you're getting, and what directories they were originally in.

   BTR also restores the .DIR and .SYS files from the backup, so it would be a pretty bad idea to specify [0,0] as the directory to restore to....

 Since a lot of VMS savesets spanned more than one tape, I have included primitive multi-tape support in BTR. When you get to the end of a tape of a multivolume save set, BTR will prompt for the next tape. Hang the next tape and press return to proceed. There's no checking to make sure that you hung the right tape, so...be careful...I also didn't bother adding any support for multi-savesets on a single tape. I mean, who does that? I never did , back when I managed VMS systems.

If you spot any problems using BTR - let me know...I'm working on Version 2, which will have a larger tape buffer, so I'll be fixing any bugs I hear about.