OPIND, an RSX11 ODS-1 Disk structure utility

  Like I said in a previous posting, I've been tinkering around with the disk structure of Files-11 ODS-1 disks (if you're an RSX11 system programmer, you're bound to want to peek under the hood of your disks eventually).

  In that previous post, I included utility HOMCHK, which could display the home block contents and create new checksums for the home block. It was handy enough - calculating and patching in the checksums was a real chore to do by hand. But it just dealt with the home block, and didn't include any capabilities to make any other changes to the home block or the other disk structures. That left you with using QIOW and logical block programs or using something like ZAP to change values in the disk structure. If you wanted to fool with the file headers, just locating the correct block that corresponds to a particular header is a pain in the sitz-platz - and they have a checksum of their own to deal with.

  So I started work on a program that could do more. I've done a slug of work like this on VAX/VMS ODS-2 disks, so I proceeded in like manner for ODS-1.

  For ODS-2, on VMS, it's easy - you can open the infrastructure files (eg, BITMAP.SYS and INDEXF.SYS) as files and read/write them like any other data file. I started out with that approach on RSX. More than  half  of the disk structure on an ODS-1 disk gets mapped into a single file - [0,0]INDEXF.SYS. It includes the home block, the file header allocation bitmap, and all the file headers, including its own. I wrote a program to open it, just like any other file. This approach really simplifies finding a particular file header, since they are all laid out in sequence, right after the index file bitmap. You just take the file number and add an offset to it that accounts for the home block and the size of the index file bitmap, and boom, there's the block number for that file header - you then use READ$ to read that block of the file.

  The file open succeeded, but then things went wrong. Whenever I tried to read a record, I got an error message. I tried pretty much every linear  combination of open options and sharing settings, spanning the vector space of possibilities, but - no dice. Still got the same error messages.

  A lengthy perusal of the manuals didn't shed any light. Consulting with several learned colleagues - no results there either. I finally searched through the old DECUS RSX SIG tapes for similar programs, and found a couple of old utilities that opened and read INDEXF.SYS. The code and documents  included with these submissions told the tale. It turns out that INDEXF.SYS's header is not really created by F11ACP/FCS/RMS like a normal file header is - it is built manually by INIT when the disk is created. As such, it doesn't include the FCS/RMS file attributes fields that, among other things,  indicate where the highest block in use  and the EOF blocks are. Without these fields having valid values, FCS doesn't know where the records begin and end, so reads and writes fail.

  Below is a snippet of code that illustrates the way around this problem. If you include a statistics block in the FDB for the open, the file's size info is returned to it from other fields in the header, that are filled in correctly - then you can copy it  to the needed fields in the FDB, and normal file accesses can be done.


        mov       #insblk,infdb+f.stbk       ;put address of a 5 word statistics block in the fdb
        opns$r  #infdb                             ;open it for shared read

        mov      insblk+4,infdb+f.hibk       ;get high block number from stat block
        mov      insblk+6,infdb+f.hibk+2   ;get low block number from stat block
        inc        insblk+6                           ;add a one, since eof is +1
        adc       insblk+4                           ;carry, if needs be
        mov      insblk+4,infdb+f.efbk       ;store 'em
        mov      insblk+6,infdb+f.efbk+2

  After loading those field, reads can successfully be done

  OK, so , then I was able to use FCS to read the records in  INDEXF.SYS. I coded a while, adding commands to format and dump headers, examine the bitmap, and what not. I then came to the point in the project where I wanted to add the ability to make changes to the index file. That's where things went wrong again...turns out that this file is already opened for write, and doesn't allow other writers to update blocks in it. That meant accessing it like a normal file was pretty much out the window for this utility. Note - Later on, Johnny Billquist pointed out that if I MOUNT the disk /UNL, I can get write access to the Index File. But, since that limited me to dealing with disks that are more or less uncorrupted, I stuck with the QIO method.

  So, I had to drop normal file access methods and fall back to accessing the disk via IO.RLB and IO.WLB QIOs. Although this greatly complicated the code, it's really a better approach for this tool. Since one of the reasons for using a utility like this is to repair damaged disk structures, it is an advantage to have it be able to work on a disk that won't mount Files-11 anymore. Logical QIOs will work fine on a disk that is mounted foreign, so you can have a shot at fixing badly scrambled up file structures that would prevent a normal disk mount from succeeding. It also allows accessing everything on the disk, not just the structures mapped by INDEXF.SYS.


 Since this utility was converted from using FCS to using QIOWs, it's not a really clean teaching example (I should have rewritten it completely rather than letting it "evolve"). The selection of registers used in subroutines is a bit of a mess, due to the changes in subroutines required, and the part that deals with file headers is a real kludgy hack, so I'm not gonna go to great lengths to explain how it works - it's just enough that it does work....Here's the commands for it...

LOAD - Loads (most) things into the Current Block
  LOAD BLOCK xyz - loads disk block number xyz into the Current Block
  LOAD HEADER xyz - loads header for file with file ID xyz into the Current Block
  LOAD HOME - loads the home block into the Current Block
  LOAD BITMAP - loads the index file header bitmap into the bitmap buffer (NOT the Current
                              Block - the bitmap is usually several blocks long and has its own storage)

DUMP - formats and dumps things to the screen
  DUMP - dumps Current Block to screen
  DUMP /ASC - dumps Current Block to screen in ASCII
  DUMP /OCT - dumps Current Block to screen in octal
  DUMP /HEX - dumps Current Block to screen in hex
  DUMP /R50 - dumps  Current Block to screen in RAD50

  DUMP HEADER xyz - dumps header for file ID xyz to screen - also /HEX, /OCT, /ASC,
                                      R50 (ie DUMP HEADER xyz /R50)

  DUMP HOME - dumps the home block to screen. Also /ASC, /HEX, /OCT and /R50

  DUMP BLOCK xyz - dumps disk block xyz to screen. Also /ASC .HEX, /OCT and /R50

  DUMP BITMAP - dumps index file bitmap to screen - also /HEX and /OCT (R50 and ASC
                              would be silly, nicht wahr?)

WRITE - writes things out
  WRITE HEADER xyz - writes the Current Block to file header xyz
  WRITE BLOCK xyz - writes the Current Block to block xyz
  WRITE HOME - writes the Current Block to the home block
  WRITE BITMAP - writes the bitmap back to disk

TEST - returns the value of a bit in the index file bitmap. Used to see if you have the right
             bit number before you SET or CLEAR a bit.
  TEST xyz - returns value of index file bitmap for header number xyz

SET - sets a bit in the index file bitmap (changes aren't made to disk until WRITE BITMAP
           is done
  SET xyz - sets bit for header for file ID xyz
  
CLEAR - clears a bit in the index file bitmap (changes not done to disk until WRITE BITMAP
  CLEAR xyz - clears bit for header for file ID xyz


FETCH - return byte from Current Block. Used to see if you have the right offset value
               before you deposit a value.
  FETCH @xyz - returns byte in offset xyz in the Current Block
  
DEPOSIT - deposits a value in the Current Block
  DEPOSIT pdq@xyz deposits byte pdq at offset xyz in the Current Block

PUT - writes the Current Block to a temporary file

GET - reads the temporary file into the Current BLock

RELOAD - reloads the home block, index file bitmap and index file header from disk

CHECKSUM - checks and diddles checksums
  CHECKSUM HEADER CHECK - tests the Current Block to see if it has a valid header                                                              checksum
  CHECKSUM HEADER UPDATE - calculates Header checksum for Current Block and
                                                          stores it in the Current Block. Not written to disk until
                                                          you do a WRITE HEADER

  CHECKSUM HOME CHECK - test the Current Block to see if it has valid home block
                                                    checksums

  CHECKSUM HOME UPDATE - calculates home block style checksums (there are two of
                                                     them. Why? Who knows) for the Current Block and stores
                                                     them in the Current Block. Not written to disk until you do
                                                     a WRITE HOME

  So, you can see from the commands above, the Current Block acts kind of like an accumulator - you read something into it, examine and modify it, and then you can write it back out. The temporary file used in GET and PUT persist across invocations, so you can GET  a block from one file/disk, and write it into another.

  Already I'm seeing things that need to be added (word and ASCII string for DEPOSIT, for example), but, it's taken a couple months already, so improvements will have to wait for V2. If you use this and can think of things you'd like in V2, let me know.

  Here's the program

opind.mac


To build
>mac opind=opind
>tkb opind/pr:0=opind

To run
>run opind
What disk?>DU2:
yes?>

  It's built as a /pr:0 task so it can do logical block IO to a mounted disk. You can also use it on a disk mounted foreign. Using it to write to a disk mounted Files-11 that is busy will potentially get you into a sparring match with F11ACP. I've used it on my system disk with no problems, but there wasn't a lot of disk IO going on at the time.


  And, I should mention, you can use this utility to really scramble up an ODS-1 disk. Proceed with caution...use at your own risk - I hope you know what you're doing...practice on a scratch disk. If you are trying to fix a damaged disk, make a physical copy first, if you can. No guarantees - there may still be bugs in here that will cause big problems. 

  A final note -   I'd like to thank Robert Brooks and Andy Goldstein for answering a question about ODS-1 index file use of extension headers - the info saved me a lot of trouble (it allowed me to special case index file extension headers (if there are any)  as occurring only on headers 6 and 7, rather than having to support them potentially being in any header).