OPIND, an RSX11 ODS-1 Disk structure utility

  Like I said in a previous posting, I've been tinkering around with the disk structure of Files-11 ODS-1 disks (if you're an RSX11 system programmer, you're bound to want to peek under the hood of your disks eventually).

  In that previous post, I included utility HOMCHK, which could display the home block contents and create new checksums for the home block. It was handy enough - calculating and patching in the checksums was a real chore to do by hand. But it just dealt with the home block, and didn't include any capabilities to make any other changes to the home block or the other disk structures. That left you with using QIOW and logical block programs or using something like ZAP to change values in the disk structure. If you wanted to fool with the file headers, just locating the correct block that corresponds to a particular header is a pain in the sitz-platz - and they have a checksum of their own to deal with.

  So I started work on a program that could do more. I've done a slug of work like this on VAX/VMS ODS-2 disks, so I proceeded in like manner for ODS-1.

  For ODS-2, on VMS, it's easy - you can open the infrastructure files (eg, BITMAP.SYS and INDEXF.SYS) as files and read/write them like any other data file. I started out with that approach on RSX. More than  half  of the disk structure on an ODS-1 disk gets mapped into a single file - [0,0]INDEXF.SYS. It includes the home block, the file header allocation bitmap, and all the file headers, including its own. I wrote a program to open it, just like any other file. This approach really simplifies finding a particular file header, since they are all laid out in sequence, right after the index file bitmap. You just take the file number and add an offset to it that accounts for the home block and the size of the index file bitmap, and boom, there's the block number for that file header - you then use READ$ to read that block of the file.

  The file open succeeded, but then things went wrong. Whenever I tried to read a record, I got an error message. I tried pretty much every linear  combination of open options and sharing settings, spanning the vector space of possibilities, but - no dice. Still got the same error messages.

  A lengthy perusal of the manuals didn't shed any light. Consulting with several learned colleagues - no results there either. I finally searched through the old DECUS RSX SIG tapes for similar programs, and found a couple of old utilities that opened and read INDEXF.SYS. The code and documents  included with these submissions told the tale. It turns out that INDEXF.SYS's header is not really created by F11ACP/FCS/RMS like a normal file header is - it is built manually by INIT when the disk is created. As such, it doesn't include the FCS/RMS file attributes fields that, among other things,  indicate where the highest block in use  and the EOF blocks are. Without these fields having valid values, FCS doesn't know where the records begin and end, so reads and writes fail.

  Below is a snippet of code that illustrates the way around this problem. If you include a statistics block in the FDB for the open, the file's size info is returned to it from other fields in the header, that are filled in correctly - then you can copy it  to the needed fields in the FDB, and normal file accesses can be done.


        mov       #insblk,infdb+f.stbk       ;put address of a 5 word statistics block in the fdb
        opns$r  #infdb                             ;open it for shared read

        mov      insblk+4,infdb+f.hibk       ;get high block number from stat block
        mov      insblk+6,infdb+f.hibk+2   ;get low block number from stat block
        inc        insblk+6                           ;add a one, since eof is +1
        adc       insblk+4                           ;carry, if needs be
        mov      insblk+4,infdb+f.efbk       ;store 'em
        mov      insblk+6,infdb+f.efbk+2

  After loading those field, reads can successfully be done

  OK, so , then I was able to use FCS to read the records in  INDEXF.SYS. I coded a while, adding commands to format and dump headers, examine the bitmap, and what not. I then came to the point in the project where I wanted to add the ability to make changes to the index file. That's where things went wrong again...turns out that this file is already opened for write, and doesn't allow other writers to update blocks in it. That meant accessing it like a normal file was pretty much out the window for this utility. Note - Later on, Johnny Billquist pointed out that if I MOUNT the disk /UNL, I can get write access to the Index File. But, since that limited me to dealing with disks that are more or less uncorrupted, I stuck with the QIO method.

  So, I had to drop normal file access methods and fall back to accessing the disk via IO.RLB and IO.WLB QIOs. Although this greatly complicated the code, it's really a better approach for this tool. Since one of the reasons for using a utility like this is to repair damaged disk structures, it is an advantage to have it be able to work on a disk that won't mount Files-11 anymore. Logical QIOs will work fine on a disk that is mounted foreign, so you can have a shot at fixing badly scrambled up file structures that would prevent a normal disk mount from succeeding. It also allows accessing everything on the disk, not just the structures mapped by INDEXF.SYS.


 Since this utility was converted from using FCS to using QIOWs, it's not a really clean teaching example (I should have rewritten it completely rather than letting it "evolve"). The selection of registers used in subroutines is a bit of a mess, due to the changes in subroutines required, and the part that deals with file headers is a real kludgy hack, so I'm not gonna go to great lengths to explain how it works - it's just enough that it does work....Here's the commands for it...

LOAD - Loads (most) things into the Current Block
  LOAD BLOCK xyz - loads disk block number xyz into the Current Block
  LOAD HEADER xyz - loads header for file with file ID xyz into the Current Block
  LOAD HOME - loads the home block into the Current Block
  LOAD BITMAP - loads the index file header bitmap into the bitmap buffer (NOT the Current
                              Block - the bitmap is usually several blocks long and has its own storage)

DUMP - formats and dumps things to the screen
  DUMP - dumps Current Block to screen
  DUMP /ASC - dumps Current Block to screen in ASCII
  DUMP /OCT - dumps Current Block to screen in octal
  DUMP /HEX - dumps Current Block to screen in hex
  DUMP /R50 - dumps  Current Block to screen in RAD50

  DUMP HEADER xyz - dumps header for file ID xyz to screen - also /HEX, /OCT, /ASC,
                                      R50 (ie DUMP HEADER xyz /R50)

  DUMP HOME - dumps the home block to screen. Also /ASC, /HEX, /OCT and /R50

  DUMP BLOCK xyz - dumps disk block xyz to screen. Also /ASC .HEX, /OCT and /R50

  DUMP BITMAP - dumps index file bitmap to screen - also /HEX and /OCT (R50 and ASC
                              would be silly, nicht wahr?)

WRITE - writes things out
  WRITE HEADER xyz - writes the Current Block to file header xyz
  WRITE BLOCK xyz - writes the Current Block to block xyz
  WRITE HOME - writes the Current Block to the home block
  WRITE BITMAP - writes the bitmap back to disk

TEST - returns the value of a bit in the index file bitmap. Used to see if you have the right
             bit number before you SET or CLEAR a bit.
  TEST xyz - returns value of index file bitmap for header number xyz

SET - sets a bit in the index file bitmap (changes aren't made to disk until WRITE BITMAP
           is done
  SET xyz - sets bit for header for file ID xyz
  
CLEAR - clears a bit in the index file bitmap (changes not done to disk until WRITE BITMAP
  CLEAR xyz - clears bit for header for file ID xyz


FETCH - return byte from Current Block. Used to see if you have the right offset value
               before you deposit a value.
  FETCH @xyz - returns byte in offset xyz in the Current Block
  
DEPOSIT - deposits a value in the Current Block
  DEPOSIT pdq@xyz deposits byte pdq at offset xyz in the Current Block

PUT - writes the Current Block to a temporary file

GET - reads the temporary file into the Current BLock

RELOAD - reloads the home block, index file bitmap and index file header from disk

CHECKSUM - checks and diddles checksums
  CHECKSUM HEADER CHECK - tests the Current Block to see if it has a valid header                                                              checksum
  CHECKSUM HEADER UPDATE - calculates Header checksum for Current Block and
                                                          stores it in the Current Block. Not written to disk until
                                                          you do a WRITE HEADER

  CHECKSUM HOME CHECK - test the Current Block to see if it has valid home block
                                                    checksums

  CHECKSUM HOME UPDATE - calculates home block style checksums (there are two of
                                                     them. Why? Who knows) for the Current Block and stores
                                                     them in the Current Block. Not written to disk until you do
                                                     a WRITE HOME

  So, you can see from the commands above, the Current Block acts kind of like an accumulator - you read something into it, examine and modify it, and then you can write it back out. The temporary file used in GET and PUT persist across invocations, so you can GET  a block from one file/disk, and write it into another.

  Already I'm seeing things that need to be added (word and ASCII string for DEPOSIT, for example), but, it's taken a couple months already, so improvements will have to wait for V2. If you use this and can think of things you'd like in V2, let me know.

  Here's the program

opind.mac


To build
>mac opind=opind
>tkb opind/pr:0=opind

To run
>run opind
What disk?>DU2:
yes?>

  It's built as a /pr:0 task so it can do logical block IO to a mounted disk. You can also use it on a disk mounted foreign. Using it to write to a disk mounted Files-11 that is busy will potentially get you into a sparring match with F11ACP. I've used it on my system disk with no problems, but there wasn't a lot of disk IO going on at the time.


  And, I should mention, you can use this utility to really scramble up an ODS-1 disk. Proceed with caution...use at your own risk - I hope you know what you're doing...practice on a scratch disk. If you are trying to fix a damaged disk, make a physical copy first, if you can. No guarantees - there may still be bugs in here that will cause big problems. 

  A final note -   I'd like to thank Robert Brooks and Andy Goldstein for answering a question about ODS-1 index file use of extension headers - the info saved me a lot of trouble (it allowed me to special case index file extension headers (if there are any)  as occurring only on headers 6 and 7, rather than having to support them potentially being in any header).

RSX program to delete a file by ACP QIO

  Using SIMH to simulate PDP11s is a real convenience. From the comfort of your easy chair, you can start and stop virtual machines at will. You can quickly copy disks to do backups and create copies to perform experiments that might prove fatal.

  Last week, however, I discovered a draw back to this ease of use. While working with several simulated machines, I got confused and started two instances of RSX11M-Plus, using the same system disk, at once.

  It worked with no complaints at all. However, the two copies of the OS were simultaneously accessing the same system disk, without coordination. That means each one was merrily assuming it had the definitive copies of things like the index file, and the storage bitmap, and could allocate things in each one freely. This is a recipe for disaster.

  Fortunately, I don't automatically execute much stuff at boot time, and I noticed what I had done in only a few minutes. I shut them both down and rebooted just one copy. It booted fine and seemed to be working OK, but, I figured, I needed to check things out rather than wait for any disk data damage to grow to a level that could no longer be ignored.

   I booted a different copy of RSX from a different system disk, that was not involved in this lapse of attention, and mounted the possibly damaged system disk as a data disk on that system (you don't want any activity going on on a disk while you run VFY). A VFY run showed it was not as bad as it could be, but, far from as good as it could have been.

  The scan showed that I had multiply allocated blocks - that is, I had at least two files that claimed to have some of the same blocks. That obviously happened when the two systems allocated some blocks that each thought were free - and weren't. Normally VFY tells you what the files involved are, so you can delete at least one of them, and then rebuild the index file and bitmap, to get them to reflect actual usage in the files. But, this time, it only listed one file, and it wasn't in any directory. Worse yet, for some reason, a lost file scan did not restore it into [1,3], and it would not delete successfully using PIP  with the /FI switch (this switch lets you access a file by File ID instead of name). All attempts to access it produced a "bad file header" error message. And the other file it was commingled with, I had no idea what or where it was.

  So, I was in a bit of a pickle. I decided to have a go at deleting it using an ACP QIO with the IO.DEL function. Simple QIOs I figure everybody is familiar with by now - they do IO to devices, like terminals, or tapes, or lots of other things. ACP QIOs are QIOs that issue instructions to the file system of a disk, rather than to the data on the disk itself. They are used for things like creating a file, setting the attributes of a file, entering and removing files from directories, and about a dozen other things, one of which is deleting a file.

 Turns out that this particular call is  a piece of cake - the QIO looks like


         qiow$      io.del,lun,efn,,iostat,,<fid,,,,,>
  
where io.del is the function code, lun is a logical unit number assigned to the disk, efn is an event flag number, iostat is a 2 word buffer for the io status, and fid is address of the three word file id, which consists of one word of file number, one word of sequence number, and one unused word, that was going to be for relative volume number, but RSX11 never got multi-disk volumes (except for SCS-11, but that's another story).

  I cobbled up a macro program that would accept a device name and file ID, and issue the delete QIO. It worked. A new VFY run didn't report on that file, but now reported on the file it had been tangled with as containing blocks that were marked free in the bitmap. That was no surprise, since when I deleted the first file, its blocks were marked as free, which included the blocks contained in the file it was entagled with. I deleted the second file, and then did a VFY with the /UP switch, to mark as used all blocks that are contained in files, and another with the /RE switch, which makes sure the storage bitmap accurately reflects the blocks marked used in file headers, and then a third with no switches, which was error free.

 Anyway, here's the program, on the off chance someone ever has need to delete a file by file ID and PIP won't do the business. In any case, it's a middln'  good example of using TPARS to parse command input. It also uses three of my favorite RSX things - GCML/CSI (well, not much CSI, but GCML), TPARS parsing, and ACP QIOs, which by coincidence are all covered in the same manual, the RSX IO Operations Manual. I spend more time using that manual than all of the rest of them put together.

fiddel.mac


  To use, 

>mac fiddel=fiddel
>tkb 
TKB>fiddel=fiddel
/
Enter Options:
TKB>task=...FID
     (optional libraries)
TKB>libr=fcsres:ro
or
TKB>suplib=fcsfsl:sv
//
>
>run fiddel           (or install it and invoke with FID)
FID>disk:fid:seq

  where disk is the name of the disk, fid is the file number, and seq is the sequence number
eg
FID>DU2:1024:17


  I should mention, that fiddel just deletes the file, but, if the file has a directory entry, it is not deleted. Usually, when you have a problem that fiddel helps with,the file won't have a directory entry - but, if it does, you'll have to delete it separately, using the /RM switch with PIP.

What I'm looking for...RSX/VMS/VAX/PDP11

  This post is about things I'm looking for for my RSX/VMS/VAX/PDP11 collection. I'll be updating it periodically. If you have any of these things, let's talk - I have DEC stuff to trade, and money to spend...

RX01 diskettes

SORT for IAS
RMS11K for IAS
RSX11D any version besides 6.2 and 4
DECNET for RSX11S
DECnet for IAS or RSX11D
MicroVAX 2000 12 MB memory cards
Audio tapes of DECUS Symposium RSX Magic sessions
The DeVIAS newsletter (pre DECUS newsletter consolidation, circa 1984 and before).

DECUS SIG newsletters, including RSX11/IAS Multi-Tasker, DeVIAS, BASIC, Datatrieve, Networks...and others.

  Back in the day, when the earth was young and rock and roll ruled the airwaves, not too long after we slew the dinosaurs with our mighty slide rules, there was an operating system called RSX11M. There was also one called RSX11D, and one called IAS,but no one seems to remember much about them - but that's another story.

  It had an enthusiastic and devoted following. The faithful all read a newsletter from DECUS called "The Multi-Tasker". It was kind of like a fanzine for PDP11 system programmers. And DECUS Symposia were a lot like SF Cons, but, that too, is another story.

  This was long before there was a World Wide Web, where everything is online and freely downloadable. You would subscribe to the Multi-Tasker, and it would be brought to you every month (well, more or less) by a uniformed representative of the Federal Government.

  Those days are long past, as are the RSX family of operating systems, and PDP11s for that matter. But there was much wisdom in these newsletters (there was also much humor, drama and low cunning in these newsletters) that is still of value to the DEC historian and RSX11 hobbyist. I recently had to look for an article in one that I didn't have in my collection, and it took a middin' slug of effort to find it (thanks again, Alan Frisbie).

 There are some issues available on assorted RSX DECUS symposium tapes (a pretty good number of the 1984 and newer are there), and a goodly slug of several years of  them are on Bruce Mitchell's Machine Intelligence and Industrial Magic web site. but many issues are not to be found anywhere online.

  I decided to scan in my collection of DECUS SIG newsletters (Multi-Taskers and others as well)  and have them online here, in case any other hobbyists have a need for them. I'll update this entry as I get them done. These aren't done to the standards of Bitsavers - I just used the scanner built into my wife's printer - but, they're better than nothing...scanning documents is a bit of a pain in the sitz-platz - I have even more appreciation for the work done by Al Kossow than I did before starting this.

Let's start with the DeVIAS Letters (named for the Delaware Valley IAS LUG). I only have a few later issues. I'd really like to find some more.. If you have any you'd like to part with, or loan to me for scanning,  please let me know....

DeVIAS Letters
April 1983
August 1984
May 1985
June 1985

At times, the DeVIAS Letter and the Multi-Tasker were combined into one newsletter - most
likely due to the fact that IAS was being discontinued, and most IAS sites were going to
be switching to RSX.

DeVIAS Letter/Multi-Tasker combined
August 1983
September 1983
November 1983
December 1983
January 1984
February 1984
March 1984
April 1984
May 1984
June 1984


Here's the Multi-Taskers 

Multi-Taskers
December/January 1982
February 1982
March 1982
April 1982 
May/June 1982
July 1982
August 1982
Septermber/October 1982

Here's some Mini-Taskers, the newsletter of the RT11 SIG. RT-11 is OK - I use it on my old PDT-11/150s, but it feels a little limited when compared to RSX.

August 1983
October 1983
January 1984
March 1984
May 1984
December 1984    This ish had a great article on RT-11 internals.


Here's some BASIC SIG newsletters. Some of these have art from Vaughn Bode's JUNKWAFFEL comics.

May 1982
July 1982
April 1983
August 1983
September 1983
January 1984
April 1984
September 1984
October 1984

  The BASIC SIG morphed into the Commercial Languages SIG, which include BASIC, COBOL and DIBOL. They still had plenty of Vaughn Bode's art, though.

January 1985


  The Cache Buffer was the newsletter of the RSTS SIG. RSTS is a pretty good timesharing system. I never really warmed up to it because it was more user oriented - unlike RSX and RT, you didn't get a chance to work with the internals much. IAS, RSX and RT, you were practically invited to roll up your sleeves and start writing internals code.

The Cache Buffer

August 1983
November/December 1983
January 1985
April 1985

Networds, the Networks SIG newsletter

June 1982
May 1983
October 1983

The Wombat Examiner, the puckishly named newsletter of the Datatrieve SIG

June 1982
November 1982
December 1983
March 1983
January 1984
April 1985
June 1985

The Data Management Systems SIG newsletter, usually called The Schema  Some of these have a lot of good RMS info in them.

July 1981
September 1981
February 1982
November 1982


  I have a few copies of  "The Special Character Set", the newsletter of the APL SIG.

September 1983
March 1984
October 1984
April 1985


The Newsletter of the Large Systems SIG (big 36 bit systems), At Large.

March1984  This issue was after DEC announced the end of the 36 bit systems.
April 1985    This issue celebrates the 20th anniversary of the SIG.
June 1985   This issue contains the story "Alice's PDP-10". It's funny.

  The Office Automation, Personal Computer, Graphics, COBOL and DIBOL SIGs got lumped together into a single newsletter. Makes sense...sort of. I guess. It's interesting to read all of the optimistic talk about DEC's plans for Personal Computing....

September 1983
January 1984
March 1984
June 1984

  The UNIX SIG, UNISIG, newsletter was called "Toolkit". To be honest, I loathe UNIX with the white hot incandescence of a galaxy full of exploding supernovae,  but, I have the newsletter, so here it is....

April 1984


  The Languages and Tools SIG had a newsletter called "The Heap".

May 1985

RSX11 program to send morse code via the display register on PiDP11s and real PDP11s.

  I do a fair number of hobby projects here that need a microcontroller to run them. I usually use a vintage processor from the 70s - usually 6800s, and often the 6802, a 6800 that has on board RAM for a stack. Why? Because I read a lot about these processors  in the 70s, but didn't have the time to experiment  with them until relatively lately.

  Anyway, these sorts of projects are pretty minimalist - they don't usually have display or a terminal output. A way to see what's going on is needed during development and debug. An old trick of embedded developers is to use a spare IO pin to drive an LED, which is used  to flash messages about what's going on.

  I usually include this, and instead of just flashing the LED a number of times to communicate during debug, I use a routine that allows it to blink messages in Morse code. This allows me to easily communicate more complicated messages about what's happening than just counting flashes would.

  Like I've mentioned in previous posts, I've been doing a lot of work with my PiDP11/70 - an 11/70 emulator with a front panel. It occurred to me that having this capability on it would be useful when doing work where it is not practical to print a message to a terminal.

  I "translated" the routine I use on 6802s to MACRO-11, as a subroutine, MORSE.MAC. Subroutine MORSE is called with up to four arguments.

R0 - length of string to send
R1 - address of string to send
R2 - sending speed in ticks/element (default is 20)
R3 - LED(s) pattern to use (default is bit 0)

  If R2 or R3 are 0, then the defaults are used. If R2 or R3 specify a value, it is "sticky" - it becomes the default for future calls that have R2 or R3 0.

morse.mac


OK, so now I have a routine to call. But, I figured a program to demo the subroutine is in order, so I wrote MORDEM.MAC (stands for MORse DEMo). MORDEM is just the standard GMCL and CSI gingerbread used by just about all RSX utilties. It accepts a filename as input, and then uses MORSE to send the contents of that file to the display register, in morse code. You can enter TI: as the file argument, and it then takes input from the keyboard.

mordem.mac

To use
>mac mordem=mordem
>mac morse=morse
>tkb
TKB>mordem/pr=mordem,morse
/
task=...mse
libr=fcsres:r0 (or suplib=fcsfsl:sv)
//

Since this is a privileged task, address space is at a premium - and FCS
uses a lot of space.

Use FCSRES or FCSFSL in the taskbuild to conserve address space, or everything won't fit, and when you run it you'll get the message below...and it won't work.
INS -- Warning - privileged task not mapped to I/O page

>run mordem
MSE>ti:      (TI: for input from keyboard - or enter a file spec)
TEST STRING INPUT
SECOND LINE
ETC
^Z

The text entered, from  the keyboard will be blinked out on the display register as morse code.

Or specify a file
>run mordem
MSE>testfile.txt
>

Or install it and do it MCR command line style
>ins mordem
>mse testfile.txt
   or

>mse ti:

  You know, all the usual stuff.

  MORDEM supports a couple of switches...

    /SPE - specify the length in ticks of one element. 20 is the default.
    /LED - specify the bit pattern to blink. Bit 0 is the default.

RSX11 program to display time on PDP11/PiDP11 front panel displays

  I mentioned in a previous post that I have a PiDP11 - a device that lets you simulate a PDP-11/70, with front panel. It still works great. I treat it just like an 11/70, and often forget that it's run by SIMH and a Raspberry PI.

  I have it set up on my desk,  running all the time. Lately, though. I've been thinking that the front panel  can do a lot more than just display the RSX11 idle loop (as amusing as that is). I got to wondering, what can I do with 16 bits worth of display register? Almost immediately, I decided on the tiredest, tritest, most over done hardware hacker's project - a clock!

  I figured, I can display the hours (1-12) in the highest 4 bits, and the minutes and seconds (0-59) in the next 12 bits, 6 bits each. It will be easy for any RSX system programmer to read, and really annoying for anyone else not used to constantly reading bits as octal numbers and converting them to decimal in their head.

  In 30 minutes or so I'd whomped up a program that did just that. But - it was too simple - all it had to do was look up the time, derive some bits for it, and then write it to the display register's address, then pause a second and loop around and do it again. Where's the fun in that? Plus, there ain't much interesting going on in it, and I like to illuminate some interesting aspect of RSX with each of these posts.

  So I decided, to rewrite it to be AST driven. ASTs (stands for Asynchronous Software Trap) are a mechanism where you can have subroutines that get called not in line with your main program, but that interrupt your main program and execute whenever certain things occur - asynchronously (thus the name).

  I'm aware, in this modern cybernetic rocket powered nuclear day and age, where most platforms have the ability to do that, and more, with threads, this sounds like pretty small potatoes. But, this "thread and a half" sort of implementation of ASTs give you a powerful  tool for dealing with things that don't occur on a predictable linear basis.

  ASTs can be triggered by lots of things - IO completion, a timer expiring, a key being typed, and the like. This lends itself to a style of programming where you set up the ASTs to run whenever the events you are interested in occur, and then just wait for them to happen. The main program can consist of just a few lines to set things up, and all the work occurs in the ASTs, when it needs to.

  So in this case, I want the clock to update once a second. RSX provides the MRKT$ directive to create marktime events. A marktime event specifies a span of time. After this time is elapsed, it can set an event flag, call an AST routine, or both. I set a marktime event to occur in 1 second. This event will set an event flag, and call  CLKAST, my clock AST routine, that does all the real work for this project.

  Setting a marktime event does not cause your code to pause and wait for it to occur - your program continues executing after the MRKT$ directive is executed. Since I don't have anything else I need done in the mainline code, I use WTSE$, Wait For Single Event Flag, to cause the mainline code to sleep until the event flag specified in the marktime event is set. When the marktime event comes due, the AST routine is called, and the event flag is set. When the AST routine completes, the WTSE$ wait is satisfied by the event flag being set, so the mainline code proceeds. I have a branch at that point, to have it go back to the beginning and arm everything again, for the second to pass.


  The AST routine, CLKAST, like I say, does all the real work. It reads the system time, formats it as bits into a register, and then writes it into the display register. Then it does an AST exit.

  OK, not the best explanation or example of ASTs in the world . Have a look at the code and it should become a lot clearer....

clock.mac

To use...

>mac clock=clock
>tkb clock/pr=clock
>run clock

  The task is built privileged (the tkb /PR switch above) so it maps the IO page and  can write to the display register.

  It's not going to prompt for any input, so you can just do a Control-C, get a new MCR prompt, and continue working with clock in the background. To stop it, do an ABO command. Or run it at boot time from the console, or from batch - you know, all the usual ways.

  To see the time in the LEDs, turn the lower console knob to "Display Register".

  In closing, I should mention that ASTs can interrupt the mainline code most any time (it's asynchronous, right?). That means, if the mainline code and the AST routine want to access the same variables, you have to give some thought to coordinating access to those variables. In a program more complicated than this example you may need to  disable and enable AST delivery in the mainline code, to create critical sections, if you need some operations to occur uninterrupted. If you start doing much of this, you'll learn all about multitasking, dining with philosophers, and deadlocks - it will be fun!

VAX/VMS page file fragmentation

   Here's an article and utility I wrote circa 1987, about pagefile  fragmentation. 

  It uncovered a problem VMS had that was patched in version 5.2. Although the specific problem that I was looking at was patched long ago, the article and program might be of interest to other VAX/VMS hobbyists out there.

  Here's the program...

pfrag.mar

  To user...

$ mac pfrag
$ link pfrag
$ run pfrag


   Page File Fragmentation Analyzer

  A while back, as Houston started to pull out of an economic
slump, the usage of an 8600 at one of my sites grew dramatically
over the course of a few weeks. As it had been lightly used
before then, the growth was tolerated pretty well. After a
while, though, its performance deteriorated, and response times
became very leisurely. Predictably enough, phones rang, meetings
were held, and harsh words were spoken about system performance.

  SHOW SYSTEM showed many processes were spending a lot of time
in state RWMPW. This state stands for Resource Wait, Modified
Page Writer Busy. A process that needs to fault a page out to
the Modified Page List will be placed in this state if the
Modifed Page Writer (a subroutine of the SWAPPER process) is
busy at the time of the fault. It is normal to occasionally see
a process "pop" through this state, but it was happening a lot
on this system, and accounted for the sluggish response. 

  Another clue was found on the console - the message

%SYSTEM-W-PAGECRIT, Page file space critical, system trying to continue

had been printed out there. Sure enough, a SHOW MEM/FILES showed
that one of the page files had almost filled up. I sheepishly
doubled its size, and re-booted, thinking that would be the end
of the problem. 

  Well, I was wrong about that. A few weeks later, the same
sluggish performance occurred. SHOW SYSTEM once again showed a
lot of processes lingering in state RWMPW. This time, however,
SHOW MEM/FILES showed that the page file in question was only
one third full, with plenty of free space available. I rushed
into the computer room (they can't find me in there), and found 

%SYSTEM-W-PAGEFRAG, Page file badly fragmented, system continuing

printed on the console. Since I knew that all of the page files
were contiguous (I always create them that way) this message had
to be referring to some other type of fragmentation - some data
structure or another must have been in a zillion pieces.
Although I was starting to use a LOT of disk space (Hmmm...do
you think DEC designed the VAX as a virtual memory machine in
order to sell more disk drives???) I doubled the page file size
again, just to try and get the system back on the air, and
resolved to get to the bottom of this. 

  The needed information was found in the usual three places - 
VAX/VMS Internals and Data Structures by Kenah and Bate, 
ANALYZE/SYSTEM, and the VMS microfiche. It turns out that a
page/swap file is just a long string of blocks (or pages,
depending on how you think about it) in a normal Files-11 file.
They contain no internal structure or management information.
Management, location, and allocation information is external to
the file. 
  Each page/swap file is described by a data structure in Non
Paged Pool called a Page File Control Block (henceforth to be
referred to as a PFL block). Here is an annotated example, shown
by ANALYZE/SYSTEM... 

80103800   PFL$L_BITMAP              80103824   addr of the allocation bitmap 
80103804   PFL$L_STARTBYTE       00000000   offset to 1st byte in bitmap
80103808   PFL$W_SIZE                   0028           size of this PFL
8010380A   PFL$B_TYPE                   23              DYN$C_PFL type
8010380B   PFL$B_PFC                     00              Page Fault Cluster size
8010380C   PFL$L_WINDOW            8023FCA0  addr of file WCB
80103810   PFL$L_VBN                     00000000   base vbn ?
80103814   PFL$L_BITMAPSIZ          00000BB8  size of the bitmap in bytes
80103818   PFL$L_FREPAGCNT       00005DC0  free pages in file
8010381C   PFL$L_MAXVBN             003FFFFF  ?
80103820   PFL$W_ERRORCNT        0000          count of errors
80103822   PFL$B_ALLOCSIZ            60              current PFC alloc. size
80103823   PFL$B_FLAGS                 01              status byte
80103824   PFL$L_BITMAPLOC        FFFFFFFF bitmap starts here

  These blocks contain the information needed by the memory
management routines to locate the page/swap files, allocate and
deallocate pages, and read/write data to the files. The PFL
block for each file has an associated allocation bitmap that
records the pages in use in that file. Like a block bitmap for a
Files-11 disk, the PFL bitmap has a bit for every page in the
file, and its state indicates whether the page is free or in
use. A free page is indicated by a bit value of 1, and a page in
use is recorded by a bit value of 0. The address of the bitmap
is at offset PFL$L_BITMAP, and the size is at PFL$L_BITMAPSIZ.
The bitmap always follows the PFL block, at offset
PFL$L_BITMAPLOC. 
  The addresses of the PFL blocks are stored in an array, and
the array is pointed to by MMG$GL_PAGSWPVC. The size of this
array is determined by two SYSGEN parameters - PAGFILCNT and
SWPFILCNT. The sum of these values plus one (the first entry is
always a pointer to a dummy PFL) determines the total number of
entries in the array. Page file and swap file entries in the
array are separated, swap file entries first. MMG$GW_MINPFIDX
contains the smallest array index that is a page file - all
indices less than this are entries for swap files, and those
greater than or equal are page files. Unused array entries all
point to location MMG$GL_NULLPFL, as does the first entry. 

  Armed with this information, I wrote PFRAG - a utility that
reports on the status of the page and swap files. For each PFL
block, PFRAG first gets the name of the associated page/swap
file, using routine GETPAGNAM. PFRAG then calls routine COUNT to
analyze the bitmap, in terms of fragmentation. The results are
formatted and printed out, and the next PFL block is processed,
until all have been examined. 

  The MMG$GL_PAGSWPVC array is used to locate each PFL block. 
Entries in this array that point to MMG$GL_NULLPFL are ignored, 
since that indicates a vacant slot. SYSGEN parameters SWPFILCNT 
and PAGFILCNT determine the size of the array, so a total of 
SGN$GW_PAGFILCNT+SGN$GW_SWPFILCNT+1 entries are examined.

  Routine GETPAGNAM is called to get the name of the file.
GETPAGNAM looks up the address of the file's Window Control
Block in the PFL, and uses it to find the File Control Block.
The FCB contains the file ID of the file. The file ID is passed
to routine GETFILNAM, which uses it to retrieve the file name by
means of XQP QIOs. This works for files that are added
interactively, by the SYSGEN INSTALL command. For files that
were opened during system startup (before the XQP has been
started), the WCB does not point to an FCB, and the file ID is
not available. The file name for these files is inferred by the
location of the PFL block address in the array. If there is a
swap file (a swap file is, by the way, optional), it will be the
first entry after the null one, and will be named
SYS$SYSROOT:[SYSEXE]SWAPFILE.SYS. There will be a page file at
boot time (or the system won't start), and it will be the
MMG$GW_MINPFIDX'th entry in the array, with name
SYS$SYSROOT:[SYSEXE]PAGEFILE.SYS. 

  Routine COUNT is called to determine how many holes are in the
bitmap, and how big the 16 largest holes are, ordered by size.
COUNT uses the FFS (Find First bit Set) instruction to find the
beginnings of unallocated holes, and the FFC (Find First bit
Clear) instruction to find the ends. The total space free, the
number of holes, and the sizes of the 16 largest holes are
returned. 

  After the data for a file has been collected, the results are
formatted and output, then the next PFL is examined, until none 
are left.


  To assemble and link PFRAG...

$ MAC PFRAG
$ MAC COUNT
$ MAC GETFILNAM
$ LINK PFRAG,COUNT,GETFILNAM,SYS$SYSTEM:SYS.STB/SEL,SCSDEF.STB/SEL

Here is a sample run, on the 8600 in question.
$ RUN PFRAG

File $100$DUA0:[SYS1.SYSEXE]SWAPFILE.SYS
Size: 44000      Total Free: 35264      Allocation Size: 96   Holes: 13        
                       Sixteen Largest Holes
    --------    --------    --------    --------    --------     --------    --------     --------
   12768      6336      5600      2304      2304      1536      1440      1344
       768        384        192        192          96            0            0            0

File _$100$DUA0:[SYS1.SYSEXE]SWAPFILE1.SYS;1
Size: 14992      Total Free: 12496      Allocation Size: 96   Holes: 5         
                       Sixteen Largest Holes
   --------    --------    --------     --------  --------  --------  --------  --------
    4896      3664      1920      1344       672         0         0         0
          0            0            0            0           0         0         0         0

File $100$DUA0:[SYS1.SYSEXE]PAGEFILE.SYS
Size: 5000       Total Free: 4806       Allocation Size: 96   Holes: 68        
                       Sixteen Largest Holes
  --------    --------   --------   --------  --------  --------   --------  --------
    2590       449       102        89        87        86        84        74
        71         66         63        59        57        54        49        48

File _$100$DUA6:[PAGE]VAX1PAGEFILE.SYS;1
Size: 60000      Total Free: 58243      Allocation Size: 96   Holes: 201       
                       Sixteen Largest Holes
   --------     --------    --------    --------     --------    --------    --------     --------
   12004      3655      2407      1882      1650      1486      1303      1299
     1009        882        860        831        788        782        718        713

File _$100$DUA7:[PAGE]VAX1PAGEFILE.SYS;1
Size: 64000      Total Free: 50922      Allocation Size: 48   Holes: 4004      
                       Sixteen Largest Holes
 --------  --------  --------  --------   --------  --------   --------  --------
      55        55        55        54        53        53        53        53
      53        52        52        51        51        50        50        50


  The PFRAG utility turned up some interesting facts about the
effects of page file fragmentation. As I suspected, page file
fragmentation tends to increase over time, and does not
completely recover until a system re-boot is performed - kind of
reminds me of POOL fragmentation on my old IAS and RSX systems.
In the sample run, above, the last page file is an example of
just such a state. This run was done in the middle of the night,
when very little was happening on the system. This page file was
80% empty, yet the largest hole was only 55 blocks long, and the
allocation factor was down to 48. When the users got to work in
the morning, this fragmentation would have made itself felt. A
system boot was done, and the problem was avoided. 

  VMS memory management attempts to minimize the amount of
physical IO to the page file by "clustering" together pages to
be written. The Modified Page Writer maintains a field in each
PFL block that tells it how big to try and make the clusters.
This field is set when the file is created to the value stored
in MPW$GW_MPWPFC (from the SYSGEN parameter MPW_WRTCLUSTER, 96
by default). When the Modified Page Writer scans the bitmap and
does not find a contiguous hole of that size, it reduces the
value in that field by 16, and tries again. If the allocation
size falls to 16, the Modified Page Writer uses a worst case
allocation routine that can allocate chunks of size 1 to 16
blocks in length. It is at this time that the Modified Page
Writer prints the fragmentation warning message on the system
console. However, the Modified Page Writer can be a bottleneck
before the warning message appears - a small allocation factor
causes extra scans of the page file bitmap to be performed, and
additional IOs to be done to write modified pages to disk. 

  To avoid this overhead, first make sure that your page and
swap files have a generous amount of free space in them. Then
use PFRAG to determine the amount of fragmentation present. If
the allocation factor for a file falls below 96, try booting the
system to get all the holes in one piece again. If the
allocation factor for a file takes a fast nose dive below 96
after a system boot, then odds are the file could really stand
to be larger - or that you need an(other) alternate page file. 
Try to spread your page and swap file needs across several disks 
and controllers. 
  Try, however, to avoid making these files any larger than is 
strictly necessary. For reasons I have yet to ascertain, large
page/swap files can increase the amount of memory required in
the SYSTEM working set. This can send the number of SYSTEM page
faults through the roof if SYSMWCNT is not large enough to deal
with it. 

  The first time my 8600 slowed down, I really did need a larger 
page file (or files). The second time, though, all the system 
needed was a re-boot, In the future, I won't let that VAX go so 
long between restarts.

Doing RMS IO while in Executive mode is OK

 I write a lot of VAX/VMS programs that change mode to Exec, to get their work done. Lots of interesting VMS data structures are visible (and some are changeable) when you are in Exec mode.

  When your program is working with  these data structures, it's only natural to want to display some of the results. You could collect the data in Exec mode, and return to User node to output it, and that's what a lot of folks do. But that can be a pain in the sitz-platz, switching back and forth all the time. Instead , I don't return to User mode to do that, I just open a file or call lib$put_output (which uses RMS for its IO) and output the data, while still in Exec mode

  "But wait!" many of you are thinking, while you clutch your pearls and recoil in horror. "You can't do RMS IO from Exec mode!!!"  Suffused with a sense of their superiority, people (often otherwise very knowledgeable people) often attempt to correct me  about my ignorant transgressions, of using RMS where it's not welcome. After all, EVERYONE knows that Executive mode is reserved for the workings of RMS - it doesn't make sense for a user's program to do RMS operations in that mode - it would hit its head on the ceiling! Or something...

  But here's the shocking, but, for some reason, not very well known fact - using RMS while in Exec mode is perfectly fine, documented and supported.  In the "OpenVMS Record Management Services Manual", section 2.5, it says...

  2.5 Allowable Program Execution Modes

RMS should not be called from kernel mode, from executive AST mode, or from executive mode when executive-mode ASTs are disabled.

  This spells out when you CAN'T use RMS. This list of places you can't use RMS DOES NOT include non-AST, AST enabled EXEC mode - and that's how I use it. On a serious note - if you can find any documentation from DEC, COMPAQ, HP  or even VSI to the contrary on this issue - I'd sure like to hear about it.

  I don't know how the urban legend, that you can't use RMS while in EXEC mode, got started - but, boy howdy, I can sure tell you it is widespread. I'm thinking about quoting that RMS manual section in all of the code that I distribute, that uses this technique, to try and cut down on the number of times I have to argue the issue.

  Really, if the complainers would stop and think a minute, they'd realize VMS is riddled with examples of RMS activity initiated and performed while in Exec mode. SYS$GETUAI and SYS$PUTUAI for instance, or the opening of process permanent files.

  So, if you spot this occurring in any of my programs, save your breath - I don't need to be "educated" about this.

  Interestingly enough, a few test programs have shown me that RMS file IO and lib$put_output work from Kernel mode as well - but this is documented as not supported, and not a good idea, so I don't do it and don't recommend it. But, if you want to see what it's like, just change the $cmexec_s call in the example program to $cmkrnl_s...it seems to work just fine...

  Anyway, here's an example program.

rmsfromexec.mar

  To use, first make sure you have CMEXEC priv, and then...

$ mac rmsfromexec
$ link rmsfromexec
$ run rmsfromexec
Command me>testfilename.txt

  You'll see the contents of the input file on your screen, and it will be copied to a file called data.dat. The file IO will have been done in EXEC mode, and the skies didn't split. Since it does change mode, however, I have to include a disclaimer- use at your own risk - file IO from EXEC mode is documented and supported, but, no program is perfect. 

RSX Radix-50 conversion utility

  Lately, I've been working on some RSX programs that do a lot of spawning of assorted tasks.

  There were some bugs involved, and I needed to check the task name arguments to the SPWN$S calls.

  These arguments are in RADIX -50 format. RADIX-50, for those of you who haven't had the pleasure of working with it is a way to squeeze three characters into two bytes. It sounds like a lot of trouble, but, it dates from the days when saving a byte here and a byte there was a big win. RADIX-50 has a limited character set - Capital letters, 0 through 9, space, $, . and %. Encoding things in RADIX-50 is why so many things in the RSX world come in multiples of threes - like filenames and extensions, and task names, for example.

  Anyway, I had reason to think some of these variables were not correct in my code. I needed to check them out - but it's hard to look at a 16 bit word and say just what three letters it represents in RADIX-50. I could have written some debug code to evaluate them, but that sounded like even more error prone work - and I didn't go into programming to look for more work to do. I also could have evaluated the words using ODT - it has a function in it that will let you see a word in RADIX-50 format. But, I really really hate ODT. I think it really stands for Odious Debugging Technique. It requires you to have up to date listing and map files handy (preferably printed out - and I hate printing things out). To be honest here, my favorite two debugging techniques for Macro-11 are the diagnostic QIO printout (if you wonder what a value is, add a print QIO to the code and display it), and the much maligned IOT trick (if you are wondering what's happening at some point in your code, put the values of interest in the registers, and then add an IOT instruction - when it gets executed, the task exits and you get to see the register values). A lot of people hate the IOT trick - it can cause problems, for instance, when  fast supervisor mode mapping gets involved. But, it's always handy and is easy to use, so I use it.

  But, like I say, RADIX-50 variables don't lend themselves to these sort of caveman's debugging techniques. So, I figured, time to write a command line utility. I like doing that - it makes me think I'm really accomplishing something.

  So, I wrote R5A. It's a utility that can translate 16 bit words into their three letter RADIX-50 format, and vice versa. It's nothing special - it uses GCML to get a command line, and a few System Library routines to translate octal to binary, binary to ascii, and RADIX-50 back and forth.

r5a.mac
  To build

>MAC R5A=R5A
>TKB R5A=R5A

To use
>RUN R5A
R5A>50712/AS
MCR
^Z

and the other way...
>RUN R5A
R5A>MCR/R5
50712

  There are two switches, that indicate what is on the command line - an R50 string, or an octal string in ascii - /R5 and /AS. /AS is the default, since that's mostly what I need to do - convert words to R50 strings. It doesn't use CSI or TPARS to evaluate the switches, like most of my utilities do (I was in a bit of a hurry, no time for niceties this go round) so the / in the switch has to immediately follow the argument.

  Natch, you can install it and invoke as a whole command line
>INS R5A/TASK=...R5A
>R5A MCR/R5
50712
R5A>




  

Basic and Fortran programs to read temperature and pressure from the BMP driver on the PiDP-11/70

    In my last entry, I wrote about a program to read the BMP180 temperature and pressure sensor connected to a PiDP-11/70, and running the BM device driver.

  That example was written in Macro-11. I understand that not every RSX enthusiast is proficient in that language, so I coded  examples in BASIC-Plus 2 and Fortran 77.

  They do the usual - assign a LUN, and do a QIOW (via WTQIO). The results are formatted and printed out - what could be easier? The conversion of the data was a lot easier in BP2 and F77  than it was in Macro, that's for certain. It did actually take me a few hours to write this - I've written thousands of lines of BP2, but that was circa 1982 - the language has changed quite a bit in the ensuing decades. And, I thought I had written my last FORMAT statement decades ago, but, here one is in the F77 example.


  Here they are....

getoneb.b2s

getonef.ftn

  To use them, compile and link like any other BP2 or F77 program. They will return the temperature and pressure in Celsius, Fahrenheit, Bars and Inches of mercury.

>run getoneb
Temp C 22.60, Temp F 72.68, Press B 1.0171,Press Hg 30.03

>run getonef
Temp C 22.60 Temp F 72.68 Press B 1.0171 Press Hg 30.03