Thursday, June 8, 2023

Unlocking the index file - an example of how to remap APRs

  The other day, I was thinking about working with RSX internals. I've been doing it for a while now, and it occurred to me that I've gotten the bulk of my internals work done with just a /PR task and a call to $SWSTK. That provides access to the exec, pool and the IO page, which are the places  where most internal meddling needs to get done. It also serializes access to things, so nothing can change out from under you while you're doing...whatever it is you're in there to do.

  But that's a small fraction of what's on an RSX11M+ system these days. You've got secondary pool, directive commons, drivers, cache, and other tasks. All things that you may have occasion to look at or change.

  An example of this came up while I was working on another project. I wanted to access the Index File of the system disk, to analyze and fix things gone wrong. It turns out that by default, Files-11 disks on RSX are mounted with the Index File write locked - you can open it for read, but not for write. As well, you can't use some VFY switches (eg, /HD) on a disk with the Index FIle write locked. There are supported ways to mount the system disk with the Index File unlocked.  Instead, on that project, I just did logical QIOs to the disk and bypassed the file system completely. But I was left wondering, if there was a way to unlock the Index File without any special mounting going on.

  A little research showed me that the Index File is accessed by the system as a normally opened file, using the file system. The write locked status is recorded as a 1 in the F.NLKS byte in its FCB - File Control Block - presumably  NLKS stands for Number of Locks. No problem, I figured - find the FCB, change that byte to 0, and I'm on my way. FCBs are bound to be in pool, right? That's where all the interesting data structures tend to be in RSX.

  Well, no, not at all. Turns out that, in recent versions of RSX11M+, the Files-11 ACP (Ancillary Control Program) task, F11ACP, constructs some of the FCBs in its own task address space - and since INDEXF.SYS for the system disk is one of the first files opened at boot time, its FCB is one of the ones that is located there. So,/PR and $SWSTK ain't gonna do the business for accessing it. If I want to manipulate that byte, I'll have to access the data in the F11ACP task in memory.

  I'm sure that you've seen diagrams of what an RSX task looks like. To the average programmer, it looks like up to  64KB of contiguous bytes in memory (for the sake of simplicity, I'm leaving supervisor mode and I&D space out of the discussion) - a 16 bit linear address space. But these tasks coexist in memory with other tasks,  on machines with 18 or 22 bit memory address spaces - so the task address space has to be mapped somehow to the wider address space of the machine. I'm discussing a 22 bit physical space  here - if you're on an 18 bit machine, the process is pretty much the same, just with 4 less bits.

  This mapping is done via memory management hardware on the system. It uses 8 APRs (Address Page Registers) to map each (up to) 8 KB chunk of the task's address space, to a same sized  (up to) 8 KB sized chunk of real memory. The physical memory that is used is determined  by multiplying the contents of the APR by 64 - effectively, moving it over by 6 bits, thus producing  a  22 bit address - Then adding in the low 13 bits of the address from your program.

  Actually each APR is two registers - the Page Address Register, where the relocation value is stored, and the Page Descriptor Register, which contains the properties of that page. But, we don't have to mess with the PDR in most cases, so I'm skipping discussing it.

  Let's look at an example of how a task's 64KB could be mapped to physical memory. It's all text - if you think I have the patience  to draw all that ASCII art, you're crazier than I am. Picture it in your imagination. This is a crazy example, since a real task would have several of these  8 KB chunks arranged together in contiguous physical memory, to simplify loading, but, this is just an example of using the APRs to remap "virtual" 16 bit memory address to 22 bit physical memory.

  FIrst of all, the top three bits in a task address determines what APR will be used to map it. For example, task address 60000's top three bits are 011 - that means it will me mapped by APR 3. Und so weiter.

   Task     Top 3 bits   APR       Physical
  Address     (APR)   Contents     Address
000000-017777   0      000000    00000000-00017777
020000-037777   1      001000    00100000-00117777
040000-057777   2      000020    00002000-00021777
060000-077777   3      100000    10000000-10017777
100000-117777   4      004000    00400000-00417777
120000-137777   5      010000    01000000-01017777
140000-157777   6      000001    00000100-00020077
160000-177777   7      160000    16000000-16017777    

  Pretty wild, nicht wahr? RSX actually abstracts these two spaces, task virtual and physical, and the use of APRs, with two uber-constructs - Windows, for 16 bit task virtual space, and Regions, for physical space. But we're discussing the lowest level, raw stuff of reality here, 

  So, in our mapping example above,  let's consider how we construct a physical address from an example task address,  using task address 120366 (all addresses octal).  In binary,  it's   1 010 000 011 110 110. The  top 3 bits are 101, so it is mapped to the 22 bit physical space according to the contents of APR 5. APR 5 contains 010000. Multiplying it by 64 (decimal) gives us 01000000. Adding the bottom 13 bits of the task address, 366,  gives us the physical address of this word - 01000366.  Got it? What could be easier?

  Note that starting addresses  of blocks in  physical space are on 32 word boundaries, due to the multiplication by 64 (or, if you prefer to think of it another way, the six bit left shift) that will occur when the physcal address is constructed..

  . SInce we'll be fiddling with the APRs, we're gonna need to be running as a privileged task, mapped to the exec, pool and the device page. Since we're considering kernel mode now, we'll be dealing with two sets of APRs - Kernel and User. Here's the mapping used by the Exec, using the Kernel mode APRs.


 Kernel         Top 3  Kernel    APR       Physical
 Address        bits    APR    contents     Memory
 000000-017777   0     KISAR0  000000    00000000-00017777
 020000-037777   1     KISAR1  000200    00020000-00037777
 040000-057777   2     KISAR2  000400    00040000-00057777
 060000-077777   3     KISAR3  000600    00060000-00107777
 100000-117777   4     KISAR4  001000    00120000-00137777 
 120000-137777   5     KISAR5    -               -  
 140000-157777   6     KISAR6    -               -
 160000-177777   7     KISAR7  177600    17760000-17777777

  It uses the Kernel Mode Instruction Space APRs to do its mapping. It maps, one for one, the first 20 K of the executive to the first 20K of physical memory. The IO page gets mapped to the highest 8KB of the physical space. APRs 5 and 6 are used for assorted dynamic mapping functions.

  Now, let's have a look at a priviieged /PR task, before $SWSTK.


 User Mode      Top 3   User    APR        Physical
 Address        bits    APR    contents     Memory
 000000-017777   0     UISAR0 copy KISAR0   00000000-00017777
 020000-037777   1     UISAR1 copy KISAR1   00020000-00037777
 040000-057777   2     UISAR2 copy KISAR2   00040000-00057777
 060000-077777   3     UISAR3 copy KISAR3   00060000-00107777
 100000-117777   4     UISAR4 copy KISAR4   00120000-00137777 
 120000-137777   5     UISAR5 some value     8KB of memory  
 140000-157777   6     UISAR6 some value     8KB of memory
 160000-177777   7     UISAR7 copy KISAR7   17760000-17777777

  The /PR task, before $SWSTK, has copies of KISAR0-KISAR4 in APRs UISAR0-UISAR4. This meeans, the first 20K of the task maps the same physical  memory as the exec. Starting at task address 120000, is the task. APRs 5 and 6 will contain some value, that points to two APRs worth of typical physical memory. This is what contains your program in the /PR task - hope your code is smaller than 16Kb in length... And then APR 7 contains a copy of KISAR7, and points to the IO page.

  Now, when you do $SWSTK, you will be in Kernel mode, and using the Kernel mode APRs. They will have contents as described below...

 Kernel         Top 3  Kernel    APR       Physical
 Address        bits    APR    contents     Memory
 000000-017777   0     KISAR0  000000      00000000-00017777
 020000-037777   1     KISAR1  000200      00020000-00037777
 040000-057777   2     KISAR2  000400      00040000-00057777
 060000-077777   3     KISAR3  000600      00060000-00107777
 100000-117777   4     KISAR4  001000      00120000-00137777 
 120000-137777   5     KISAR5  copy UISR5  8KB of memory
 140000-157777   6     KISAR6  copy UISR6  8KB of memory
 160000-177777   7     KISAR7  177600      17760000-17777777

  You're executing in Kernel mode now, so you are using the Kernel mode APRs. The values in the User mode APRs don't matter. The only change from the normal settings of the Kernel mode APRs is that your task's APR 5 and 6 values have been copied into KISAR5 and 6. Your access to things is serialized now, so you can modify things to be more to your liking without thngs changing out from under you...if you know what you're doing.

  There are much better explanations and diagrams explaining all of this in the PDP-11/70 processor manual, and in the Task Builder manual, in the Privileged Task chapter. 

  Well, there's been a whole lot of typing now without any code or explanations thereof. OK, enough theory - we're actually tring to do something here. Heck. this blog entry is getting longer than the durn example program. Time to discuss what I actually did.

  So anyway, like I said, so long ago, I want to alter the byte value at offset  F.NLKS in the FCB for INDEXF.SYS, so I need the address of that FCB. A look at exec module THF11L.MAC gave me a general notion of how to proceed getting it, and a  perusal of the Data Structures and Lists manual tells me that this is recorded in the Volume Control Block of the disk, at offset V.IFWI. The VCB is pointed to by the UCB for the disk, at offset U.VCB. And the UCB is found by examining the chain of UCBs off of the DCB for the disk controller, looking for one with the right unit number. You find the right DCB by looking from the start of the DCBs at word $DEVHD. This is all pretty straight ahead link list following and value comparing. One wrinkle that affects finding $DEVHD and some other locations in the Exec, is that many of these locations are vectored. This means that their value as recorded in the system symbol table actually contains a pointer (a vector) to the real locations. You have to use the GIN$ (Get INfomation) directvive to make the Exec yield up their true values.

  Here's the directive DPB and its data structure....

        vecdpb: gin$    gi.vec,vecbuf,vecsiz

        vecbuf: .word   0           ;flags
        $kisr6: .word   kisar6      ;kernel apr
        $kisr5: .word   kisar5      ;kernel apr
        devhd:  .word   $devhd      ;device listhead
        pool:   .word   $pool       ;where everything used to go
        exsiz:  .word   $exsiz      ;size of the exec
        vecsiz = <.-vecbuf>/2

  And here's the invocation of the DPB.

        dir$    #vecdpb         ;Get executive vectors

  The table  of vectored locations contains a list of the symbols needed, specifiying the names you'll be using for them, and the directive looks up the real addresses for you.and fills them in.

  Anyhow, after doing the above,  the program does a middlin' amount of checking that the input made sense, and that this is a system where I can find the FCB and change it. Anything sketchy or exotic, it's outta there.

  So, like I said above, the program needs to traverse  the DCB linked list, starting at its head, $DEVHD, When we find a DCB with the correct name, it goes through the array of UCBs attached to that DCB until one is found that matches the unit number from the command line.

  The program does some more checks on the UCB stat field - makes sure that it's mounted, not mounted foreign, and not marked for dismount.

    bitb    #us.mnt!us.mdm!us.for,u.sts(r2) ;it it mounted right?
 
  Note that us.mnt has "negative" logic - bit set means not mounted, bit clear means mounted.. 

 Then, the program gets the address of the VCB (Volume Control Block) from the UCB at offset U.VCB.

        mov     u.vcb(r2),r4    ;addr of vcb to r4

 The VCB has the address of the Index File WCB (Window Control Block) at offset V.IFWI.

        mov     v.ifwi(r4),r4   ;addr of index file window block to r4

 And finally, the WCB has the address of the FCB (File Control Block) at W.FCB.

        mov     w.fcb(r4),r3    ;get addr of fcb from the wcb

  Given that address, the program does some checking to make sure that it has a value that is in the adress range of the F11ACP task. If it passes this scrutiny, we're finally there - mapping  another tasks memory using  APRs, the whole point of this post and program.

  We need to map the F11ACP task with one of  our task's APRs. Since we're a privileged task, you'll recall, we only have APR5 and 6 to work with. Our program is les than 8 KB, so it fits into APR 5. That means that APR 6 is not used - so it's the logical choice.

  But where do we need to map it? The first thing we have to do is to map it to the partition that contains the F11ACP task. Most every task loaded into memory exists in its own system created subpartition. We need the physical address of that partition. Fortunately,  the address of the TCB (Task Control Block) of the ACP task for a mounted disk is stored at offset U.ACP,  in the UCB of that disk.

      mov    u.acp(r2),r2    ;get tcb address of the acp

  Then we can get the PCB (Partition Control Block) address from that TCB, at offset T.PCB.

       mov    t.pcb(r2),r0    ;get pcb address

  After we save the existing contents of KISAR6...

      mov    @$kisr6,-(sp)   ;save current apr 6 value

  ...we can then load the address of the partition into KISAR6. The address of the partition is stored in the PCB at offset P.REL, in "APR" physical address style format - it's already shifted left 6 bits, so it can be loaded straight into an APR.

      mov    p.rel(r0),@$kisr6   ;map to the acp's partition...

  Alrighty then. Now any accesses we make via our APR6 (any task addresses between 140000 anf 157777) will go to the memory used by the  F11ACP's task  partition. Are we ready to access the lock byte now? Not hardly. The first thing that is in a task parition is the task header. We need to access it to find the address of a data segment of the task.

  There's two kinds of F11ACP tasks we could be dealing with - "regular" tasks, and I&D space tasks. These days, most everybody will be running I&D space F11ACP tasks. But, if you're on an 11 that doesn't have  I&D support, we have to be prepared for that eventuality. It makes a difference, because non-I&D tasks only have two address "Windows" (remember above when I said that task address space is abstracted by an uber concept called "Windows") and I&D tasks have 4 address Windows. We have to look into the task header (that we arecurrently mapped to) and get the starting address of the data window. 

  OK, so we will want to access the data windows in the task header. Set up to do that.

    mov    @#140000+h.wnd,r1   ;get the pointer to the window blocks
                               ;add 140000 to it, since we are now
                               ;accessing the partition (and thus
                               ;the header) by our APR 6

  h.wnd is the offset into the task header,  to the task window info.


  Then - which kind of task are we? We're still pointing at the TCB with r2, so we can check easily enough

    bit    #t4.dsp,t.st4(r2)   ;is this an i/d space task?

  If that bit is not set, then this is a normal task, and the data window is the first window.  We find the starting address of that Window 

; This case, we want to get the offset for window #0 - root d-space.
; We need to skip the first word, that is the count of windows in the task,
; and the offset to where the windows start ,w.boff

    mov   2+w.boff(r1),r4      ;window 0 for non I&D system


If it is an I&D task, the data Windows is the second Window. Add in w.blgh (len of a window)
 to the address expression to skip over to it.

40$: mov   2+w.blgh+w.boff(r1),r4 ;get the second window offset

  R4 now contains the offset in the partition to the desired data window. But, handily enough, it's an offset in APR, physical address  format - it's been divided by 64, upper bits trimmed off, and like that So we can add it to the location of the partition that is at p.rel(r), which is also in APR format, and get a value suitable for loading into an APR to point to physical memeory. We have no further need for the task header, so we can now map KISAR6  to  that data window.

    add    p.rel(r0),r4    ;add the  base physical address/32 of
                           ;partition
    mov    r4,@$kisr6      ;point kisar6 at the window


  Finally, we can think about using the address of the FCB that we retrieved, lo these many lines of code ago...well. almost. F11ACP uses APR5 to access that window, and the address it stored in the WCB reflects that. We are using APR6, so we need to add 20000 (octal) to that, so that our APR6 based access uses the right address.

    add    #20000,r3        ;adjust for access via KISAR6

   Finally, we're here - R3 points to the task address that maps to the physical location of the FCB for the Index File.  Check the FIle ID and Sequence Number, to be sure we have the right victim in our sigts. Then we can adjust the count in f.lcks(r3), and we're done. 

    mov     f.fnum(r3),fnum  ;save the file ID number
    mov     f.fseq(r3),fseq  ;save the file sequence number
    mov     f.nacs(r3),fnacb ;save the two bytes that have the...
                             ;..count of accesses and count of locks
    movb    f.nlck(r3),plocks  ;save current count of locks
 
  And then the remaining code does some safety checks and sets or clears, or just reports the value in f.nlcks(r3) as requested)

  When our changes to the lock count are done, all we have to do is restore APR 6, and fall through to the exit from the #SWSTK section. What could be easier?

    mov     (sp)+,@$kisr6    ;put original KISAR6 back

donex:  return               ;drop back to normal space


  Well, that's all done. Heck, it took longer to write about all of this than it did to code the program. You'll never find a worse, more confusing  and incomplete explanation of how memory management works on PDP11s.  I encourage you to seek out the DEC manuals (processor architecture manuals, and the Task Builder manual)  to get the complete whole story.

  Here's the program.



  Here's how to assemble and link it....assuming source is in directory [iul]

>mac iul,iul/-sp=[1,1]exemc.mlb/ml,[11,10]rsxmc/pa:1,[iul]iul
>
>tkb iul/pr/-cp,iul/-sp=iul,[3,54]rsxvec.stb/ss,[1,1]exelib.olb/lb

  And here's how to use it....

>run iul
Target?>DU0:       (whatever disk you're working on)


   Switches are
/CLR
/SET
/LIS

  So, to clear the locked bit,
.run iul
Target?>DU0:/CLR
Status 000001                       how did it go? 1 is success anything else is an error
Locks Before 000001          number of locks before change
Locks After  000000             number of locks after change
FNUM   000001                      File ID number - this better be 1....
FSEQ   000001                       File Sequence number - this better be 1 as well....
FNAC Before 000401          contents of word at F.FNAC(FCB). high byte is number of locks
                                                    low byte is number of accessors
FNAC After  000001             FNAC after change 
K6 before 020356                KISAR6 since before we fool with it
K6 after 013007                    KISAR6 while we fool with it
Window 122250                   addr of WCB

 To set it back on after you're done

>run iul
Target?>DU0:/SET
Status 000001
Locks Before 000000
Locks After  000001
FNUM   000001
FSEQ   000001
FNAC Before 000001
FNAC After  000401
K6 before 020356
K6 after 013007
Window 122250

To see what value it is and not change it...
>run iul
Target?>DU0:/LST
Status 000001
Locks Before 000001
Locks After  000001
FNUM   000001
FSEQ   000001
FNAC Before 000401
FNAC After  000401
K6 before 020356
K6 after 013007
Window 122250
or

Target?>DU0:

  The default action with no switches is to just list the state.

Once again, this is not supported, use at your own risk, This program changes mode to kernel and changes disk structures in memory - it's that sort of code.