DECUS APL-11 fixes for ID space and RSX11M-Plus 4.6

  Recently, Mark Matlock and I were talking about APL-11. This was a version of Iverson's language, APL, for the PDP11. It was written for DEC by OMSI, and sold as a layered product. Later on, it got donated to the DECUS library.

  Mark mentioned that it would be a lot more useful when running on RSX if it had more room in its workspace (an APL workspace is storage for variables and programs). He reckoned that using I&D space with M+ would be a good way to just about double the space available.

  I&D space is a feature found on some of the newer members of the PDP11 family. It allows the use of a second set of APRs (Address Page Registers). It allows a full set of APRs to be used to map the code in a program, and a full second set to map the data. If you have the right mix of code and data, you can wind up with double the address space.

 APL-11 was pretty old, and predated I&D space for RSX. Apparently people in the past had been able to produce I&D space versions, with mixed results, from the DECUS kit, but attempts to do so on current versions of RSX11M+ produced a task that would start and then  immediately exit without doing anything. Even worse, attempts to create even the stock, non-I&D version from the DECUS sources also produced a task that did the same thing. The old .TSK file from the kit, however, runs just fine on M+ 4.6

  I figured, waddahell, I'll have a crack at it. I mean, the sources are included, how hard could it be? A month later, I knew the answer to that question - hard - very hard. Ironically enough, this wasn't my first challenging encounter with APL. In 1971, I took a freshman computer science course. It taught the concepts of programming, using APL and PL/1. I hated both of them and swore never to work with computers again. It was 8 years later, when the typesetting company I worked at got automated using an 11/70 running IAS, that I got exposed to computing again.

  So, anyway, first I get the source kit and load 'er in. The kit had the ability to produce several different variants - single and double precision, with and without floating point processor, and like that. I picked the double precision variant, with FPP support, and got to work. First thing I wanted to do was try and get this thing to build a normal version, without I&D, that would actually  run. Sure enough, following the instructions, assembling and linking it produced a task that would start, and then exit immediately, without printing any error messages. Not a lot to go on there...

>
>run apl7
>

  Normally, I'd link in the PDP11 symbolic debugger and see where it was going wrong - but APL-11 uses an ODL from Hell - the damned thing was over 100 lines long. A few abortive attempts to get the symbolic debugger to work with it, and I fell back to stones knives and bearskins - linking in ODT (Odious Debugging Technique), and sprinkling in IOTs. IOT instructions cause a register dump and exit, or a simple breakpoint when in ODT. Note - in order to get IOT dump and exit to work, it's necessary to change the 1 in the SST vector entry in module ERROR.MAC, to a 0 - otherwise it gets trapped.

SSTVT::                                          ;SST VECTOR TABLE
         .WORD   SYSER        ;ODD OR NON-EXISTENT MEMORY
         .WORD   0                  ;MEMORY PROTECT ERROR
         .WORD   SYSER        ;T-BIT OR BPT
         .WORD   0                   ;IOT INSTRUCTION
         .WORD   SYSER         ;RESERVED INSTRUCTION
        .WORD   TERROR      ;BAD EMT
        .WORD   TERROR      ;TRAP INSTRUCTION

  The program started, logically enough, in a routine called INIT. INIT called a subroutine called $LOAD, with a RAD50 string argument of APLDF1. If $LOAD returns with the carry bit set, INIT calls EXIT$S and...exits. $LOAD was always returning with the carry bit set. Returning with carry bit set is an RSX convention for reporting failure. So, I was wondering, why does $LOAD return a failure status?

  $LOAD turns out to be a system routine that loads an overlay segment, of the name passed as the argument. It was the work of an instant to extract LOAD.OBJ from SYSLIB, and disassemble it with ORCAM (ORCAM is a utility, written by DJ Dunstan and Chris Doran that will disassemble an object file back into MACRO11 source. It's available on the APL11G.DSK virtual disk, UIC [373,221]. ORCAM is MACRO spelled backwards - get it?).

>lbr load.obj = [1,1]syslib.olb/ex:load
>orcam load=load

  So, a perusal of routine $LOAD showed it to be pretty straightforward - it loops through a table of segments (pointed to by N.OVPT, offset N.STBL), looking for a segment with a name that matches the RAD50 string in the argument. But - stepping through the loop and examining the name field of each segment  showed that all of the segments in the table had all 0's in the name field....what's up with that? When it doesn't match any of the strings with the argument, $LOAD hits the end of the table and returns...carry set.

  So...segments got no names...so you can't load them. I recalled that the ZAP utility can list segments in a task, using the /LI switch. I tried it on the APL task I had just made.

>ZAP APL7.TSK/LI
>zap apl7.tsk/li
ZAP Version 04.00  Copyright (c) Mentec, Inc., U.S.A. 1999
Segment table
000002: 000000-056663         Root
000061: 056664-057327           Overlay
000062: 057330-062523           Overlay
000066: 057330-061103            Overlay

  and so forth, for all the segments. The name is supposed to be in there between the address extents and the "root" or "overlay". It occurred to me to have a look at the old .TSK that comes with the DECUS  kit. Here's how it displayed.

>zap apl7ok.tsk/li
ZAP Version 04.00  Copyright (c) Mentec, Inc., U.S.A. 1999
Segment table
000002: 000000-060103 ACOPY   Root
000063: 060104-060577 DECNAM    Overlay
000064: 060600-063707 .CSI1           Overlay
000070: 060600-061373 PARSE        Overlay
000071: 061374-062567 PARSFN      Overlay
000073: 061374-063153 PARSDI       Overlay
000075: 060600-061407 DLFNB        Overlay
and like that...the names of the segments are clearly there...

  So, now I'm seeing why the old one works - it has the names,  and newly constructed ones don't - but...what changed and why? 

  I got to thinking that something changed between the old version of RSX11M the working task that was included in the kit was built on, and the 4.6 version of RSX11M+ I was using. I loaded up a copy of RSX11M 3.2 in SIMH and copied the APL kit to it. Creating a new version of the APL task there produced a task that worked, and that ZAP showed named segments for.  

A lengthy reading session with several different versions of the Task Builder manual shed a little light on this situation. Most of the APL program relies on autoload of segments - but in three places in the program, segments are manually loaded by calling $LOAD instead of autoloading. The TKB manual says that segment names are not used by autoloading, and are not included in the segment descriptors of tasks that use autoloading.

  The manuals all the way back to RSX11m V1 state that the segment names will be omitted in tasks that use autoloading, and included only in tasks that use manual loading - but, up to at least RSX11m V3.2, the segment names are still present in autoloaded tasks. That allows APL to call the manual $LOAD routine successfully - if it was built on  versions of RSX that still had the names in the segment descriptors.

  But, how does the Task Builder know if a task is an autoload task or not, so it can decide to include or omit the segment names?? I created some little test programs that used autoload and manual loading, alone and together, to see what caused a task to know it's an autoload task and not include the segment names. to see. Here's the results..

                                      3.2    4.6
ODL has * & $LOAD                y     n
ODL has $LOAD, no *             y      y
ODL has * only                        n n
ODL has no $LOAD, no *     n      n

  So, it looks like, if the ODL file incudes a reference to module LOAD, in M3.2, it will have segment names included in the segment table, even if it has an * (autoload indicator) in the ODL file. However, M+ 4.6, if it has an * in the ODL, the segment table will have no names, module LOAD present or no. That explains how the old task in the kit, and the version I produced on RSX11M 3.2 could work, but the ones I produced on M+ 4.6 couldn't.

  OK ,so what to do about it? Well, I fiddled around with trying to get the required modules to autoload, so we could drop the calls to $LOAD, but, no soap. Couldn't get it to go.  Maybe I need to know more about how autoload works - but my goal was to produce a bigger version of APL-11, not to spend a lot of time becoming  a perfect master of the arcane TKB arts. 

  As an alternative, I figured, that if $LOAD could be modified to find the right segments, it could LOAD them and things would proceed normally.

  A basic Segment Descriptor looks like this.

   +----------+------------------------------+            | Status   | Relative Disk Address        |
           +----------+------ -----------------------+
           |             Load Address                |
           +-----------------------------------------+
           |           Length in Bytes               |
           +-----------------------------------------+
           |       Link Up (away from root)          |
           +-----------------------------------------+
           |       Link Down (toward root)           |
           +-----------------------------------------+
           |       Link Next (to neighbor)           |
           +-----------------------------------------+
           |             Segment Name                |
           +---               in                  ---+
           |               Radix-50                  |
           +-----------------------------------------+
           |   Window Descriptor Address             |
           +-----------------------------------------+

  There's a few additional optional fields at the end of the descriptor, that support memory resident overlays, and for support of I&D, but they aren't of any interest in this caper.

  So, I need something unique in the Segment Descriptor to identify it by when $LOAD looks for it, instead of the Segment Name, which is all zeroes. Load Address is right out - every overlay segment at the same level in a tree will have the same load address. The links are not unique either. But, the Relative Disk Address (the relative address of what block the segment occurs in the task file) is unique in each descriptor. You can get it from the ZAP /LI, or read it from the MAP file.

  I modified the $LOAD routine to expect a word argument rather than two words of R50, and changed a few lines to it to make it compare that word argument against the Relative Disk Address field while looking for a match. 

  Here's the original (the comments are mine - ORCAM is good, but not THAT good).

26$:    TST     (R2)            ;are we at the end of the segment table yet?
        SEC                     ;set status assuming we are
        BEQ     236$            ;if we were done, jump to exit
        CMP     14(R2),2(R0)    ;do the first three R50 chars match in arg?
        BNE     54$             ;if not, skip to next entry
        CMP     16(R2),4(R0)    ;the first three chars matched - do next 3?
        BEQ     74$             ;if these match, it's our segment, br to loading
                                ;or fall through to loop to next entry
54$:

  Here's the changes...not really much to it...

26$:    TST     (R2)            ;are we at the end of the segment table yet?
        SEC                     ;assume not
        BEQ     236$            ;if not, get out and fail
        mov     (r2),r3         ;move status and disk addr to r3
        bic     #170000,r3      ;mask off the 4 status bits

        cmp     r3,2(r0)        ;is this the segment we're looking for?
        beq     74$             ;if so, go load it
                                ;else fall through, to loop to next
54$:

  LOAD also contained a couple of jumps to an unknown global address, D0000000. I just replaced those with JMP    0 - they were for fatal errors, so a bad address stack dump and exit would do just as well.

  OK, I assemble it,

>mac load=load

and add a factor statement for it to the root of the ODL

BASE0:  .FCTR   gload-O1-*(O11,O13,O14,O15,O16,O17,O18,O19)

and add the factor 

gload:  .fctr   load.obj

  While editing the ODL file, I also delete all references to PATCH or PATCH1, and also edit APL7.CMD and remove line

EXTSCT  =PATCH1:400

  We got no room for patch space - we're trying to make the WS bigger...

 Then I task build it. Trouble - I get  errors from the task build - multiply defined symbols for $AUTO and $ALPBL. Those are for autoloading. The map file shows those symbols present in LOAD module - but that's the routine for manual loading - what's up with that? A look in LOAD.MAC tells the tale. There are a couple of "dummy" definitions for those, so that manual loading tasks don't try and access autoloading code. See below...

$AUTO::
        RTS     PC
$ALBPL::
        MOV     #<IE.OVR&377>,@#$DSW

  But, we need autoloading for most of the program, so highjacking these is right out. I delete from the first occurrence of ".DSABL    LSB" to just before the .END in LOAD.MAC (this also gets rid of a few hundred bytes in an ASECT section that we don't need).

   Success - it task builds with no errors. I run it, and it exits. But, that's what I expect at this point - I haven't put the numbers in for the Relative Block Addresses yet. I edit INIT.MAC, XERROR.MAC, and CLRCOR.MAC (the three macro files that have $LOAD calls in them), and substitute the block numbers for the RAD50 segment names of APLDF1,APLDF2,ROOT1, and ERSEG0 through ERSEG5. I subtract two from the disk number from the map file, since the map file numbers are absolute offsets from the beginning of the task file. Here's one example from INIT

  This is the call to $LOAD - tells us the argument is at ROTSEG

        MOV     #ROTSEG,R0      ;load root data segment
        CALL    $LOAD           ;do it

  Here's ROTSEG, with its original RAD50 argument of APLDF1
ROTSEG: .BYTE   3,0             ;load synchronously
        .RAD50  /APLDF1/        ;low data

  Now find segment APLDF1 in the map file and get the block address.
  Subtract two from it to make it "relative" and put it in as a word
  where APLDF1 R50 string was.

*** Segment: APLDF1
R/W mem  limits: 074724 074735 000012 00010.
Disk blk limits: 000237 000237 000001 00001.

ROTSEG: .BYTE   3,0             ;load synchronously
        .word    235            ;low data
        .word    0              ;filler word
;       .RAD50  /APLDF1/        ;low data

  The filler word is put in there, since the RAD50 name is two words long. Keeping the arglist  the same length is not a bad idea, and is absolutely required in the XERROR segments, since they are in a table indexed by length. I leave the RAD 50 argument in, commented out, since it documents what segment we are mucking with. 

  Finish doing that to the other arguments, reassemble and re-TKB, et viola! We have a runnable task!
>run apl7
TERMINAL..tt
WELCOME TO APL-11 X2.1
CLEAR WS

      a_1 2 3 4 5 6 7 8 9 10
      a
1 2 3 4 5 6 7 8 9 10
  Here, the  command assigns (APL uses "_" for the assignment operator, instead of "=" - Iverson was one of those fussy math types who hate using "=" for assignment. Why "_" instead? Because ASCII character for 95  used to be the left arrow symbol, before they changed it to the underscore. You see the same thing in RSX-15's MCR commands) a vector to variable "a".  Typing "a" causes the value of "a" to be printed out.
Looks like it's working...that's progress.
  OK, so now, after all that work, I'm where I should have started - APL-11, running more or less as it came, with no address space enhancements. This setup can run with a /INC memory increment of 35000 (36000 and above cause "not enough APRs" error message), and produce a workspace of 29720 bytes (.bxwa cause APL to display the workspace size).
>run apl7/inc=35000
TERMINAL..tt

WELCOME TO APL-11 X2.1
CLEAR WS
      .bxwa
29720
  OK, finally,  let's try TKB'ing with /ID added to the task build command. I do so, and  run the task. Natch, it doesn't do anything...adding /ID to the task build command caused all the segments to shift around in the task, so, I get the new locations from the map file, edit them into INIT, CLRCOR and XERROR, re-assemble. re-LBR  and re-TKB. Then, I get a running task. This time, I can use a /INC value of 64000 without getting the "not enough APRs" message. Success! Not so fast...when I run it with /INC that big, it loads and starts, displays the banner- but won't execute any APL commands - it just exits silently when you enter one. Now, after all this work, that's a disappointment. 

  Trying lots of different values for /INC, a pattern emerges, though. /INC values up to 37000 starts a task that runs correctly and executes commands. Going much higher, though, causes the command failure problem. With /INC=37000, the .bxwa command shows a workspace size of 31752. That's getting close to a suspiciously "binary" number. It's getting close to 32767, hex 7FFF - the largest possible positive value for 16 bit 2's complement numbers. Mark Matlock pointed out a note in a DECUS SIG tape submission, by Bob Awde, about a similar problem that occurred in a previous version, where ASR and ASL got used to convert the amount of free space from the number of bytes to number of words, and vice versa. When the size of the WS got large, it could cause a shift left to turn the size negative, or an unwanted sign extension could similarly gum up the works on a shift right. I thought all of those had been  previously fixed, but, I had a look in CLRCOR.MAC and found one case left. I changed

    ASR     R0              ;MAKE A WORD SIZE FOR THE BLOCK

to

    CLC
    ROR     R0              ;MAKE A WORD SIZE FOR THE BLOCK

  The clear carry keeps a bit set in carry from shifting into the sign bit and making the value negative. OK, now assemble, re-LBR and re-TKB (fortunately a two line change didn't cause the segments to shift locations in the task file, so editing the values into the three MAC files from the map file wasn't needed this time) and run it again. This time, it's all roses. Large /INC values don't cause commands to fail. and a much larger /INC is possible. Max is now 64000

 >run apl7/inc=64000
TERMINAL..tt

WELCOME TO APL-11 X2.1
CLEAR WS
      .bxwa
53256

  Alright Alright Alright.... Not bad - we have a workspace now that is over twice as big as when we started. Next, let's take advantage of another address space conserving tactic - the FCS Supervisor mode library. Most modern 11's have Supervisor mode , as well as I&D. Supervisor mode gets us a whole new set of APRs, and it's easy to add. All we have to do is add the line
SUBLIB=FCSFSL:SV
to the TKB command file, and remove all of the references to SYSLIB and NOANSLIB in the ODL file, that called out all the FCS routines by name. Then, re-TKB. Naturally, that caused the segments in the task file to move around...so, edit in the segment disk addresses again, reassemble, re-LBR and re-TKB again...and once again, a running task. However, it didn't gain us much in the way of workspace size. /INC=64000 is still the max /INC you can use (make sense - that's 8 whole APRs worth of increase - ya only get 8 per space).

>run apl7/inc=64000
TERMINAL..tt

WELCOME TO APL-11 X2.1
CLEAR WS
      .bxwa
53272

  We wind up with an increase of around 20 bytes or so. But, it's still worthwhile to use FCSFSL. FCSFSL saves main memory.  FCSFSL also puts all the FCS code into supervisor mode - that means you can now "flatten" the code and remove all of the overlaying there, which will reduce a lot of overlay disk IO. But, that's a whole 'nother story....
  Here's the files for the project...
aaareadme.txt    A step by step guide to changes required to produce APL7.TSK
blog.txt               A copy of this article in friendly ascii text mode
apl11.dsk           An RSX11 virtual RL02 containing the APL11 DECUS kit, no fixes done
apl11g.dsk          An RSX11 virtual RL02 with all fixes and WS increases done. 
apl7.tsk              If you just want to run the task, here's a copy  of the final result.