Saturday, January 20, 2018

Fixes for DISM32 VAX/VMS disassembler

  A couple of weeks back, I had occasion to need to modify DVDRIVER, the VMS device driver that controls floppy drives and ST506 hard drives on MicroVAX/VAXstation 2000s. (see my previous post about 3.5 inch floppies on MV2000s). Peter Coghlan had researched the changes required to make this happen, but his work was for VMS Version 5.5-1 and 5.5-2 - I needed the equivalent mods for VMS 7.3. I needed to find the proper offsets for his changes, in the later VMS version.

  The traditional tool for pulling a driver apart has been DISM32. This utility, written by Andy Pavlin, has been invaluable over the years. It does an incredible job at converting all types of executable files back into MACRO. But, it's an old tool - first written in 1982, and last updated in 1991. It considerably predates VMS 7.3, which is what I'm using.

  I figured, waddahell, let's download it and give it a try. It only took a few minutes Googling to find DISM32 V4.6, the last version, and download it. It came as  a zip file worth of Fortran sources, and a command file to compile and link them all. 

  Right out of the box, it had a problem. The compile failed due to referencing some undefined fields for structure CRFDEF3. A little  research showed that the fields were just FILL positions in the structure. The code wasn't so much using them as checking for a value of 0 - which they should be all the time. So I commented those lines out.

  So, now it compiled, with a slug of informational messages - mostly complaining about unused variables. That didn't bother me - they most likely got added as part of future changes that never got done. It did have a few messages indicating that an unsafe branch  into a loop or block had occurred. That worried me a little more. I checked the indicated spots in the source, and determined that those branches weren't going to cause any problems. So, I tried my freshly linked DISM32 on a few simple executables, and it seemed to still be doing the business.

  Flush with the feeling of success, I gave it a try on DVDRIVER.EXE. No soap - DISM32 blew up and exited after producing only the header part of a MACRO source file. A little more research showed that DISM32 would work fine on some drivers, and blow up on others.

  The error message indicated that a number was overflowing the FORMAT specification (remember FORMAT statements? I barely do...) in a routine that prints out the versions of system libraries that were used.  I increased an I1 spec to an I2, and that problem was over with.

  But, it still didn't complete successfully - now it blew up with a message that indicated that a write to the MACRO output file overflowed its buffer. This error took a little more study, and some time with the debugger. It turned out that the write in question involved the name of the driver, extracted from the Driver Prologue Table. There was support in DISM32 for new style and old style DPTs, which varied in length, and, more importantly, the location of the counted string that holds the driver name. The write statement in question, however, unlike other parts of DISM32 that use the DPT, had not been conditionalized to pick the correct  DPT style - it always used the old style DPT layout . This meant that it wound up using a byte four bytes before the actual counted string, as the length of the name. In this driver, the byte it picked had a large value, so the write failed. In other drivers, that byte happened to contain a smaller value - so those drivers could be decoded successfully.

  I replaced the old style name variable with a variable that would have the correct (old or new) style offset, as required, , and now DISM32 successfully completes.

  On the off chance that there are any other VAX hobbyists out there that need DISM32, I've produced  V4.7, that incorporates the changes described above.

dism32_47.zip

  So, there ya have it...



No comments:

Post a Comment

Comments?