Here's an article and utility I wrote circa 1987, about pagefile fragmentation.
It uncovered a problem VMS had that was patched in version 5.2. Although the specific problem that I was looking at was patched long ago, the article and program might be of interest to other VAX/VMS hobbyists out there.
Here's the program...
pfrag.mar
To user...
$ mac pfrag
$ link pfrag
$ run pfrag
Page File Fragmentation Analyzer
A while back, as Houston started to pull out of an economic
slump, the usage of an 8600 at one of my sites grew dramatically
over the course of a few weeks. As it had been lightly used
before then, the growth was tolerated pretty well. After a
while, though, its performance deteriorated, and response times
became very leisurely. Predictably enough, phones rang, meetings
were held, and harsh words were spoken about system performance.
SHOW SYSTEM showed many processes were spending a lot of time
in state RWMPW. This state stands for Resource Wait, Modified
Page Writer Busy. A process that needs to fault a page out to
the Modified Page List will be placed in this state if the
Modifed Page Writer (a subroutine of the SWAPPER process) is
busy at the time of the fault. It is normal to occasionally see
a process "pop" through this state, but it was happening a lot
on this system, and accounted for the sluggish response.
Another clue was found on the console - the message
%SYSTEM-W-PAGECRIT, Page file space critical, system trying to continue
had been printed out there. Sure enough, a SHOW MEM/FILES showed
that one of the page files had almost filled up. I sheepishly
doubled its size, and re-booted, thinking that would be the end
of the problem.
Well, I was wrong about that. A few weeks later, the same
sluggish performance occurred. SHOW SYSTEM once again showed a
lot of processes lingering in state RWMPW. This time, however,
SHOW MEM/FILES showed that the page file in question was only
one third full, with plenty of free space available. I rushed
into the computer room (they can't find me in there), and found
%SYSTEM-W-PAGEFRAG, Page file badly fragmented, system continuing
printed on the console. Since I knew that all of the page files
were contiguous (I always create them that way) this message had
to be referring to some other type of fragmentation - some data
structure or another must have been in a zillion pieces.
Although I was starting to use a LOT of disk space (Hmmm...do
you think DEC designed the VAX as a virtual memory machine in
order to sell more disk drives???) I doubled the page file size
again, just to try and get the system back on the air, and
resolved to get to the bottom of this.
The needed information was found in the usual three places -
VAX/VMS Internals and Data Structures by Kenah and Bate,
ANALYZE/SYSTEM, and the VMS microfiche. It turns out that a
page/swap file is just a long string of blocks (or pages,
depending on how you think about it) in a normal Files-11 file.
They contain no internal structure or management information.
Management, location, and allocation information is external to
the file.
Each page/swap file is described by a data structure in Non
Paged Pool called a Page File Control Block (henceforth to be
referred to as a PFL block). Here is an annotated example, shown
by ANALYZE/SYSTEM...
80103800 PFL$L_BITMAP 80103824 addr of the allocation bitmap
80103804 PFL$L_STARTBYTE 00000000 offset to 1st byte in bitmap
80103808 PFL$W_SIZE 0028 size of this PFL
8010380A PFL$B_TYPE 23 DYN$C_PFL type
8010380B PFL$B_PFC 00 Page Fault Cluster size
8010380C PFL$L_WINDOW 8023FCA0 addr of file WCB
80103810 PFL$L_VBN 00000000 base vbn ?
80103814 PFL$L_BITMAPSIZ 00000BB8 size of the bitmap in bytes
80103818 PFL$L_FREPAGCNT 00005DC0 free pages in file
8010381C PFL$L_MAXVBN 003FFFFF ?
80103820 PFL$W_ERRORCNT 0000 count of errors
80103822 PFL$B_ALLOCSIZ 60 current PFC alloc. size
80103823 PFL$B_FLAGS 01 status byte
80103824 PFL$L_BITMAPLOC FFFFFFFF bitmap starts here
These blocks contain the information needed by the memory
management routines to locate the page/swap files, allocate and
deallocate pages, and read/write data to the files. The PFL
block for each file has an associated allocation bitmap that
records the pages in use in that file. Like a block bitmap for a
Files-11 disk, the PFL bitmap has a bit for every page in the
file, and its state indicates whether the page is free or in
use. A free page is indicated by a bit value of 1, and a page in
use is recorded by a bit value of 0. The address of the bitmap
is at offset PFL$L_BITMAP, and the size is at PFL$L_BITMAPSIZ.
The bitmap always follows the PFL block, at offset
PFL$L_BITMAPLOC.
The addresses of the PFL blocks are stored in an array, and
the array is pointed to by MMG$GL_PAGSWPVC. The size of this
array is determined by two SYSGEN parameters - PAGFILCNT and
SWPFILCNT. The sum of these values plus one (the first entry is
always a pointer to a dummy PFL) determines the total number of
entries in the array. Page file and swap file entries in the
array are separated, swap file entries first. MMG$GW_MINPFIDX
contains the smallest array index that is a page file - all
indices less than this are entries for swap files, and those
greater than or equal are page files. Unused array entries all
point to location MMG$GL_NULLPFL, as does the first entry.
Armed with this information, I wrote PFRAG - a utility that
reports on the status of the page and swap files. For each PFL
block, PFRAG first gets the name of the associated page/swap
file, using routine GETPAGNAM. PFRAG then calls routine COUNT to
analyze the bitmap, in terms of fragmentation. The results are
formatted and printed out, and the next PFL block is processed,
until all have been examined.
The MMG$GL_PAGSWPVC array is used to locate each PFL block.
Entries in this array that point to MMG$GL_NULLPFL are ignored,
since that indicates a vacant slot. SYSGEN parameters SWPFILCNT
and PAGFILCNT determine the size of the array, so a total of
SGN$GW_PAGFILCNT+SGN$GW_SWPFILCNT+1 entries are examined.
Routine GETPAGNAM is called to get the name of the file.
GETPAGNAM looks up the address of the file's Window Control
Block in the PFL, and uses it to find the File Control Block.
The FCB contains the file ID of the file. The file ID is passed
to routine GETFILNAM, which uses it to retrieve the file name by
means of XQP QIOs. This works for files that are added
interactively, by the SYSGEN INSTALL command. For files that
were opened during system startup (before the XQP has been
started), the WCB does not point to an FCB, and the file ID is
not available. The file name for these files is inferred by the
location of the PFL block address in the array. If there is a
swap file (a swap file is, by the way, optional), it will be the
first entry after the null one, and will be named
SYS$SYSROOT:[SYSEXE]SWAPFILE.SYS. There will be a page file at
boot time (or the system won't start), and it will be the
MMG$GW_MINPFIDX'th entry in the array, with name
SYS$SYSROOT:[SYSEXE]PAGEFILE.SYS.
Routine COUNT is called to determine how many holes are in the
bitmap, and how big the 16 largest holes are, ordered by size.
COUNT uses the FFS (Find First bit Set) instruction to find the
beginnings of unallocated holes, and the FFC (Find First bit
Clear) instruction to find the ends. The total space free, the
number of holes, and the sizes of the 16 largest holes are
returned.
After the data for a file has been collected, the results are
formatted and output, then the next PFL is examined, until none
are left.
To assemble and link PFRAG...
$ MAC PFRAG
$ MAC COUNT
$ MAC GETFILNAM
$ LINK PFRAG,COUNT,GETFILNAM,SYS$SYSTEM:SYS.STB/SEL,SCSDEF.STB/SEL
Here is a sample run, on the 8600 in question.
$ RUN PFRAG
File $100$DUA0:[SYS1.SYSEXE]SWAPFILE.SYS
Size: 44000 Total Free: 35264 Allocation Size: 96 Holes: 13
Sixteen Largest Holes
-------- -------- -------- -------- -------- -------- -------- --------
12768 6336 5600 2304 2304 1536 1440 1344
768 384 192 192 96 0 0 0
File _$100$DUA0:[SYS1.SYSEXE]SWAPFILE1.SYS;1
Size: 14992 Total Free: 12496 Allocation Size: 96 Holes: 5
Sixteen Largest Holes
-------- -------- -------- -------- -------- -------- -------- --------
4896 3664 1920 1344 672 0 0 0
0 0 0 0 0 0 0 0
File $100$DUA0:[SYS1.SYSEXE]PAGEFILE.SYS
Size: 5000 Total Free: 4806 Allocation Size: 96 Holes: 68
Sixteen Largest Holes
-------- -------- -------- -------- -------- -------- -------- --------
2590 449 102 89 87 86 84 74
71 66 63 59 57 54 49 48
File _$100$DUA6:[PAGE]VAX1PAGEFILE.SYS;1
Size: 60000 Total Free: 58243 Allocation Size: 96 Holes: 201
Sixteen Largest Holes
-------- -------- -------- -------- -------- -------- -------- --------
12004 3655 2407 1882 1650 1486 1303 1299
1009 882 860 831 788 782 718 713
File _$100$DUA7:[PAGE]VAX1PAGEFILE.SYS;1
Size: 64000 Total Free: 50922 Allocation Size: 48 Holes: 4004
Sixteen Largest Holes
-------- -------- -------- -------- -------- -------- -------- --------
55 55 55 54 53 53 53 53
53 52 52 51 51 50 50 50
The PFRAG utility turned up some interesting facts about the
effects of page file fragmentation. As I suspected, page file
fragmentation tends to increase over time, and does not
completely recover until a system re-boot is performed - kind of
reminds me of POOL fragmentation on my old IAS and RSX systems.
In the sample run, above, the last page file is an example of
just such a state. This run was done in the middle of the night,
when very little was happening on the system. This page file was
80% empty, yet the largest hole was only 55 blocks long, and the
allocation factor was down to 48. When the users got to work in
the morning, this fragmentation would have made itself felt. A
system boot was done, and the problem was avoided.
VMS memory management attempts to minimize the amount of
physical IO to the page file by "clustering" together pages to
be written. The Modified Page Writer maintains a field in each
PFL block that tells it how big to try and make the clusters.
This field is set when the file is created to the value stored
in MPW$GW_MPWPFC (from the SYSGEN parameter MPW_WRTCLUSTER, 96
by default). When the Modified Page Writer scans the bitmap and
does not find a contiguous hole of that size, it reduces the
value in that field by 16, and tries again. If the allocation
size falls to 16, the Modified Page Writer uses a worst case
allocation routine that can allocate chunks of size 1 to 16
blocks in length. It is at this time that the Modified Page
Writer prints the fragmentation warning message on the system
console. However, the Modified Page Writer can be a bottleneck
before the warning message appears - a small allocation factor
causes extra scans of the page file bitmap to be performed, and
additional IOs to be done to write modified pages to disk.
To avoid this overhead, first make sure that your page and
swap files have a generous amount of free space in them. Then
use PFRAG to determine the amount of fragmentation present. If
the allocation factor for a file falls below 96, try booting the
system to get all the holes in one piece again. If the
allocation factor for a file takes a fast nose dive below 96
after a system boot, then odds are the file could really stand
to be larger - or that you need an(other) alternate page file.
Try to spread your page and swap file needs across several disks
and controllers.
Try, however, to avoid making these files any larger than is
strictly necessary. For reasons I have yet to ascertain, large
page/swap files can increase the amount of memory required in
the SYSTEM working set. This can send the number of SYSTEM page
faults through the roof if SYSMWCNT is not large enough to deal
with it.
The first time my 8600 slowed down, I really did need a larger
page file (or files). The second time, though, all the system
needed was a re-boot, In the future, I won't let that VAX go so
long between restarts.
No comments:
Post a Comment
Comments?