Release 0.10 or later of OpenMCL uses a different memory management scheme than previous versions did. Those earlier versions would allocate a block of memory (of specified size) at startup and would allocate lisp objects within that block. When that block filled with live (non-GCed) objects, the lisp would signal a "heap full" condition. The heap size imposed a limit on the size of the largest object that could be allocated.
The new strategy involves reserving a very large (1GB, by default) block at startup and consuming (and relinquishing) its contents as the size of the live lisp heap data grows and shrinks. After the initial heap image loads and after each full GC, the lisp kernel will try to ensure that a specified amount (the "lisp-heap-gc-threshold") of free memory is available. The inital value of this kernel variable is 4MB; it can be manipulated from Lisp (see below.)
The large reserved memory block consumes very little in the way of system resources; memory that's actually committed to the lisp heap (live data and the "threshold" area where allocation takes place) consumes finite resources (physical memory and swap space). The lisp's consumption of those resources is proportional to its actual memory usage, which is generally a good thing.
This scheme is much more flexible than the old one, but it may also increase the possibility that those resources can become exhausted. Neither the new scheme nor the old handles that situation gracefully; under the old scheme, a program that consumes lots of memory may have run into an artificial limit on heap size before exhausting virtual memory.
The -R
or --heap-reserve
command-line
option can be use to limit the size of the reserved block and
therefore bound heap expansion. Running
% openmcl --heap-reserve 8M
would provide an execution environment that's very similar to that provided by earlier OpenMCL versions.
Syntax | lisp-heap-gc-threshold |
Description | Returns the value of the kernel variable that specifies the amount of free space to leave in the heap after full GC. |
Syntax | lisp-heap-gc-threshold new-threshold |
Description | Sets the value of the kernel variable that specifies the
amount of free space to leave in the heap after full GC to
new-value , which should be a non-negative fixnum.
Returns the value of that kernel variable (which may be somewhat
larger than what was specified |
Arguments |
|
Syntax | use-lisp-heap-gc-threshold |
Description | Tries to grow or shrink lisp's heap space, so that the free space is (approximately) equal to the current heap threshold. Returns NIL |
LinuxPPC imposes a 2GB limit on the address space of a process. It's hard to find more than 1GB of free, contiguous memory at startup under either LinuxPPC or DarwinPPC, so the 1GB figure that OpenMCL reserves seems to be as close to "infinity" as is practical.
The changes in heap allocation strategy don't affect the limits
(ARRAY-DIMENSION-LIMIT
, etc.) on the maximum size of
an allocatable lisp object. Those limits are imposed by OpenMCL's
tagging scheme; ARRAY-DIMENSION-LIMIT could increase by another
5 bits or so (from 24 bits to 29 : you don't want bignums to be
valid array indices, and if you think you do, you're wrong ...),
but doing so would add a few bytes of overhead to every array and
array-like object. This doesn't seem like a worthwhile tradeoff.
SAVE-APPLICATION
identifies code vectors and the
pnames of interned symbols and copies these objects to a "pure"
area of the image file it creates. (The "pure" area accounts
for most of what the ROOM
function reports as "static"
space.)
When the resulting image file is loaded, the pure area of the file is now memory-mapped with read-only access. Code and pure data are paged in from the image file as needed (and don't compete for global virtual memory resources with other memory areas.)
Code-vectors and interned symbol pnames are immutable : it is an error to try to change the contents of such an object. Previously, that error would have manifested itself in some random way. In the new scheme, it'll manifest itself as an "unhandled exception" error in the Lisp kernel. The kernel could probably be made to detect a spurious, accidental write to read-only space and signal a lisp error in that case, but it doesn't yet do so.
The image file should be opened and/or mapped in some mode which
disallows writing to the memory-mapped regions of the file from
other processes. I'm not sure of how to do that; writing to the
file when it's mapped by OpenMCL can have unpredictable and
unpleasant results. SAVE-APPLICATION
will delete
its output file's directory entry and create a new file; one may
need to exercise care when using file system utilities (like
tar
, for instance) that might overwrite an existing image
file.