Classic Mac OS memory management

The primary concern of the original engineers appears to have been fragmentation – that is, the repeated allocation and deallocation of memory through pointers leading to many small isolated areas of memory which cannot be used because they are too small, even though the total free memory may be sufficient to satisfy a particular request for memory. To solve this, Apple engineers used the concept of a relocatable handle, a reference to memory which allowed the actual data referred to be moved without invalidating the handle. Apple's scheme was simple – a handle was simply a pointer into a (non-relocatable) table of further pointers, which in turn pointed to the data. If a memory request required compaction of memory, this was done and the table, called the master pointer block, was updated. The machine itself implemented two areas in memory available for this scheme – the system heap (used for the OS), and the application heap. As long as only one application at a time was run, the system worked well. Since the entire application heap was dissolved when the application quit, fragmentation was minimized. The memory management system had weaknesses; the system heap was not protected from errant applications, as would have been possible if the system architecture had supported memory protection, and this was frequently the cause of system problems and crashes. In addition, the handle-based approach also opened up a source of programming errors, where pointers to data within such relocatable blocks could not be guaranteed to remain valid across calls that might cause memory to move. This was a real problem for almost every system API that existed. Because of the transparency of system-owned data structures at the time, the APIs could do little to solve this. Thus the onus was on the programmer not to create such pointers, or at least manage them very carefully by dereferencing all handles after every such API call. Since many programmers were not generally familiar with this approach, early Mac programs suffered frequently from faults arising from this. Palm OS and 16-bit Windows use a similar scheme for memory management, but the Palm and Windows versions make programmer error more difficult. For instance, in Mac OS, to convert a handle to a pointer, a program just de-references the handle directly, but if the handle is not locked, the pointer can become invalid quickly. Calls to lock and unlock handles are not balanced; ten calls to are undone by a single call to . In Palm OS and Windows, handles are an opaque type and must be de-referenced with on Palm OS or on Windows. When a Palm or Windows application is finished with a handle, it calls or . Palm OS and Windows keep a lock count for blocks; after three calls to , a block will only become unlocked after three calls to . Addressing the problem of nested locks and unlocks can be straightforward (although tedious) by employing various methods, but these intrude upon the readability of the associated code block and require awareness and discipline on the part of the coder. ==Memory leaks and stale references==

Awareness and discipline are also necessary to avoid memory "leaks" (failure to deallocate within the scope of the allocation) and to avoid references to stale handles after release (which usually resulted in a hard crash—annoying on a single-tasking system, potentially disastrous if other programs are running). ==Switcher==

The situation worsened with the advent of Switcher, which was a way for a Mac with 512 KB or more of memory to run multiple applications at once. This was a necessary step forward for users, who found the one-app-at-a-time approach very limiting. Because Apple was now committed to its memory management model, as well as compatibility with existing applications, it was forced to adopt a scheme where each application was allocated its own heap from the available RAM. The amount of actual RAM allocated to each heap was set by a value coded into the metadata of each application, set by the programmer. Sometimes this value wasn't enough for particular kinds of work, so the value setting had to be exposed to the user to allow them to tweak the heap size to suit their own requirements. While popular among "power users", this exposure of a technical implementation detail was against the grain of the Mac user philosophy. Apart from exposing users to esoteric technicalities, it was inefficient, since an application would be made to grab all of its allotted RAM, even if it left most of it subsequently unused. Another application might be memory starved, but would be unable to utilize the free memory "owned" by another application. Virtual memory schemes, which made more memory available by paging unused portions of memory to disk, were made available by third-party utilities like Connectix Virtual, and then by Apple in System 7. This increased Macintosh memory capacity at a performance cost, but did not add protected memory or prevent the memory manager's heap compaction that would invalidate some pointers. ==32-bit clean== Originally the Macintosh had 128 KB of RAM, with a true limit of 4 MB, despite being soldered. This limit was first reached with the Macintosh Plus and its user upgradable memory. These first few Macintosh computers use the 68000 CPU, a 32-bit processor which has only 24 physical address lines. The 24 address lines allow the processor to address up to 16 MB of memory (224 bytes), seen as sufficient at the time. The structure of the memory map divided this address space into maximums of 4 MB of RAM, 4 MB of ROM and the remaining 8 MB of addresses split between the SCC, IWM and VIA chips. This was fixed by changing the memory map with the Macintosh II, allowing up to 8 MB of RAM, by shrinking ROM and I/O addresses to 1 MB each and allocating the remaining 6 MB addresses to the NuBus slots. Connectix products MAXIMA, RAM Doubler, and Virtual allowed accessing and reallocating the 6 MB addresses for the NuBus cards for a total of 14 MB, minus 1 MB per slot occupied. Because memory was a scarce resource, the authors of Classic Mac OS decided to take advantage of the unused byte in each address. The original Memory Manager (up until the advent of System 7) placed flags in the high 8 bits of each 32-bit pointer and handle. Each address contained flags such as "locked", "purgeable", or "resource", which were stored in the master pointer table. When used as an actual address, these flags were masked off and ignored by the CPU. ==Object orientation==

The rise of object-oriented languages for programming the Mac – first Object Pascal, then later C++ – also caused problems for the memory model adopted. At first, it would seem natural that objects would be implemented via handles, to gain the advantage of being relocatable. These languages, as they were originally designed, used pointers for objects, which would lead to fragmentation issues. A solution, implemented by the THINK (later Symantec) compilers, was to use Handles internally for objects, but use a pointer syntax to access them. This seemed a good idea at first, but soon deep problems emerged, since programmers could not tell whether they were dealing with a relocatable or fixed block, and so had no way to know whether to take on the task of locking objects or not. Needless to say this led to huge numbers of bugs and problems with these early object implementations. Later compilers did not attempt to do this, but used real pointers, often implementing their own memory allocation schemes to work around the Mac OS memory model. While the Mac OS memory model, with all its inherent problems, remained this way right through to Mac OS 9, due to severe application compatibility constraints, the increasing availability of cheap RAM meant that by and large most users could upgrade their way out of a corner. The memory was not used efficiently, but it was abundant enough that the issue never became critical. This is ironic given that the purpose of the original design was to maximise the use of very limited amounts of memory. finally did away with the whole scheme, implementing a modern paged virtual memory scheme. A subset of the older memory model APIs still exists for compatibility as part of Carbon, but maps to the modern memory manager (a thread-safe malloc| implementation) underneath. ==References==