Solaris Virtual Memory

First let's discuss how to get some information about hardware of the system on which the processes run. Total amount of memory installed on particular Solaris system one can get from prtconf command.

$ prtconf
System Configuration: Sun Microsystems sun4u
Memory size: 8192 Megabytes
System Peripherals (Software Nodes):

The number of CPUs present in the system can be retrieved from prtdiag command:

System Configuration: Sun Microsystems sun4u Sun Fire V240

System clock frequency: 160 MHZ

Memory size: 8GB

==================================== CPUs ====================================

E$ CPU CPU Temperature

CPU Freq Size Implementation Mask Die Amb. Status Location

--- -------- ---------- --------------------- ----- ---- ---- ------ --------

0 1280 MHz 1MB SUNW,UltraSPARC-IIIi 2.4 - - online MB/P0

1 1280 MHz 1MB SUNW,UltraSPARC-IIIi 2.4 - - online MB/P1

Amount of free memory on Solaris can be misleading as the system cashes /tmp in the memory.

NEWS CONTENTS

• 20070830 : UNIX memconf utility ( UNIX memconf utility, Aug 30, 2007 )
• 20060116 : Solaris 8 Memory Architecture ( Solaris 8 Memory Architecture, Jan 16, 2006 )
• 20060116 : Solaris Internals book, Second Edition ( Solaris Internals book, Second Edition, )

Old News ;-)

[Aug 30, 2007] UNIX memconf utility

[Jan 16, 2006] Solaris 8 Memory Architecture

A common question pre-Solaris 8 users ask is "Where has all my memory gone"? The vmstat command, used to report virtual memory statistics, often reports that free memory (measured in Kbytes in the free column) is zero or close to zero on a pre-Solaris 8 system that has been up and running for a while.

Most likely, memory is being used to cache file system data, since the virtual memory system is shared by applications, data, the kernel, and file system data. By default, any free memory is used to cache data read from or written to the file system (including NFS). The size of the file system cache is dynamic -- it grows or shrinks depending on free memory.

The idea of this memory allocation scheme is to simultaneously enhance file system performance and optimize the use of an important system resource -- virtual memory. The two computing tasks of running applications and reading and writing data compete equally for system memory.

Generally, sharing a pool of memory is not an issue on small memory systems with low compute power, but with today's powerful desktop systems and servers, the file system cache can overwhelm the memory pool and make application performance suffer. Another drawback is that file system performance is tied to how quickly the virtual memory system can free memory.

Even worse, it is difficult to measure memory usage amongst the consumers of memory on the system. The vmstat command is often the first tool users run to examine virtual memory usage, but pre-Solaris 8 versions do a poor job of indicating why a system is paging (running an algorithm that moves data out from physical memory to disk, and back into physical memory from disk).

So, the question becomes: is it because the system is caching file system data, or is it because memory is a bottleneck and the system is struggling to keep up?

Solaris Internals book, Second Edition

• Solaris 8 mdb ::memstat plugin
• C source for pmap "maps" example
• Perl souce for prtmem.pl example

Recommended Links

Softpanorama Recommended

Top articles

Sites

Solaris Internals

UNIX memconf utility

Solaris 8 Memory Architecture

A common question pre-Solaris 8 users ask is "Where has all my memory gone"? The vmstat command, used to report virtual memory statistics, often reports that free memory (measured in Kbytes in the free column) is zero or close to zero on a pre-Solaris 8 system that has been up and running for a while.

Most likely, memory is being used to cache file system data, since the virtual memory system is shared by applications, data, the kernel, and file system data. By default, any free memory is used to cache data read from or written to the file system (including NFS). The size of the file system cache is dynamic -- it grows or shrinks depending on free memory.

The idea of this memory allocation scheme is to simultaneously enhance file system performance and optimize the use of an important system resource -- virtual memory. The two computing tasks of running applications and reading and writing data compete equally for system memory.

Generally, sharing a pool of memory is not an issue on small memory systems with low compute power, but with today's powerful desktop systems and servers, the file system cache can overwhelm the memory pool and make application performance suffer. Another drawback is that file system performance is tied to how quickly the virtual memory system can free memory.

Even worse, it is difficult to measure memory usage amongst the consumers of memory on the system. The vmstat command is often the first tool users run to examine virtual memory usage, but pre-Solaris 8 versions do a poor job of indicating why a system is paging (running an algorithm that moves data out from physical memory to disk, and back into physical memory from disk).

So, the question becomes: is it because the system is caching file system data, or is it because memory is a bottleneck and the system is struggling to keep up?

The Least Recently Used Paging Algorithm

To better understand how the virtual memory system works, let's examine the least recently used paging algorithm (LRU). The LRU paging algorithm is controlled by the amount of available primary memory. If free primary memory falls below an established watermark, the system attempts to reclaim memory by scanning pages and looking for pages that have not been accessed recently. Pages that meet this criteria are moved to the free list, meaning they can be paged out to the swap device, thus freeing the page in primary memory.

For example, here are the watermarks used by the LRU paging algorithm for a system with 128 Mbytes of primary memory:

When Free Primary Memory Reaches ... The Watermark Is ... And the LRU Algorithm ... 2 MB lotfree (will run up to 120 MHz) runs the page scanner. The page scanner scans pages in memory that have not been used recently and moves them to the free list. 1 MB desfree starts conserving memory used by the kernel 512 KB minfree stops allocating memory for user programs and file system read operations

Priority Paging

To alleviate the pressure on virtual memory induced by heavy file system activity, priority paging was introduced for the Solaris 2.6 and Solaris 7 operating environments.

Priority paging introduces a new watermark in the LRU paging algorithm, cachefree:

When Free Primary Memory Reaches ... The Watermark Is ... And the LRU Algorithm ... 2X lotsfree cachefree (will run up to 120 MHz) only frees pages with file system data that have not been referenced recently. Executables and shared libraries are untouched 2 MB lotsfree runs the page scanner. The page scanner scans pages in memory that have not been used recently and moves them to the free list 1 MB desfree starts conserving memory used by the kernel 512 KB minfree stops allocating memory for user programs and file system read operations

Priority paging helps performance on desktop systems with more than 64 Mbytes of primary memory, especially when a windowing system such as the Common Desktop Environment (CDE) is used.

It is also a boon for servers running online transaction processing (OLTP) applications. OLTP applications will see better response time, as will applications that use batch processing, such as high performance computing (HPC) environments. Also, servers running mixed applications (for example, NFS and database systems) will benefit from priority paging.

By default, priority paging is not enabled on systems running the Solaris 7 operating environment. To enable priority paging, set the following system variable in the /etc/system file and reboot:

   set priority_paging=1

... ... ...

The Solaris 8 Cyclical Page Cache

Priority paging is not available in the Solaris 8 operating environment, for good reason. Solaris 8 introduces a new file system caching architecture, the cyclical page cache, that eliminates most of the problems with virtual memory resulting from file system cache activity.

Caution - Do not set the priority_paging system variable on systems running Solaris 8. Also, be sure to remove this system variable from the /etc/system file on systems to be upgraded to Solaris 8.

The new file system caching architecture in Solaris 8 introduces a new free list, the file system free list. The file system free list is dedicated to caching file system data only -- other objects in virtual memory such as applications, uninitialized application data, the kernel, shared libraries, and so on are managed on a separate free list.

The result is a much cleaner architecture where the file system cache no longer competes for virtual memory resources, since it effectively pages against itself. Heavy file system activity does not force applications out of primary memory -- it now throws only other file system pages out.

Because of these changes, the tools used to report virtual memory statistics report significantly different values that you should take into account when monitoring memory usage on your system.

The most obvious changes are:

• Higher Page Reclaims This is considered normal during heavy file system activity.
• Higher Free Memory Values The amount of free memory will be higher, since the free memory count now includes a large component of the file system cache.
• Zero Scan Rates Scan rates will be almost zero, unless there is a shortage of system wide available memory. Scanning is no longer used to replace the free list during normal file system activity.

The pre-Solaris 8 problem of not being able to differentiate between a true memory shortage or simply heavy file system activity has been eliminated. The scan rate (sr) column in the vmstat command is now a true indication of a memory bottleneck, and any non-zero values appearing in the sr column mean you don't have enough primary memory to keep up with the demand on the system.

Also, the vmstat command has been updated to report virtual memory usage statistics based on the cyclical page cache. Use the vmstat -p command to report paging activity details for applications (executable), data (anonymous), and file system activity, as shown in the following example:

---

% vmstat -p 3
     memory           page          executable      anonymous      filesystem 
   swap  free  re  mf  fr  de  sr  epi  epo  epf  api  apo  apf  fpi  fpo  fpf
 473160 40080   1   2   1   0   0    0    0    0    0    0    0   11    0    0
 416920 45168   0   1   0   0   0    0    0    0    0    0    0    0    0    0
 416920 45168   0   0   0   0   0    0    0    0    0    0    0    0    0    0

---

Tuning virtual memory on systems running Solaris 8 is unnecessary -- the virtual memory system has been optimized for the best performance straight out of the box, providing you with a compelling reason to upgrade your computing environment (especially servers) to Solaris 8.

Memory and swapping

Solaris processes

Supporting Multiple Page Sizes in the Solaris Operating System

Beginning with the Solaris 9 OS, multiple page sizes can be supported on UltraSPARC processors so administrators can optimize performance by changing the page size on behalf of an application. Typical performance measurement tools do not provide sufficient detail for evaluating the impact of page size and do not provide the needed support to make optimal page size choices.
This article explains how to use new tools to determine the potential performance gain. In addition, it explains how to configure larger page sizes using the multiple page size support (MPSS) feature of the Solaris 9 OS. The article addresses the following topics:

• "Understanding Why Virtual-to-Physical Address Translation Affects Performance"
• "Working With Multiple Page Sizes in the Solaris OS"
• "Configuring for Multiple Page Sizes"