/3GB and /PAE switches

I recently had to troubleshoot some old 2003 servers that were experiencing performance, memory and specific services issues such as SNMP would just die with the error code 1501 and the description,The SNMP Service encountered an error while setting up the incoming transports.\n The IP transport has been dropped out.

I found the article below which really helped…

Scenarios using /3GB and /PAE switch and appropriate uses of these switches – full credit to http://blogs.technet.com/b/perfguru/archive/2007/12/05/scenarios-using-3gb-and-pae-switch-and-appropriate-uses-of-these-switches.aspx

Hello Folks,

There seems to be a lot of confusion and myths as far as the /3GB and /PAE Switch being used on the Windows server platform operating systems. I am trying to clarify by the below explanation

You should not use /3GB with /PAE ( >4GB of physical memory) PAE switch if we have more than 4GB of RAM

The /PAE parameter enables Physical Address Extension (PAE). This parameter directs the system to load the PAE version of the Windows kernel. This will allow a 32Bit Operating system to read beyond the 4 Gigabyte limit. (PAE kernel file: Ntkrnlpa.exe)/3GB support, just like AWE support, is included in ‘lower level’ versions of the operating system so that application, utility and adapter vendors can develop and test their products without the need to buy more expensive versions of the operating system

The virtual address space of processes and applications is still limited to 2 GB unless the /3GB switch is used in the Boot.ini file. When the physical RAM in the system exceeds 16 GB and the /3GB switch is used, the operating system will ignore the additional RAM until the /3GB switch is removed. This is because of the increased size of the kernel required to support more Page Table Entries. The assumption is made that the administrator would rather not lose the /3GB functionality silently and automatically; therefore, this requires the administrator to explicitly change this setting.

The /3GB switch allocates 3 GB of virtual address space to an application that uses IMAGE_FILE_LARGE_ADDRESS_AWARE in the process header. This switch allows applications to address 1 GB of additional virtual address space above 2 GB.

The virtual address space of processes and applications is still limited to 2 GB, unless the /3GB switch is used in the Boot.ini file.

The most common problem which we face with /3 GB and /PAE is the issue with allocation of Sys PTEs and OS suffering in performance when high demand for memory is placed.

1. /3 GB switch that was implemented when windows did not support more than 4 GB of RAM. What the 3GB switch did was to give memory intensive applications more virtual address space, ensuring that it could have more files in the memory at any given time. This effectively reduced the number of pages being paged out to the disk, and ensured faster performance of database programs. The fallout was that the kernel space reduced to 1GB, this effectively reduced the memory allocation of sysptes.

2. /PAE switch ensured that the OS is able to see more than 4 GB of RAM. An AWE aware application can take advantage of this and put its pages in the in the upper 4GB of space specified to it. The Downside of this however is that with this, in order to address a page in the memory, the OS needs 2 sysptes instead of 1. So this effectively halves the number of available sysptes.

The virtual address space was designed to be 4 GB in total in the windows architecture which was assumed to be a plethora of space. Later when more and more powerful application stated to come into picture, the virtual address space limit per process was still maintained at 2 GB. On a server with more than 4 GB of RAM we can enable PAE switch and then the application process now starts using 2 GB virtual address space and then with the help of AWE and PAE starts address more physical RAM to max of 64 GB. But onto server’s which have only 4GB or less amount of RAM will limit the virtual address space of the application process to 2 GB.

Now in cases like an exchange server or SQL server with 4 GB RAM, we need the application to gain more resource than the OS itself because the OS is not doing any resource intensive work but the application is doing it. So in those situations we do want the OS to waste the virtual address space which it is not using. Hence we enable the /3GB switch and give the application 3 GB space. During accessing the virtual memory on consistence basis there is lot of fragmentations which happen. Increasing the space to 3 GB, the applications now can leverage allocating larger blocks of memory to itself keeping relative data in one single large block. This enhances the application performances to a great extent. But the caveat to increasing the user space to 3 GB is, if in case the OS needs more than one GB address space it will never get it. So then the /USERVA comes into picture and then we can fine tune and revert back few MBs to the OS from the 3 GB address space allocated to the applications.

On server which use the PAE switch should ideally disable the /3GB because the moment we start using the PAE the OS needs more resources now, and application will start allocation the RAM directly with help of AWE configuration. So when application needs more than 2 GB space it moves to AWE memory blocks and hence the performance remains best.

The conflict is when we use the /3GB with /PAE simultaneously. When we are using PAE the OS needs to manage lot of resources which needs more memory than 1 GB. So certain operations have performance issue and then fail intermittently. To come up with this limitation of address space infinitely the next version of windows that is x64 bit has 16TB of virtual address space and this is virtually inexhaustible.

A program that requests 3 GB of memory is more likely to be able to have more of its memory remains in physical memory rather than be paged out. This increases the performance of programs that are capable of using the /3GB switch.

The theoretical 64-bit virtual address space is 16 exabytes (18,446,744,073,709,551,616 bytes, or approximately 17.2 billion gigabytes). Unlike on the x86 where the default address space is divided in two parts (half for a process and half for the system), the 64-bit address is divided into a number of different sized regions whose components match conceptually the portions of user, system, and session space. The various sizes of these regions, listed in Table below, represent current implementation limits that could easily be extended in future releases. (The sizes on the 32-bit x86 platform are included for comparison purposes.) Clearly, 64-bits provides a tremendous leap in terms of address space sizes.