| Main | Site Index | Download | Disclaimer | Privacy |


 

 

 

 

Windows’s Virtual Address Space: A Short MSDN Story

 

 

 

What do we have in this Module?

  1. Introduction

  2. Virtual Address Space and Physical Storage

  3. Page State

  4. Scope of Allocated Memory

  5. Memory Protection

  6. Copy-on-Write Protection

  7. Loading Applications and DLLs

  8. Virtual Address Space – 64 bits

 

 

 

 

 

 

 

 

 

 

Introduction

 

The virtual addresses used by a process do not represent the actual physical location of an object in memory. Instead, the system maintains a page map for each process, which is an internal data structure used to translate virtual addresses into corresponding physical addresses. Each time a thread references an address, the system translates the virtual address to a physical address. The virtual address space is divided into partitions as follows: The 2 GB partition in low memory (0x00000000 through 0x7FFFFFFF) is available to the process, and the 2 GB partition in high memory (0x80000000 through 0xFFFFFFFF) is reserved for the system. Windows Server 2003 Enterprise Edition, Windows 2000 Advanced Server, and Windows NT 4.0 SP3 Server Enterprise Edition:  Optionally, the 3 GB partition in low memory (0x00000000 through 0xBFFFFFFF) is available to the process, and the 1 GB partition in high memory (0xC0000000 through 0xFFFFFFFF) is reserved for the system.

Windows Me/98/95:  The following table describes the partitions.

 

Range

Usage

0K - ~64K (0xFFFF)

Not writable. This boundary is approximate due to the way the system loads some features of Microsoft® MS-DOS®. This memory is private to the process.

~64K (0x10000) - 4 MB (0x3FFFFF)

Reserved for MS-DOS compatibility. This memory is fully readable and writable by the process. However, this range of memory may have some MS-DOS–related structures or code in it, so processes should not arbitrarily read from or write to it. This memory is private to the process.

4MB (0x400000) - 2GB (0x7FFFFFFF)

Available for code and user data. User data is readable and writable by the process. Code is execute-only. This memory is private to the process.

2GB (0x80000000) - 3GB (0xBFFFFFFF)

Shared area, readable and writable by all processes. A number of system DLLs and other data are loaded into this space.

3GB (0xC0000000) - 4GB (0xFFFFFFFF)

System memory, readable or writable by any process. However, this is where low-level system code resides, so writing to this region may corrupt the system, with potentially catastrophic consequences.

 

Table 1.

 

Virtual Address Space and Physical Storage

 

The virtual address space of each process is much larger than the total physical memory available to all processes. To increase the size of physical storage, the system uses the disk for additional storage. The total amount of storage available to all executing processes is the sum of the physical memory and the free space on disk available to the paging file, a disk file used to increase the amount of physical storage. Physical storage and the virtual address space of each process are organized into pages, units of memory, whose size depends on the host computer. For example, on x86 computers the host page size is 4 kilobytes.

To maximize its flexibility in managing memory, the system can move pages of physical memory to and from a paging file on disk. When a page is moved in physical memory, the system updates the page maps of the affected processes. When the system needs space in physical memory, it moves the least recently used pages of physical memory to the paging file. Manipulation of physical memory by the system is completely transparent to applications, which operate only in their virtual address spaces.

 

Page State

 

The pages of a process's virtual address space can be in one of the following states.

 

State

Description

Free

The page is neither committed nor reserved. The page is not accessible to the process. It is available to be committed, reserved, or simultaneously reserved and committed. Attempting to read from or write to a free page results in an access violation exception.

A process can use the VirtualFree() or VirtualFreeEx() function to release reserved or committed pages of its address space, returning them to the free state.

Reserved

The page has been reserved for future use. The range of addresses cannot be used by other allocation functions. The page is not accessible and has no physical storage associated with it. It is available to be committed.

A process can use the VirtualAlloc() or VirtualAllocEx() function to reserve pages of its address space and later to commit the reserved pages. It can use VirtualFree() or VirtualFreeEx() to decommit committed pages and return them to the reserved state.

Committed

Physical storage is allocated for the page, and access is controlled by a memory protection option. The system initializes and loads each committed page into physical memory only during the first attempt to read or write to that page. When the process terminates, the system releases the storage for committed pages.

A process can use VirtualAlloc() or VirtualAllocEx() to allocate committed pages. These functions can commit pages that are already committed. The GlobalAlloc() and LocalAlloc() functions allocate committed pages with read/write access.

 

Table 2.

 

Scope of Allocated Memory

 

All memory a process allocates by using the memory allocation functions (HeapAlloc(), VirtualAlloc(), GlobalAlloc(), or LocalAlloc()) is accessible only to the process. However, memory allocated by a DLL is allocated in the address space of the process that called the DLL and is not accessible to other processes using the same DLL. To create shared memory, you must use file mapping.

Named file mapping provides an easy way to create a block of shared memory. A process can specify a name when it uses the CreateFileMapping() function to create a file-mapping object. Other processes can specify the same name to either the CreateFileMapping() or OpenFileMapping() function to obtain a handle to the mapping object.

Each process specifies its handle to the file-mapping object in the MapViewOfFile() function to map a view of the file into its own address space. The views of all processes for a single file-mapping object are mapped into the same sharable pages of physical storage. However, the virtual addresses of the mapped views can vary from one process to another, unless the MapViewOfFileEx() function is used to map the view at a specified address. Although sharable, the pages of physical storage used for a mapped file view are not global; they are not accessible to processes that have not mapped a view of the file.

Any pages committed by mapping a view of a file are released when the last process with a view of the mapping object either terminates or unmaps its view by calling the UnmapViewOfFile() function. At this time, the specified file (if any) associated with the mapping object is updated. A specified file can also be forced to update by calling the FlushViewOfFile() function. If multiple processes have write access to shared memory, you must synchronize access to the memory.

 

 

Memory Protection

 

Memory that belongs to a process is implicitly protected by its private virtual address space. In addition, Windows provides memory protection using the virtual memory hardware. The implementation of this protection varies with the processor. For example, code pages in the address space of a process can be marked read-only and protected from modification by user-mode threads. The following table lists the memory-protection options provided by Windows. You must specify one of the following values when allocating or protecting a page in memory.

 

Value

Meaning

PAGE_EXECUTE

Enables execute access to the committed region of pages. An attempt to read or write to the committed region results in an access violation.

This flag is not supported by CreateFileMapping().

PAGE_EXECUTE_READ

Enables execute and read access to the committed region of pages. An attempt to write to the committed region results in an access violation.

This flag is not supported by CreateFileMapping().

PAGE_EXECUTE_READWRITE

Enables execute, read, and write access to the committed region of pages. This flag is not supported by CreateFileMapping().

PAGE_EXECUTE_WRITECOPY

Enables execute, read, and write access to the committed region of image file code pages. The pages are shared read-on-write and copy-on-write. This flag is not supported by VirtualAlloc(), VirtualAllocEx(), or CreateFileMapping().

PAGE_NOACCESS

Disables all access to the committed region of pages. An attempt to read from, write to, or execute the committed region results in an access violation exception, called a general protection (GP) fault. This flag is not supported by CreateFileMapping().

PAGE_READONLY

Enables read access to the committed region of pages. An attempt to write to the committed region results in an access violation. If the system differentiates between read-only access and execute access, an attempt to execute code in the committed region results in an access violation.

PAGE_READWRITE

Enables both read and write access to the committed region of pages.

PAGE_WRITECOPY

Gives copy-on-write protection to the committed region of pages.

This flag is not supported by VirtualAlloc() or VirtualAllocEx().

Windows Me/98/95:  This flag is not supported.

 

Table 3.

 

The following are modifiers that can be used in addition to the options provided in the previous table, except as noted.

 

Value

Meaning

PAGE_GUARD

Pages in the region become guard pages. Any attempt to access a guard page causes the system to raise a STATUS_GUARD_PAGE_VIOLATION exception and turn off the guard page status. Guard pages thus act as a one-time access alarm.

When an access attempt leads the system to turn off guard page status, the underlying page protection takes over.

If a guard page exception occurs during a system service, the service typically returns a failure status indicator.

This value cannot be used with PAGE_NOACCESS.

This flag is not supported by CreateFileMapping().

Windows Me/98/95:  This flag is not supported. To simulate this behavior, use PAGE_NOACCESS.

PAGE_NOCACHE

Does not allow caching of the committed regions of pages in the CPU cache. The hardware attributes for the physical memory should be specified as "no cache." This is not recommended for general usage. It is useful for device drivers; for example, mapping a video frame buffer with no caching.

This value cannot be used with PAGE_NOACCESS.

This flag is not supported by CreateFileMapping().

PAGE_WRITECOMBINE

Enables write-combined memory accesses. When enabled, the processor caches memory write requests to optimize performance. Thus, if two requests are made to write to the same memory address, only the more recent write may occur.

Note that the PAGE_GUARD and PAGE_NOCACHE flags cannot be specified with PAGE_WRITECOMBINE. If an attempt is made to do so, the SYSTEM_INVALID_PAGE_PROTECTION NT error code is returned by the function. This flag is not supported by CreateFileMapping().

 

Table 4.

 

Copy-on-Write Protection

 

Copy-on-write protection is an optimization that allows multiple processes to map their virtual address spaces such that they share a physical page until one of the processes modifies the page. This is part of a technique called lazy evaluation, which allows the system to conserve physical memory and time by not performing an operation until absolutely necessary. For example, suppose two processes load pages from the same DLL into their virtual memory spaces. These virtual memory pages are mapped to the same physical memory pages for both processes. As long as neither process writes to these pages, they can map to and share, the same physical pages, as shown in the following diagram.

 

Boxes and arrows of process 1 and 2 pages mapped to same physical memory

 

Figure 1.

 

If Process 1 writes to one of these pages, the contents of the physical page are copied to another physical page and the virtual memory map is updated for Process 1. Both processes now have their own instance of the page in physical memory. Therefore, it is not possible for one process to write to a shared physical page and for the other process to see the changes.

 

Boxes and arrows of processes and physical memory remapping

 

Figure 2.

 

Loading Applications and DLLs

 

When multiple instances of the same Windows-based application are loaded, each instance is run in its own protected virtual address space. However, their instance handles (hInstance) typically have the same value. This value represents the base address of the application in its virtual address space. If each instance can be loaded into its default base address, it can map to and share the same physical pages with the other instances, using copy-on-write protection. The system allows these instances to share the same physical pages until one of them modifies a page. If for some reason one of these instances cannot be loaded in the desired base address, it receives its own physical pages.

DLLs are created with a default base address. Every process that uses a DLL will try to load the DLL within its own address space at the default virtual address for the DLL. If multiple applications can load a DLL at its default virtual address, they can share the same physical pages for the DLL. If for some reason a process cannot load the DLL at the default address, it loads the DLL elsewhere. Copy-on-write protection forces some of the DLL's pages to be copied into different physical pages for this process, because the fixups for jump instructions are written within the DLL's pages, and they will be different for this process. If the code section contains many references to the data section, this can cause the entire code section to be copied to new physical pages.

 

Virtual Address Space – 64 bits

 

By default, 64-bit Microsoft® Windows®-based applications have a user-mode address space of 8 terabytes. However, you can specify that the system should allocate all memory for the application below 2 gigabytes. This feature is beneficial for 64-bit applications if the following conditions are true:

All pointers are still 64-bit pointers, but the system insures that every memory allocation occurs below the 2G, so that if the application truncates a pointer, no significant data is lost. Pointers can be truncated to 32-bit values, and then extended to 64-bit values by either sign extension or zero extension. To specify this memory limitation, use the /LARGEADDRESSAWARE:NO linker option. However, be aware that problems can occur when using this option. If you build a DLL that uses this option and the DLL is called by an application that does not use this option, the DLL could truncate a 64-bit pointer whose upper 32 bits are significant. This can cause application failure without any warning.

 

 

---------------End--------------

 

 

 

Further reading and digging:

 

  1. Microsoft Visual C++, online MSDN.
  2. Structure, enum, union and typedef story can be found C & C++ struct, enum, typedef etc..
  3. For Multibytes, Unicode characters and Localization please refer to Unicode & Multibyte 1 (Story) and Unicode & Multibyte 2 (Implementation).
  4. Check the best selling C / C++ and Windows books at Amazon.com.

 

 

 

 

 

 

 

 


| Main | Site Index | Download | Disclaimer | Privacy |