转自mathworks，matlab Memory Management Guide，解决out of memory

最新推荐文章于 2024-08-09 00:34:26 发布

immortal_06

最新推荐文章于 2024-08-09 00:34:26 发布

阅读量1.7k

点赞数

文章标签： matlab system variables windows documentation command

1106 - Memory Management Guide

This Technical Support Guide is intended to help you understand and prevent 'Out of Memory' errors in MATLAB.

Section 1: Why Do I Get 'Out of Memory' Errors in MATLAB?

Virtual memory refers to the ability of your computer to use your hard drive as if it were the random access memory (RAM) of your computer. This ability is present on all operating systems supported by MATLAB (if you are not familiar with memory management, you may want to read Section 5 of this document). Thus, if you have a lot of hard drive space, it might appear that memory is almost unlimited. So why do people get 'out of memory' errors in MATLAB?

One cause of 'Out of Memory' errors is that your system has in fact run out of heap space to hold all of your variables. This means that there is no unallocated virtual address space on your computer for MATLAB to use, and therefore no new variables can be created.

The second major cause of 'Out of Memory' errors is memory fragmentation. This means that there is memory available, but there is no contiguous piece of memory that is large enough to hold the specified variable. When virtual memory is used and freed during normal operation of MATLAB, memory becomes fragmented. This means that the amount of total free memory may be greater than the amount of contiguous free memory. Since a matrix in MATLAB must be stored in a contiguous block of virtual memory, the largest matrix that can be created at a particular point in time is limited by the amount of contiguous free virtual address space.

32-Bit Architectures

Most computer platforms today are 32-bit architectures, meaning that the length of a pointer or the size of a processor instruction can be at most 32 bits long. This limitation on the sizes of pointers implies that the memory addresses can be a maximum of 32 bits long, which results in a maximum of power(2,32) possible memory addresses. Since modern operating systems are byte addressable, this translates to 4 gigabytes (GB) of addressable memory. Therefore, under 32 bit architectures, the seemingly unlimited virtual memory space is actually limited to approximately 4 GB.

The Windows operating system further reduces the theoretical limit down to 2 GB of virtual memory due to a design decision to reserve the upper 2 GB of address space. On several versions of Windows, this limit can be moved to allow for an extra 1 GB of memory. For more information about this, please refer to Section 2.

Several 32-bit UNIX operating systems also reserve the upper 2 GB of address space, thereby limiting the virtual memory available to 2 GB. If you need to find out exactly how your operating system reserves virtual memory, we suggest you contact the operating system's vendor.

When you launch MATLAB, approximately 0.8 GB of virtual address space is used to load the heap, stack, DLLs, and other operating system services. For Windows, this limits the net available virtual address space to approximately 1.2 GB to store variables and other data needed by MATLAB.

64-Bit Architectures

If you are using a 64-bit operating system in conjunction with a 64-bit version of MATLAB, it is theoretically possible to access approximately power(2,64) bytes of memory. This far exceeds the amount of memory available to any computer on the market today, so the amount of memory that is available to MATLAB will be limited by the amount of memory available on the computer. As this is typically related to the amount of available virtual memory (or swap space), you may want to consult Section 6 of this document. Please note, however, that there is still a limit on the largest single array available in MATLAB. For more information about this topic, please consult Solution 1-1CAT7.

Viewing Memory Usage

If you are on a Windows platform, you can see the memory allocation along with the largest contiguous block of memory available with the following methods.

In MATLAB 7.0 (R14) and later versions, use the command:
```
      feature('memstats')
```
This can also return a value containing the number of bytes in the largest available memory block:
```
      m = feature('memstats')
```
In MATLAB 6.5 (R13):

Download dumpmemmex.dll
(instructions on how to do this can be found in Solution 1-19IT9).
Place it on your MATLAB path.
Type the following at the MATLAB command prompt:
```
      dumpmemmex
```

For information regarding the output of the above commands, see Solution 1-19IT9.

Section 2: How Do I Avoid 'Out Of Memory' Errors in MATLAB?

For information on avoiding 'Out Of Memory' errors, please see Tech Note 1107.

Section 3: How Do I Defragment the MATLAB Workspace Memory?

There are five functions that you can use to free and defragment MATLAB workspace memory. They are listed in the table below. Click on the each link to see the documentation for that command.

  CLEAR  Removes variables from memory  PACK   Writes existing variables off to disk, and then reloads them contiguously  QUIT   Exits MATLAB and returns all allocated memory to the system  SAVE   Selectively stores variables to disk  LOAD   Reloads a data file saved with the SAVE command

You can use the SAVE and LOAD commands in conjunction with the QUIT command to free memory by:

Saving any needed variables with the SAVE command
Quitting MATLAB to free all memory allocated to MATLAB
Starting a new MATLAB session and loading the saved variables back into the clean MATLAB workspace.
For additional information on the five commands mentioned above, type
```
  help FUNCTIONNAME
```
in the MATLAB Command Window where FUNCTIONNAME refers to one of the five commands.

Note: There are some limitations to these commands. MATLAB relies on the C runtime library and operating system calls to allocate and deallocate memory. Depending on the operating system and MATLAB memory manager, these commands may be handled differently. Also, these commands do not affect the Java class objects created by the Java Virtual Machine (JVM) used in recent versions of MATLAB.

Section 4: How Do I Change the MATLAB Memory Manager?

MATLAB 7.0 (R14) and later may be started with several different memory managers. Although in general MATLAB will select the most suitable memory manager for a given architecture and operating system, certain applications may benefit from specifying a memory manager other than the default.

To specify a non default memory manager, You must set an environmental variable (named MATLAB_MEM_MGR), or supply the name of the desired manager to MATLAB as a command line argument (via use of the -memmgr switch). For information about how to start MATLAB with this option, please refer to the documentation for Windows or UNIX, Linux, and Mac operating systems.

The following memory managers are available:
- cache -- This is the default on 64-bit platforms. It was the only non debug memory manager for versions of MATLAB prior to version 7.0 (R14). It provides limited integrity checking.
- compact and fast -- These are useful for large models or MATLAB code that uses many structure or object variables. Compact is the default selection on 32-bit UNIX platforms and Fast is the default on 32 bit Windows. These have little benefit over cache when working with large matricies. Fast is optimized for Windows but is otherwise similar to compact.
- debug -- This performs memory integrity checking. It is useful for debugging memory problems caused by user-created MEX-files.
Section 5: How Does an Operating System Manage Memory?

A computer's memory must accommodate both the operating system's processes and user processes. The following outline describes what happens when a user starts a program such as MATLAB:
- User starts the program.
- The operating system creates an address space for the program to run in, which involves setting up appropriate page tables.
- The program's text and data segments are mapped from the executable file on disk to the virtual address space; the former being mapped as read only, the latter being mapped as read-write.
- The stack and heap are initialized.
- Control is transferred to the program.
- The CPU tries to access the first instruction. Since it is not in memory, a page fault is generated. This causes the operating system to intervene and allocate one or more pages of physical memory and to copy the code from the file, hence the name "Demand Paged Virtual Memory."
- The CPU then returns control to the program, which continues executing until it encounters another page fault. At this point, step 6 is repeated. If there is no free physical memory, the operating system will have to either discard (if read only) or swap out (write data to swap file if read write) pages before they can be reused. Typically a least recently used (LRU) algorithm is used for deciding which pages get tossed.
- This pattern continues for the life of the program.
When a user executes a program, the operating system creates an address space for it to run in. This address space will include the instructions for the program itself, as well as any data it requires. In a system with memory protection, each process is restricted by the operating system to accessing only the memory in its own address space. However, the combined program memory requirements often exceed the system's amount of physical memory installed on a computer. So, modern operating systems use a portion of the hard disk called a swap file to extend the amount of available memory. This technique, called virtual memory, treats physical memory as a cache of the most recently used data. In a virtual memory system, memory is divided into units called pages (A page is typically 4-8 Kb in size). The set of addresses that identify locations in virtual memory is called the virtual address space. Each process is allocated a virtual address space. The virtual address space can range from 0-4 GB on a 32-bit architecture.

A process's address space contains the set of instructions and data that is mapped into virtual memory when a program is executed. Virtual memory addresses are translated into physical memory addresses through the use of a look-up table, called a page table. In addition to mapping the entire process into virtual memory, a subset of pages is also mapped into physical memory. As each instruction in the process executes, it is either found in physical memory or is not found (called a page fault). When a page fault occurs, the page that is needed must be moved from the hard disk into physical memory before the instruction can be executed. To make room for the new page, the operating system may need to decide which page to move out of physical memory. This is called swapping. A page fault is time consuming because retrieving data from the hard disk is much slower than obtaining it directly from the physical memory. Operating systems attempt to minimize the number of page faults by swapping multiple pages at once. This becomes a trade-off between operating system overhead and the time saved by minimizing page faults, which is affected by the size of each process's working set.

The operating system takes care of swapping and translating from virtual address space to physical address space. This means that developers have a flat address space at their disposal. With a virtual memory scheme, the amount of memory available is seemingly limited only by the amount of hard drive space. Virtual memory has more memory available than physical memory because it removes the restriction that an entire process must be loaded into memory at one time.

Section 6: How Do I Set the Swap Space for My Operating System?

How you set the swap space for your computer depends on the operating system that you are running on.
- On UNIX, information about swap space can be procured by typing pstat -s at the UNIX command prompt. For detailed information on changing swap space, ask your system administrator.
- On Linux, swap space can be changed by using the mkswap and swapon commands. For more information on the above commands, type man command_name at the Linux prompt.
- In Windows, you can set the amount of virtual memory available using the steps outlined below.
For Windows XP:
1. Right-click on My Computer icon and select Properties.
2. Select the Advanced tab
3. Select the Settings tab under Performance.
4. Within the Performance Options GUI, select the Advanced tab.
5. Click on the Change button to increase the virtual memory.
6. In the Paging File Size section, select Automatic.
For Windows 2000:
1. Right-click on My Computer icon and select Properties
2. Select the Advanced tab and choose Performance Options.
3. Click the Change button to modify the virtual memory.
For Windows NT, 95, 98, and Me:
1. Right-click on My Computer icon and select Properties
2. Select the Performance tab and click on the Change button to increase the virtual memory.
Because modern 32-bit machines have physical memory sizes that approach or exceed their virtual address space, a single copy of MATLAB frequently does not see an increase in available memory from an increase in swap space.