Linux静态数组大小,[转载]静态数组占用空间超过2GB时

gcc需要添加-mcmodel=medium选项。

relocation truncated to fit:

R_X86_64_PC32 against

symbol在静态数组占用空间超过2GB时出现，

这个时候应该使用medium内存模型：

-mcmodel=medium

而不是默认的small内存模型。

http://www.ualberta.ca/CNS/RESEARCH/LinuxClusters/64-bit.html#cf

64-bit Programming Issues

The major advantage of a 64-bit environment is that it gives

your programs access to memory and files in excess of

2GB, if necessary. Although utilizing the extra

capacity is straightforward, you should become aquainted with the

traps and pitfalls that may impact your source code and the

compiler flags you use. These issues are outlined here in the

context of the AICT Linux Cluster. If you

have a comment or question about this topic, please send a note to

research.support@ualberta.ca.

Contents

In a 64-bit environment, the size of the C long data type is 64 bits. It therefore carries up to

19 significant figures. Consequently, assigning a long to a double, which carries

only 16 significant figures, may result in loss of accuracy.

Incidentally, the long long type is 64 bits

long in both 32- and 64-bit environments.

Also in C, exercise caution with the data type of integer

constants. The default is int. Hence, the

result of an expression like 1<<32 is zero. If

necessary, the default can be overridden with an appropriate

suffix: U for unsigned

int, L for (signed) long, and UL for unsigned long. Accordingly, 1L<<32 yields

4294967296.

Besides the C long type, pointers are

also 64-bit quantities. In most programs, whether C or Fortran,

this fact is largely irrelevant. However, if a C function returns a

pointer and the function is defined without an explicit return

type, such that int is assumed, the return

value will be truncated. This will lead to a memory access error

(or introduce a subtle bug).

#include

#include

#define MEG (1<<20)

foo (void)

{

int *a = malloc (200*MEG*sizeof(int));

*a = 1;

printf ("foo(): address: %p: value: %dn", a, *a);

return (a);

}

int

main (void)

{

int *b = foo();

printf ("main(): address: %pn", b);

printf ("main(): value: %dn", *b);

exit (EXIT_SUCCESS);

}

$ pgcc unprototyped.c

PGC-W-0095-Type cast required for this conversion (unprototyped.c: 14)

PGC-W-0095-Type cast required for this conversion (unprototyped.c: 20)

PGC-W-0155-Pointer value created from a nonlong integral type (unprototyped.c: 20)

PGC/x86-64 Linux/x86-64 6.1-6: compilation completed with warnings

$ ./a.out

foo(): address: 0x2a9557a010: value: 1

main(): address: 0xffffffff9557a010

Segmentation fault

Default sizes for Fortran INTEGER (and

REAL) variables remain unchanged at 32

bits. However, the non-standard Cray (integer) POINTER type—which is distinct from the standard

Fortran 90/95 POINTER attribute—is 64 bits,

but this should not require any intervention.

In both C and Fortran, array bounds can be as large as

264. Consequently, you should be careful to declare

index variables as long and INTEGER*8, if necessary.

The Portland Group, Intel, and GNU compilers

installed on the AICT Linux Cluster are configured to produce

64-bit executables by default. Additional compiler flags such as

the GNU -m64 are unnecessary for this

purpose.

New or existing programs that use less than 2GB of

memory will probably not require any modifications to their source

code or to the way they are compiled. A large-memory program, on

the other hand, in addition to the data size issues mentioned

above, may need additional compiler flags, depending on the

compiler used and how data is implemented.

Typically, large data objects are declared as

either static global arrays or

dynamically allocated memory. These are

associated with the data and heap memory

segments, respectively (see Understanding Memory for a detailed description

of these terms). Large heap (and stack) segments do not need any

special attention. However, if the size of the data segment (sum of

all static global arrays) exceeds 2GB, use the

appropriate flags from the following two tables.

C/C++ compilers

flag

Portland

pgcc

-mcmodel=medium

pgCC

-mcmodel=medium

Intel

icc

–

GNU

gcc

–

g++

–

Table 1 C/C++ compiler flags for accommodating large data

segments.

Fortran compilers

flag

Portland

pgf77

-mcmodel=medium

pg90/pg95

-mcmodel=medium [-i8]¹

Intel

ifort

-mcmodel=medium -i-dynamic² [-i8]¹

GNU

g77

-mcmodel=medium

¹ Makes intrinsic array enquiry functions return

INTEGER*8 values.

² Add $INTEL_LIB_PATH to LD_LIBRARY_PATH.

Table 2 Fortran compiler flags for accommodating large data

segments.

The -mcmodel flag selects the memory

model or memory access modes for a program. Three memory

models are supported by the compilers: small

(text+data<2GB), medium

(text<2GB,

data>2GB), and large

(text>2GB,

data>2GB). Memory segments less than

2GB can be accessed with direct references, while

larger segments require indirect or relative references.

Because its addressing modes are more efficient, the small model

is faster than the medium model. Therefore, unless your program

needs support for large data, it is best to use the small model.

Text segments larger than 2GB are rare, so the large

model can be ignored, and in any case, it is available only with

the Intel compiler.

The default memory model for the Portland compilers is small. On

the other hand, the default for both the Intel and GNU C/C++

compilers is medium, and for the Fortran compilers it is small.

If you build your program by compiling and linking

in separate stages, be sure to include the -mcmodel flag (if used) on the linker command line as

well.

When using -mcmodel=medium with the

Intel Fortran compiler, the additional -i-dynamic flag is required to link the appropriate

libraries. Because these libraries are dynamically linked, your

executable must be made aware of their location. One way to do this

is to amend the environment variable LD_LIBRARY_PATH with the value of INTEL_LIB_PATH as follows (bash syntax shown).

export

LD_LIBRARY_PATH=$INTEL_LIB_PATH:$LD_LIBRARY_PATH

This command will also be necessary in batch job scripts.

Another way is to embed $INTEL_LIB_PATH

directly into the executable file using a compiler (linker) flag.

For example,

ifort -mcmodel=medium -i-dynamic -i8 source.f90

-Wl,-rpath,$INTEL_LIB_PATH

The -i8 flag should be used for Fortran

90/95 code that calls any instrinsic array enquiry functions, such

as SIZE. Without it, the return values of

these functions are truncated to 32 bits. This is true for both

Portland and Intel compilers.