Linking is theprocess of combining various pieces of code and data together toform a single executable that can be loaded in memory. Linking canbe done at compile time, at load time (by loaders) and also at runtime (by application programs). The process of linking dates backto late 1940s, when it was done manually. Now, we havelinkers that support complex features, such asdynamically linked shared libraries. This article is a succinctdiscussion of all aspects of linking, ranging from relocation andsymbol resolution to supporting position-independent sharedlibraries. To keep things simple and understandable, I target allmy discussions to ELF (executable and linking format) executableson the x86 architecture (Linux) and use the GNU compiler (GCC) andlinker (ld). However, the basic concepts of linking remain thesame, regardless of the operating system, processor architecture orobject file format being used.
Consider two program files, a.c and b.c. As we invoke the GCCon a.c b.c at the shell prompt, the following actions takeplace:
gcc a.c b.c
-
Run preprocessor on a.c and store the result inintermediate preprocessed file.
cpp other-command-line options a.c /tmp/a.i -
Run compiler proper on a.i and generate theassembler code in a.s
cc1 other-command-line options /tmp/a.i -o /tmp/a.s -
Run assembler on a.s and generate the object filea.o
as other-command-line options /tmp/a.s -o /tmp/a.o
cpp, cc1 and as are the GNU's preprocessor, compiler properand assembler, respectively. They are a part of the standard GCCdistribution.
Repeat the above steps for file b.c. Now we have anotherobject file, b.o. The linker's job is to take these input objectfiles (a.o and b.o) and generate the final executable:
ld other-command-line-options /tmp/a.o /tmp/b.o -o a.out
The final executable (a.out) then is ready to be loaded. Torun the executable, we type its name at the shell prompt:
./a.out
The shell invokes the loader function, which copies the codeand data in the executable file a.out into memory, and thentransfers control to the beginning of the program. The loader is aprogram called execve, which loads the code and data of theexecutable object file into memory and then runs the program byjumping to the first instruction.
a.out was first coined as the Assembler OUTput in a.outobject files. Since then, object formats have changed variedly, butthe name continues to be used.
Linkers and loaders perform various related but conceptuallydifferent tasks:
-
Program Loading. This refers to copying a programimage from hard disk to the main memory in order to put the programin a ready-to-run state. In some cases, program loading also mightinvolve allocating storage space or mapping virtual addresses todisk pages.
-
Relocation. Compilers and assemblers generate theobject code for each input module with a starting address of zero.Relocation is the process of assigning load addresses to differentparts of the program by merging all sections of the same type intoone section. The code and data section also are adjusted so theypoint to the correct runtime addresses.
-
Symbol Resolution. A program is made up of multiplesubprograms; reference of one subprogram to another is made throughsymbols. A linker's job is to resolve the reference by noting thesymbol's location and patching the caller's object code.
So a considerable overlap exists between the functions oflinkers and loaders. One way to think of them is: the loader doesthe program loading; the linker does the symbol resolution; andeither of them can do the relocation.
Object files comes in three forms:
-
Relocatable object file, which contains binary codeand data in a form that can be combined with other relocatableobject files at compile time to create an executable objectfile.
-
Executable object file, which contains binary codeand data in a form that can be directly loaded into memory andexecuted.
-
Shared object file, which is a special type ofrelocatable object file that can be loaded into memory and linkeddynamically, either at load time or at run time.
Compilers and assemblers generate relocatable object files(also shared object files). Linkers combine these object filestogether to generate executable object files.
Object files vary from system to system. The first UNIXsystem used the a.out format. Early versions of System V used theCOFF (common object file format). Windows NT uses a variant of COFFcalled PE (portable executable) format; IBM uses its own IBM 360format. Modern UNIX systems, such as Linux and Solaris use the UNIXELF (executable and linking format). This article concentratesmainly on ELF.
The above figure shows the format of a typical ELFrelocatable object file. The ELF header starts with a 4-byte magicstring, \177ELF. The various sections in the ELF relocatable objectfile are:
-
.text, the machine code of the compiledprogram.
-
.rodata, read-only data, such as the format stringsin printf statements.
-
.data, initialized global variables.
-
.bss, uninitialized global variables. BSS standsfor block storage start, and this section actually occupies nospace in the object file; it is merely a placer holder.
-
.symtab, a symbol table with information aboutfunctions and global variables defined and referenced in theprogram. This table does not contain any entries for localvariables; those are maintained on the stack.
-
.rel.text, a list of locations in the .text sectionthat need to be modified when the linker combines this object filewith other object files.
-
.rel.data, relocation information for globalvariables referenced but not defined in the current module.
-
.debug, a debugging symbol table with entries forlocal and global variables. This section is present only if thecompiler is invoked with a -g option.
-
.line, a mapping between line numbers in theoriginal C source program and machine code instructions in the.text section. This information is required by debuggerprograms.
-
.strtab, a string table for the symbol tables inthe .symtab and .debug sections.
Symbols and Symbol Resolution
Every relocatable object file has a symbol table andassociated symbols. In the context of a linker, the following kindsof symbols are present:
-
Global symbols defined by the module and referencedby other modules. All non-static functions and global variablesfall in this category.
-
Global symbols referenced by the input module butdefined elsewhere. All functions and variables with externdeclaration fall in this category.
-
Local symbols defined and referenced exclusively bythe input module. All static functions and static variables fallhere.
The linker resolves symbol references by associating eachreference with exactly one symbol definition from the symbol tablesof its input relocatable object files. Resolution of local symbolsto a module is straightforward, as a module cannot have multipledefinitions of local symbols. Resolving references to globalsymbols is trickier, however. At compile time, the compiler exportseach global symbol as either strong or weak. Functions andinitialized global variables get strong weight, while globaluninitialized variables are weak. Now, the linker resolves thesymbols using the following rules:
-
Multiple strong symbols are not allowed.
-
Given a single strong symbol and multiple weaksymbols, choose the strong symbol.
-
Given multiple weak symbols, choose any of the weaksymbols.
For example, linking the following two programs produceslinktime errors:
/* foo.c */ /* bar.c */
int foo () { int foo () {
return 0; return 1;
} }
int main () {
foo ();
}
The linker will generate an error message because foo (strongsymbol as its global function) is defined twice.
gcc foo.c bar.c /tmp/ccM1DKre.o: In function 'foo': /tmp/ccM1DKre.o(.text+0x0): multiple definition of 'foo' /tmp/ccIhvEMn.o(.text+0x0): first defined here collect2: ld returned 1 exit status
Collect2 is a wrapper over linker ld that is called byGCC.
A static library is a collection of concatenated object filesof similar type. These libraries are stored on disk in an archive.An archive also contains some directory information that makes itfaster to search for something. Each ELF archive starts with themagic eight character string !<arch>\n, where \n is anewline.
Static libraries are passed as arguments to compiler tools(linker), which copy only the object modules referenced by theprogram. On UNIX systems, libc.a contains all the C libraryfunctions, including printf and fopen, that are used by most of theprograms.
gcc foo.o bar.o /usr/lib/libc.a /usr/lib/libm.a
libm.a is the standard math library on UNIX systems thatcontains the object modules for math functions such as like sqrt,sin, cos and so on.
During the process of symbol resolution using staticlibraries, linker scans the relocatable object files and archivesfrom left to right as input on the command line. During this scan,linker maintains a set of O, relocatable object files that go intothe executable; a set U, unresolved symbols; and a set of D,symbols defined in previous input modules. Initially, all threesets are empty.
-
For each input argument on the command line, linkerdetermines if input is an object file or an archive. If input is arelocatable object file, linker adds it to set O, updates U and Dand proceeds to next input file.
-
If input is an archive, it scans through the listof member modules that constitute the archive to match anyunresolved symbols present in U. If some archive member defines anyunresolved symbol that archive member is added to the list O, and Uand D are updated per symbols found in the archive member. Thisprocess is iterated for all member object files.
-
After all the input arguments are processed throughthe above two steps, if U is found to be not empty, linker printsan error report and terminates. Otherwise, it merges and relocatesthe object files in O to build the output executable file.
This also explains why static libraries are placed at the endof the linker command. Special care must be taken in cases ofcyclic dependencies between libraries. Input libraries must beordered so each symbol is referenced by a member of an archive andat least one definition of a symbol is followed by a reference toit on the command line. Also, if an unresolved symbol is defined inmore than one static library modules, the definition is picked fromthe first library found in the command line.
Relocation
Once the linker has resolved all the symbols, each symbolreference has exactly one definition. At this point, linker startsthe process of relocation, which involves the following twosteps:
-
Relocating sections and symbol definitions. Linkermerges all the sections of the same type into a new single section.For example, linker merges all the .data sections of all the inputrelocatable object files into a single .data section for the finalexecutable. A similar process is carried out for the .code section.The linker then assigns runtime memory addresses to new aggregatesections, to each section defined by the input module and also toeach symbol. After the completion of this step, every instructionand global variable in the program has a unique loadtimeaddress.
-
Relocating symbol reference within sections. Inthis step, linker modifies every symbol reference in the code anddata sections so they point to the correct loadtimeaddresses.
Whenever assembler encounters an unresolved symbol, itgenerates a relocation entry for that object and places it in the.relo.text/.relo.data sections. A relocation entry containsinformation about how to resolve the reference. A typical ELFrelocation entry contains the following members:
-
Offset, a section offset of the reference thatneeds to be relocated. For a relocatable file, this value is thebyte offset from the beginning of the section to the storage unitaffected by relocation.
-
Symbol, a symbol the modified reference shouldpoint to. It is the symbol table index with respect to which therelocation must be made.
-
Type, the relocation type, normally R_386_PC32,that signifies PC-relative addressing. R_386_32 signifies absoluteaddressing.
The linker iterates over all the relocation entries presentin the relocatable object modules and relocates the unresolvedsymbols depending on the type. For R_386_PC32, the relocatingaddress is calculated as S + A - P; for R_386_32 type, the addressis calculated as S + A. In these calculations, S denotes the valueof the symbol from the relocation entry, P denotes the sectionoffset or address of the storage unit being relocated (computedusing the value of offset from relocation entry) and A is theaddress needed to compute the value of the relocatablefield.
Static libraries above have some significant disadvantages;for example, consider standard functions such as printf and scanf.They are used almost by every application. Now, if a system isrunning 50-100 processes, each process has its own copy ofexecutable code for printf and scanf. This takes up significantspace in the memory. Shared libraries, on the other hand, addressthe disadvantages of static libraries. A shared library is anobject module that can be loaded at run time at an arbitrary memoryaddress, and it can be linked to by a program in memory. Sharedlibraries often are called as shared objects. On most UNIX systemsthey are denoted with a .so suffix; HP-UX uses a .sl suffix andMicrosoft refer to them as DLLs (dynamic link libraries).
To build a shared object, the compiler driver is invoked witha special option:
gcc -shared -fPIC -o libfoo.so a.o b.o
The above command tells the compiler driver to generate ashared library, libfoo.so, comprised of the object modules a.o andb.o. The -fPIC option tells the compiler to generate positionindependent code (PIC).
Now, suppose the main object module is bar.o, which hasdependencies on a.o and b.o. In this case, the linker is invokedwith:
gcc bar.o ./libfoo.so
This command creates an executable file, a.out, in a formthat can be linked to libfoo.so at load time. Here a.out does notcontain the object modules a.o and b.o, which would have beenincluded had we created a static library instead of a sharedlibrary. The executable simply contains some relocation and symboltable information that allow references to code and data inlibfoo.so to be resolved at run time. Thus, a.out here is apartially executable file that still has its dependency inlibfoo.so. The executable also contains a .interp section thatcontains the name of the dynamic linker, which itself is a sharedobject on Linux systems (ld-linux.so). So, when the executable isloaded into memory, the loader passes control to the dynamiclinker. The dynamic linker contains some start-up code that mapsthe shared libraries to the program's address space. It then doesthe following:
-
relocates the text and data of libfoo.so intomemory segment; and
-
relocates any references in a.out to symbolsdefined by libfoo.so.
Finally, the dynamic linker passes control to theapplication. From this point on, location of shared object is fixedin the memory.
Loading Shared Libraries fromApplications
Shared libraries can be loaded from applications even in themiddle of their executions. An application can request a dynamiclinker to load and link shared libraries, even without linkingthose shared libraries to the executable. Linux, Solaris and othersystems provides a series of function calls that can be used todynamically load a shared object. Linux provides system calls, suchas dlopen, dlsym and dlclose, that can be used to load a sharedobject, to look up a symbol in that shared object and to close theshared object, respectively. On Windows, LoadLibrary andGetProcAddress functions replace dlopen and dlsym,respectively.
Here's a list of Linux tools that can be used to exploreobject/executable files.
-
ar: creates static libraries.
-
objdump: this is the most important binary tool; itcan be used to display all the information in an object binaryfile.
-
strings: list all the printable strings in a binaryfile.
-
nm: lists the symbols defined in the symbol tableof an object file.
-
ldd: lists the shared libraries on which the objectbinary is dependent.
-
strip: deletes the symbol table information.
Linkers andLoaders by John Levine
Linkersand Libraries Guide from Sun
Sandeep Grover works forQuickLogic Software(India) Pvt. Ltd.
1727
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
