(OS X) Loading Code at Runtime

To load dynamic libraries at runtime, apps should use a set of efficient and portable functions, calleddynamic loader compatibility functions. Using these functions ensures that dynamic libraries are loaded in the most efficient way and facilitates the porting of apps from one platform to another.

Two important factors that determine the performance of apps are their launch times and their memory footprints. Reducing the size of an app’s executable file and minimizing its use of memory once it’s launched make the app launch faster and use less memory once it’s launched. Using dynamic libraries instead of static libraries reduces the executable file size of an app. They also allow apps to delay loading libraries with special functionality only when they’re needed instead of at launch time. This feature contributes further to reduced launch times and efficient memory use.

What Are Dynamic Libraries?

Most of an app’s functionality is implemented in libraries of executable code. When an app is linked with a library using a static linker, the code that the app uses is copied to the generated executable file. A static linker collects compiled source code, known as object code, and library code into one executable file that is loaded into memory in its entirety at runtime. The kind of library that becomes part of an app’s executable file is known as a static library. Static libraries are collections or archives of object files.

Figure 1  App using static libraries

Figure 2   App using dynamic libraries

Using dynamic libraries, programs can benefit from improvements to the libraries they use automatically because their link to the libraries is dynamic, not static. That is, the functionality of the client apps can be improved and extended without requiring app developers to recompile the apps. Apps written for OS X benefit from this feature because all system libraries in OS X are dynamic libraries. This is how apps that use Carbon or Cocoa technologies benefit from improvements to OS X.

Dynamic libraries help to distribute an app’s functionality into distinct modules that can be loaded as they are needed. Dynamic libraries can be loaded either when the app launches or as it runs. Libraries that are loaded at launch time are called dependent libraries. Libraries that are loaded at runtime are called dynamically loaded libraries.

This article uses the word image to refer to an app file or a dynamic library. App binaries contain the app’s code and the code from the static libraries the app uses. The dynamic libraries the app loads at launch time or runtime are separate images.

Opening Dynamic Libraries

dlopen

void* dlopen(
   const char* path,
   int mode);

The Library Search Process

The first parameter to dlopen(3) OS X Developer Tools Manual Page is the name of the dynamic library to open. This may be a filename or a partially or fully qualified pathname. For example,libCelsus.dylib , lib/libCelsus.dylib , or /usr/local/libCelsus.dylib.

The dynamic loader searches for libraries in the directories specified by a set of environment variables and the process’s current working directory. These variables, when defined, must contain a colon-separated list of pathnames (absolute or relative) in which the dynamic loader searches for libraries.Table 1 lists the variables.

Table 1  Environment variables that define dynamic-loader search paths

Environment variable	Default value
LD_LIBRARY_PATH	No default value
DYLD_LIBRARY_PATH	No default value
DYLD_FALLBACK_LIBRARY_PATH	$HOME/lib;/usr/local/lib;/usr/lib

When the library name is a filename (that is, when it doesn’t include directory names), the dynamic loader searches for the library in several locations until it finds it, in the following order:

1. $LD_LIBRARY_PATH

2. $DYLD_LIBRARY_PATH

3. The process’s working directory

4. $DYLD_FALLBACK_LIBRARY_PATH

When the library name contains at least one directory name, that is, when the name is a pathname (relative or fully qualified), the dynamic loader searches for the library in the following order:

1. $DYLD_LIBRARY_PATH using the filename

2. The given pathname

3. $DYLD_FALLBACK_LIBRARY_PATH using the filename

For example, say you set the environment variables introduced earlier as shown in the following table.

Environment variable	Value
LD_LIBRARY_PATH	./lib
DYLD_LIBRARY_PATH	/usr/local/dylibs
DYLD_FALLBACK_LIBRARY_PATH	/usr/local/lib

Assuming your app calls dlopen with the filename libCelsus.dylib, the dynamic loader would attempt to open the library using the following pathnames, in order:

Pathname	Description
./lib/libCelsus.dylib	LD_LIBRARY_PATH environment variable
/usr/local/dylibs/libCelsus.dylib	DYLD_LIBRARY_PATH environment variable
libCelsus.dylib	Current working directory
/usr/local/lib/libCelsus.dylib	DYLD_FALLBACK_LIBRARY_PATH environment variable

If the app calls dlopen with the pathname /libs/libCelsus.dylib, the dynamic loader tries to find the library using these pathnames, in order:

Pathname	Description
/usr/local/dylibs/libCelsus.dylib	DYLD_LIBRARY_PATH environment variable
/libs/libCelsus.dylib	Path as given
/usr/local/lib/libCelsus.dylib	DYLD_FALLBACK_LIBRARY_PATH environment variable

Specifying the Scope and Binding Behavior of Exported Symbols

The second parameter of the dlopen(3) OS X Developer Tools Manual Page function specifies two properties: the scope of the library's exported symbols in the current process and when to bind the app’s references the those symbols.

Symbol scope directly affects the performance of apps. Therefore, it’s important that you set the appropriate scope for a library your app opens at runtime.

A dynamically loaded library’s exported symbols can be in one of two levels of scope in the current process: global and local. The main difference between the scopes is that the symbols in the global scope are available to all images in the process, including other dynamically loaded libraries. Symbols in the local scope can be used only by the image that opened the library. See “Using Symbols” for more information.

When the dynamic loader searches for symbols, it performs string comparisons with every symbol in the search scope. Reducing the number of symbols the dynamic loader has to go through to find the desired symbol improves your app’s performance. Opening all dynamically loaded libraries into the local scope instead of the global scope maximizes symbol search performance.

You should never need to open a dynamic library into the process’s global scope so that all modules in the app have access to its symbols. Instead, each module that uses the library should open it into its local scope. When done, the module should close the library. If you want the symbols exported by the library to be available to all images in the process, consider making the library a dependent library of the app.

Important:  All runtime loaded dynamic libraries should be opened into the local scope. Adhering to this rule makes finding symbols at runtime as fast as possible.

Specifying the Scope and Binding Behavior of Exported Symbols

The second parameter of the dlopen(3) OS X Developer Tools Manual Page function specifies two properties: the scope of the library's exported symbols in the current process and when to bind the app’s references the those symbols.

Symbol scope directly affects the performance of apps. Therefore, it’s important that you set the appropriate scope for a library your app opens at runtime.

A dynamically loaded library’s exported symbols can be in one of two levels of scope in the current process: global and local. The main difference between the scopes is that the symbols in the global scope are available to all images in the process, including other dynamically loaded libraries. Symbols in the local scope can be used only by the image that opened the library. See “Using Symbols” for more information.

When the dynamic loader searches for symbols, it performs string comparisons with every symbol in the search scope. Reducing the number of symbols the dynamic loader has to go through to find the desired symbol improves your app’s performance. Opening all dynamically loaded libraries into the local scope instead of the global scope maximizes symbol search performance.

You should never need to open a dynamic library into the process’s global scope so that all modules in the app have access to its symbols. Instead, each module that uses the library should open it into its local scope. When done, the module should close the library. If you want the symbols exported by the library to be available to all images in the process, consider making the library a dependent library of the app.

Using Symbols

void* dlsym(
   void* handle,
   const char* symbol);

After opening a dynamic library using dlopen(3) OS X Developer Tools Manual Page, an image uses the dlsym(3) OS X Developer Tools Manual Page function to get the address of the desired symbol before using it. This function takes two parameters. The first one specifies in which libraries the dynamic loader looks for the symbol. The second parameter specifies the name of the symbol. For example:

symbol_pointer = dlsym(library_handle, "my_symbol")

This invocation tells the dynamic loader to search for a symbol named my_symbol among the symbols exported by the dynamically loaded library represented by the library_handle variable.

There are three scopes the dynamic loader can search for a symbol: a specific dynamic library, the current image's dependent libraries, and the global scope of the process:

• The local scope: To search the symbols exported by a particular dynamic library that has been loaded using dlopen, you providedlsym with the handle to that library. This is the most efficient usage model.

• The next scope: This search scope is useful only when a module has interposed a symbol exported by a dependent library.For example, you may need to intercept all calls to a system function to perform bookkeeping before calling the real implementation. In that case, in your custom definition of the function, you get the address of the function you interposed by invoking dlsym with the RTLD_NEXT special handle instead of the handle to a particular library. Such a call returns the address of the function that would have been executed if you hadn’t masked out that implementation with your own. Therefore, only the dependent libraries of the current image are searched; any other libraries, including libraries opened by the image making the dlsym call, are not searched. Also, in a flat namespace, the search starts in the first dependent library listed after the current one when the app was linked.

• The global scope: To search the global scope, you call dlsymwith the RTLD_DEFAULT special handle. The dynamic loader searches the dependent libraries (loaded at launch time) and the dynamically loaded libraries (loaded at runtime withRTLD_GLOBAL ) for the first match of the symbol name given todlsym . You should avoid performing global symbol searches because they are the most inefficient.

Examples:

Figure 1  App with dependent library hierarchy

The app image can access the exported symbols in libArt.dylib and libBus.dylib directly, as shown in Listing 2.

Listing 2  App image using symbols exported by dependent libraries through undefined external references

#include <stdio.h>

extern char* A_name(); // libArt.dylib

extern char* dependencies(); // libBus.dylib

int main(void) {

printf("[%s] libArt.A_name() = %s\n", __FILE__, A_name());

printf("[%s] libBus.dependencies() = %s\n", __FILE__, dependencies());

}

The app image, however, cannot directly access the symbols exported by libBus1.dylib,libBus1a.dylib, and libBus2.dylib because those libraries are not dependent libraries of the app image. To gain access to those symbols, the app image has to open the corresponding libraries usingdlopen, as shown in Listing 3.

Listing 3  App image using a symbol exported by a dynamic library loaded at runtime

#include <stdio.h>

#include <dlfcn.h>

int main(void) {

void* Bus1a_handle = dlopen("libBus1a.dylib", RTLD_LOCAL);

if (Bus1a_handle) {

char* (*b1a_name)() = dlsym(Bus1a_handle, "B1a_name");

if (b1a_name) {

printf("[%s] libBus1a.B1a_name() = %s\n",

__FILE__, b1a_name());

}

}

else {

printf("[%s] Unable to open libBus1a.dylib: %s\n",

__FILE__, dlerror());

}

dlclose(Bus1a_handle);

}

Using C++ Classes

How client developers use a C++ class depends on whether the dynamic library that implements it is loaded when the client is loaded (dependent library) or at a later point (runtime loaded library). That is, clients can create and delete objects with thenew and delete operators. Classes implemented in libraries loaded at runtime with dlopen(3) OS X Developer Tools Manual Page are called runtime loaded classes.

A runtime loaded class must be instantiated by the client using that class’s factory functions, declared as part of the class’s interface. Factory functions create and destroy instances of a specific class: Constructor functions instantiate objects and destructor functions destroy them. Clients must use factory functions instead of new and delete because the dynamic loader doesn’t have access to a runtime loaded class’s constructors and destructors. When the client calls a factory function, the library invokes the appropriate constructor and destructor on the client’s behalf. After you create an instance of a runtime loaded class, you invoke its member functions the same way you would call them if the class were defined locally.

The interface for C++ classes implemented in dynamic libraries is made up of at least the class declaration and a set of factory functions. The class interface includes one type definition per constructor function. To use a factory function, you must create an object of the appropriate type and get the address of the function with dlsym(3) OS X Developer Tools Manual Page. You can then call the factory function to create or destroy an object of the class.

Listing 8 shows the interface to the Person class, implemented in the Person library.

Listing 8  C++ class interface

/* File: Person.h */

class Person {

private:

char _person_name[30];

public:

Person();

Person(char* name);

virtual void set_name(char person_name[]);

virtual char* name();

};

// Constructor functions and function types.

extern "C" Person* NewPerson(void);

typedef Person * Person_creator(void);

extern "C" Person* NewPersonWithName(char name[]);

typedef Person * PersonWithName_creator(char name[]);

// Destructor function and function type.

extern "C" void DeletePerson(Person* person);

typedef void Person_disposer(Person*);

Listing 9 shows a possible implementation of the Person class.

Listing 9  Implementation of the Person class in the Person library

/* File: Person.cpp */

#include <iostream>

#include "Person.h"

#define EXPORT __attribute__((visibility("default")))

EXPORT

Person::Person() {

char default_name[] = "<no value>";

this->set_name(default_name);

}

EXPORT

Person::Person(char *name) {

this->set_name(name);

}

EXPORT

Person* NewPerson(void) {

return new Person;

}

EXPORT

Person* NewPersonWithName(char name[]) {

return new Person(name);

}

EXPORT

void DeletePerson(Person* person) {

delete person;

}

void Person::set_name(char name[]) {

strcpy(_person_name, name);

}

char* Person::name(void) {

return _person_name;

}

Note that the Person class has two constructor functions, NewPerson and NewPersonWithName. Each function declaration has a corresponding type, Person_creator and PersonWithName_creator.Listing 10 and Listing 11 show how a client may use the Person library.

Listing 10  Client using a C++dependent library

/* File: Client.cpp */

#include <iostream>

#include "Person.h"

int main() {

using std::cout;

using std::cerr;

// Create Person objects.

Person* person1 = new Person();

char person_name[] = "Cendrine";

Person* person2 = new Person(person_name);

cout << "[" << __FILE__ << "] person1->name() = " << person1->name() << "\n";

cout << "[" << __FILE__ << "] person2->name() = " << person2->name() << "\n";

// Use Person objects.

char person1_name[] = "Floriane";

person1->set_name(person1_name);

cout << "[" << __FILE__ << "] person1->name() = " << person1->name() << "\n";

char person2_name[] = "Marcelle";

person2->set_name(person2_name);

cout << "[" << __FILE__ << "] person2->name() = " << person2->name() << "\n";

// Destroy Person objects.

delete person1;

delete person2;

return 0;

}

Listing 11  Client using a C++ dynamically loaded library

/* File: Client.cpp */

#include <iostream>

#include <dlfcn.h>

#include "Person.h"

int main() {

using std::cout;

using std::cerr;

// Open the library.

void* lib_handle = dlopen("./libPerson.dylib", RTLD_LOCAL);

if (!lib_handle) {

exit(EXIT_FAILURE);

}

// Get the NewPerson function.

Person_creator* NewPerson = (Person_creator*)dlsym(lib_handle, "NewPerson");

if (!NewPerson) {

exit(EXIT_FAILURE);

}

// Get the NewPersonWithName function.

PersonWithName_creator* NewPersonWithName = (PersonWithName_creator*)dlsym(lib_handle, "NewPersonWithName");

if (!NewPersonWithName) {

exit(EXIT_FAILURE);

}

// Get the DeletePerson function.

Person_disposer* DeletePerson =

(Person_disposer*)dlsym(lib_handle, "DeletePerson");

if (!DeletePerson) {

exit(EXIT_FAILURE);

}

// Create Person objects.

Person* person1 = NewPerson();

char person_name[] = "Cendrine";

Person* person2 = NewPersonWithName(person_name);

cout << "[" << __FILE__ << "] person1->name() = " << person1->name() << "\n";

cout << "[" << __FILE__ << "] person2->name() = " << person2->name() << "\n";

// Use Person objects.

char person1_name[] = "Floriane";

person1->set_name(person1_name);

cout << "[" << __FILE__ << "] person1->name() = " << person1->name() << "\n";

char person2_name[] = "Marcelle";

person2->set_name(person2_name);

cout << "[" << __FILE__ << "] person2->name() = " << person2->name() << "\n";

// Destroy Person objects.

DeletePerson(person1);

DeletePerson(person2);

// Close the library.

if (dlclose(lib_handle) != 0) {

exit(EXIT_FAILURE);

}

return 0;

}

Using Objective-C Classes

Listing 12  Interface to the Person class and its Titling category

/* File: Person.h */

#import <Foundation/Foundation.h>

@protocol Person

- (void)setName:(NSString*)name;

- (NSString*)name;

@end

@interface Person : NSObject <Person> {

@private

NSString* _person_name;

}

@end

/* File: Titling.h */

#import <Foundation/Foundation.h>

#import "Person.h"

@protocol Titling

- (void)setTitle:(NSString*)title;

@end

@interface Person (Titling) <Titling>

@end

A client compiled with these interfaces and linked with the Person library can create objects that implement the interfaces in a very straightforward way, as shown in Listing 13.

Listing 13  Example of a client that uses the Person library as a dependent library

/* File: Client.m */

#import <Foundation/Foundation.h>

#import "Person.h"

#import "Titling.h"

int main() {

@autoreleasepool {

// Create an instance of Person.

Person<Titling>* person = [[Person alloc] init];

// Use person.

[person setName:@"Perrine LeVan"];

[person setTitle:@"Ms."];

NSLog(@"[%s] main: [person name] = %@", __FILE__, [person name]);

}

return(EXIT_SUCCESS);

}

When the Person library is a runtime loaded library, however, the client must obtain a reference to the Person class from the Objective-C runtime after loading the library, using objc_getClass. It can then use that reference to instantiate a Person object.However, the variable that holds the instance must by typed as an NSObject that implements the Person and Titling protocols to avoid compiler warnings. When done, the client closes the library, as shown in “Using Weakly Linked Symbols.”

Listing 14  Example of a client that uses the Person libraryas a runtime loaded library

/* File: Client.m */

#import <Foundation/Foundation.h>

#import <objc/runtime.h>

#import <dlfcn.h>

#import "Person.h"

#import "Titling.h"

int main() {

@autoreleasepool {

// Open the library.

void* lib_handle = dlopen("./libPerson.dylib", RTLD_LOCAL);

if (!lib_handle) {

NSLog(@"[%s] main: Unable to open library: %s\n",

__FILE__, dlerror());

exit(EXIT_FAILURE);

}

// Get the Person class (required with runtime-loaded libraries).

Class Person_class = objc_getClass("Person");

if (!Person_class) {

NSLog(@"[%s] main: Unable to get Person class", __FILE__);

exit(EXIT_FAILURE);

}

// Create an instance of Person.

NSLog(@"[%s] main: Instantiating Person_class", __FILE__);

NSObject<Person,Titling>* person = [[Person_class alloc] init];

// Use person.

[person setName:@"Perrine LeVan"];

[person setTitle:@"Ms."];

NSLog(@"[%s] main: [person name] = %@", __FILE__, [person name]);

// Close the library.

if (dlclose(lib_handle) != 0) {

NSLog(@"[%s] Unable to close library: %s\n",

__FILE__, dlerror());

exit(EXIT_FAILURE);

}

}

return(EXIT_SUCCESS);

}

Getting Information About the Symbol at a Particular Address

int dladdr(
   const void* addr,
   Dl_info* info);

One of the dynamic loader compatibility (DLC) functions, dladdr(3) OS X Developer Tools Manual Page, provides information on the image and nearest symbol that corresponds to an address. You can use this function to obtain information about the library that exports a particular symbol.

The information dladdr provides is returned through an output parameter of type Dl_info. These are the names of the structure’s fields as well as their descriptions:

• dli_fname: The pathname of the image

• dli_fbase: The base address of the image within the process

• dli_sname: The name of the symbol with an address that is equal to or lower than the address provided to dladdr

• dli_saddr: The address of the symbol indicated by dli_sname

Listing 15 shows how an image can get information about a symbol:

Listing 15  Getting information about a symbol

#include <stdio.h>

#include <dlfcn.h>

extern char* dependencies();

int main(void) {

// Get information on dependencies().

Dl_info info;

if (dladdr(dependencies, &info)) {

printf("[%s] Info on dependencies():\n", __FILE__);

printf("[%s] Pathname: %s\n", __FILE__, info.dli_fname);

printf("[%s] Base address: %p\n", __FILE__, info.dli_fbase);

printf("[%s] Nearest symbol: %s\n", __FILE__, info.dli_sname);

printf("[%s] Symbol address: %p\n", __FILE__, info.dli_saddr);

}

else {

printf("[%s] Unable to find image containing the address %x\n",

__FILE__, &dependencies);

}

}