Ruby：C Extensions

C Extensions

This guide explains how to package a C extension as a rubygem.

To get started, we’ll add a C method to the hola gem from the make your own gem guide. Then there’s some generaladvice and some references for further reading.

Tutorial: Add a C extension tohola

At the end of the make your own gem guide, hola looked like this:

% tree
.
├── bin
│   └── hola
├── hola.gemspec
├── lib
│   ├── hola
│   │   └── translator.rb
│   └── hola.rb
├── Rakefile
└── test
    └── test_hola.rb

Now we’ll add in a simple C extension.

Create an ext directory

Just as we put the ruby source files in the lib/holadirectory, we put the C extension source files in ext/hola. This directory contains two files:

% tree ext
ext
└── hola
    ├── extconf.rb
    └── hola.c

Create extconf.rb

extconf.rb tells RubyGems how to build the extension. Forhola, it reads:

% cat ext/hola/extconf.rb
require 'mkmf'

create_makefile('hola/hola')

mkmf, a ruby standard library, automates the process of creating a Makefile which, in turn, automates the process of building the extension. When RubyGems installs a gem, it runs the gem’s extconf.rb and then runs make on the resulting Makefile. The output from make is a dynamically linked library. The naming conventions for these libraries is platform dependent. You can use this bit of code to find the filename extension used on your platform for dynamically linked libraries:

% irb -rrbconfig
>> RbConfig::CONFIG['DLEXT']"
# extensions are .bundle on my OS
=> bundle

Create the C extension

In general, a good convention for the c file name is ext/<gem_name>/<gem_name>.c. While it is somewhat annoying to repeat the gem name in so many places, it helps to avoid naming conflicts, which are even more annoying.

The C source code for the extension is in hola.c, which reads:

% cat ext/hola/hola.c
#include <ruby.h>

/* our new native method; it just returns
 * the string "bonjour!" */
static VALUE hola_bonjour(VALUE self) {
  return rb_str_new2("bonjour!");
}

/* ruby calls this to load the extension */
void Init_hola(void) {
  /* assume we haven't yet defined Hola */
  VALUE klass = rb_define_class("Hola",
      rb_cObject);

  /* the hola_bonjour function can be called
   * from ruby as "Hola.bonjour" */
  rb_define_singleton_method(klass,
      "bonjour", hola_bonjour, 0);
}

ruby.h provides declarations for ruby’s C API, which is where the rb_... methods come from. The Makefilegenerated by mkmf ensures that the compiler knows where this header file is. Some references for ruby’s C API are given at the end of this guide.

Again, the name of the file is important, because it matches the second hola in the hola/hola that we passed to create_makefile in extconf.rb. Similarly, the name of the Init_hola function is important, because ruby looks for a function with this name when it loads hola/hola.so. There is a summary of these naming rulesat the end of the guide.

Modify the gemspec

For RubyGems to know that a gem contains a C extension, we have to tell it about extconf.rb, and we have to include the C source files in the files list. As you should expect, this happens in the gemspec:

% cat hola.gemspec
Gem::Specification.new do |s|
  # ... (other stuff) ...

  s.files = Dir.glob('lib/**/*.rb') +
            Dir.glob('ext/**/*.{c,h,rb}')
  s.extensions = ['ext/hola/extconf.rb']
  s.executables = ['hola']

  # ... (other stuff) ...
end

This gemspec computes the files list dynamically; when you run gem build, RubyGems will include all matching files in the gem. RubyGems automatically adds the extensionsand executables to the gem, so you don’t have to list them under files.

Load the extension

The final step is to require the shared object so that ruby will load the extension with the rest of the gem. To do this, simply require hola/hola in lib/hola.rb:

% cat lib/hola.rb
require 'hola/hola'

... (rest of file unchanged) ...

This works because RubyGems copies the shared object from ext to lib when the gem is installed. It seems a bit like magic, but now you can build and install the gem to see what’s going on:

% gem build hola.gemspec
Successfully built RubyGem
Name: hola
Version: 0.0.1
File: hola-0.0.1.gem

The gem build command creates a .gem file from the .gemspec. The Gemfile includes all of the source files, but it doesn’t compile the extension; that happens when the gem is installed:

% gem install hola-0.0.1.gem
Building native extensions.  This could take a while...
Successfully installed hola-0.0.1
1 gem installed

Where exactly the gem is installed to depends on how ruby is set up. On Linux with rvm, the gem is installed into a directory structure resembling this:

% tree ~/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/
/home/john/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/
├── bin
│   └── hola
├── ext
│   └── hola
│       ├── extconf.rb
│       ├── hola.c
│       ├── hola.o
│       ├── hola.so
│       └── Makefile
├── lib
│   ├── hola
│   │   ├── hola.so        # <----
│   │   └── translator.rb
│   └── hola.rb
└── test
    └── test_hola.rb

You can see that the ext/hola directory contains the extconf.rb and hola.c files. You can also see that the install has generated a Makefile, a hola.o object file generated by the compiler, and the finished product, hola.so. And as mentioned before, notice that hola.sohas been copied to lib/hola.

When you require the gem, RubyGems adds hola’s libdirectory to $LOAD_PATH, which is where require looks for files:

% irb -rubygems
ruby-1.8.7-p352 :001 > require 'hola'
 => true
ruby-1.8.7-p352 :002 > puts $LOAD_PATH
/home/john/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/lib
... (lots of standard paths) ...
.
=> nil

Finally, you can call the C extension’s bonjour method from ruby code:

ruby-1.8.7-p352 :003 > Hola.bonjour
=> "bonjour!"

The string "bonjour!" came from our C extension. Hooray!

Of course, you should also add a test to the test suite:

% cat test/test_hola.rb
require 'test/unit'
require 'hola'

class HolaTest < Test::Unit::TestCase

  ... (other tests) ...

  def test_bonjour
    assert_equal "bonjour!", Hola.bonjour
  end
end

Add Rake tasks

Building and installing the gem every time you make a change quickly gets tedious. To speed things up, it helps to add some extra tasks to your Rakefile that automate the build process. The following should work on any unix. For cross-platform Rake tasks see rake-compiler.

require 'rake/testtask'
require 'rake/clean'

NAME = 'hola'

# rule to build the extension: this says
# that the extension should be rebuilt
# after any change to the files in ext
file "lib/#{NAME}/#{NAME}.so" =>
    Dir.glob("ext/#{NAME}/*{.rb,.c}") do
  Dir.chdir("ext/#{NAME}") do
    # this does essentially the same thing
    # as what RubyGems does
    ruby "extconf.rb"
    sh "make"
  end
  cp "ext/#{NAME}/#{NAME}.so", "lib/#{NAME}"
end

# make the :test task depend on the shared
# object, so it will be built automatically
# before running the tests
task :test => "lib/#{NAME}/#{NAME}.so"

# use 'rake clean' and 'rake clobber' to
# easily delete generated files
CLEAN.include('ext/**/*{.o,.log,.so}')
CLEAN.include('ext/**/Makefile')
CLOBBER.include('lib/**/*.so')

# the same as before
Rake::TestTask.new do |t|
  t.libs << 'test'
end

desc "Run tests"
task :default => :test

Now typing rake will build (or rebuild) the extension before running the tests, as necessary:

% rake
(in /home/john/rubygems_hola)
/home/john/.rvm/rubies/ruby-1.8.7-p352/bin/ruby extconf.rb
creating Makefile
make
# gcc output ...
cp ext/hola/hola.so lib/hola
Loaded suite /home/john/.rvm/gems/ruby-1.8.7-p352/gems/rake-0.8.7/lib/rake/rake_test_loader
Started
....
Finished in 0.00182 seconds.

4 tests, 4 assertions, 0 failures, 0 errors

If the C source and extconf.rb build script have not changed, then running rake a second time runs only the test suite.

The code for this tutorial is available on github.

A good alternative to writing your own Rakefile like the one above is to use the rake compiler, which does the same stuff – and much more – out of the box. It also lets you more easily target multiple platforms and interpreters.

Advice

Extension Naming

To avoid unintended interactions between gems, it’s a good idea for each gem to keep all of its files in a single directory. This is the motivation behind the naming conventions used in the tutorial. To summarize, the suggested conventions for a gem with name $g are:

1. ext/$g is the directory that contains the source files and extconf.rb
2. ext/$g/$g.c is the main source file (there may be others)
3. ext/$g/$g.c contains a function Init_$g
4. ext/$g/extconf.rb calls create_makefile('$g/$g')
5. the gemspec sets extensions = ['ext/$g/extconf.rb'] and sets files to list any C source or header files in ext/$g
6. the first require in lib/$g.rb is require '$g/$g'

An alternative is to name the extension like $g_extinstead of $g/$g. The result is that the $g_ext.so file is installed into the gem’s lib directory, and it can be required from lib/$g.rb as require '$g_ext'. This also works, though it is perhaps not as clean as the first convention.

Wrapping Existing Libraries

A common reason for writing a C extension is to wrap an existing C or C++ library. This can be done manually (see this tutorial – part 1 and part 2), but several tools also exist:

• SWIG, the Simplified Wrapper Interface Generator, is mature and probably the most popular
• rb++ is nicer in several ways but is less stable

Multi-Implementation Extensions

There are several ruby implementations. C extensions that use the ruby C API can be loaded by the standard ruby interpreter (the MRI – Matz’s Ruby Interpreter) and other C-based interpreters, but they cannot be loaded into JRuby(ruby on the Java Virtual machine) or IronRuby (ruby on the Common Language Runtime (.NET)), for example.

See ruby-ffi for a way to build extensions that work with other Ruby implementations.

References

This guide is based largely on this excellent two-part tutorial:

• part 1
• part 2

The main references for ruby’s C API are:

• a chapter in the Pickaxe book
• the README.EXT file