C Extensions
This guide explains how to package a C extension as a rubygem.
To get started, we’ll add a C method to the hola
gem from the make your own gem guide. Then there’s some generaladvice and some references for further reading.
Tutorial: Add a C extension tohola
At the end of the make your own gem guide, hola
looked like this:
% tree
.
├── bin
│ └── hola
├── hola.gemspec
├── lib
│ ├── hola
│ │ └── translator.rb
│ └── hola.rb
├── Rakefile
└── test
└── test_hola.rb
Now we’ll add in a simple C extension.
Create an ext
directory
Just as we put the ruby source files in the lib/hola
directory, we put the C extension source files in ext/hola
. This directory contains two files:
% tree ext
ext
└── hola
├── extconf.rb
└── hola.c
Create extconf.rb
extconf.rb
tells RubyGems how to build the extension. Forhola
, it reads:
% cat ext/hola/extconf.rb
require 'mkmf'
create_makefile('hola/hola')
mkmf
, a ruby standard library, automates the process of creating a Makefile which, in turn, automates the process of building the extension. When RubyGems installs a gem, it runs the gem’s extconf.rb
and then runs make
on the resulting Makefile
. The output from make is a dynamically linked library. The naming conventions for these libraries is platform dependent. You can use this bit of code to find the filename extension used on your platform for dynamically linked libraries:
% irb -rrbconfig
>> RbConfig::CONFIG['DLEXT']"
# extensions are .bundle on my OS
=> bundle
Create the C extension
In general, a good convention for the c file name is ext/<gem_name>/<gem_name>.c
. While it is somewhat annoying to repeat the gem name in so many places, it helps to avoid naming conflicts, which are even more annoying.
The C source code for the extension is in hola.c
, which reads:
% cat ext/hola/hola.c
#include <ruby.h>
/* our new native method; it just returns
* the string "bonjour!" */
static VALUE hola_bonjour(VALUE self) {
return rb_str_new2("bonjour!");
}
/* ruby calls this to load the extension */
void Init_hola(void) {
/* assume we haven't yet defined Hola */
VALUE klass = rb_define_class("Hola",
rb_cObject);
/* the hola_bonjour function can be called
* from ruby as "Hola.bonjour" */
rb_define_singleton_method(klass,
"bonjour", hola_bonjour, 0);
}
ruby.h
provides declarations for ruby’s C API, which is where the rb_...
methods come from. The Makefile
generated by mkmf
ensures that the compiler knows where this header file is. Some references for ruby’s C API are given at the end of this guide.
Again, the name of the file is important, because it matches the second hola
in the hola/hola
that we passed to create_makefile
in extconf.rb
. Similarly, the name of the Init_hola
function is important, because ruby looks for a function with this name when it loads hola/hola.so
. There is a summary of these naming rulesat the end of the guide.
Modify the gemspec
For RubyGems to know that a gem contains a C extension, we have to tell it about extconf.rb
, and we have to include the C source files in the files
list. As you should expect, this happens in the gemspec:
% cat hola.gemspec
Gem::Specification.new do |s|
# ... (other stuff) ...
s.files = Dir.glob('lib/**/*.rb') +
Dir.glob('ext/**/*.{c,h,rb}')
s.extensions = ['ext/hola/extconf.rb']
s.executables = ['hola']
# ... (other stuff) ...
end
This gemspec computes the files
list dynamically; when you run gem build
, RubyGems will include all matching files in the gem. RubyGems automatically adds the extensions
and executables
to the gem, so you don’t have to list them under files
.
Load the extension
The final step is to require the shared object so that ruby will load the extension with the rest of the gem. To do this, simply require hola/hola
in lib/hola.rb
:
% cat lib/hola.rb
require 'hola/hola'
... (rest of file unchanged) ...
This works because RubyGems copies the shared object from ext
to lib
when the gem is installed. It seems a bit like magic, but now you can build and install the gem to see what’s going on:
% gem build hola.gemspec
Successfully built RubyGem
Name: hola
Version: 0.0.1
File: hola-0.0.1.gem
The gem build
command creates a .gem
file from the .gemspec
. The Gemfile includes all of the source files, but it doesn’t compile the extension; that happens when the gem is installed:
% gem install hola-0.0.1.gem
Building native extensions. This could take a while...
Successfully installed hola-0.0.1
1 gem installed
Where exactly the gem is installed to depends on how ruby is set up. On Linux with rvm, the gem is installed into a directory structure resembling this:
% tree ~/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/
/home/john/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/
├── bin
│ └── hola
├── ext
│ └── hola
│ ├── extconf.rb
│ ├── hola.c
│ ├── hola.o
│ ├── hola.so
│ └── Makefile
├── lib
│ ├── hola
│ │ ├── hola.so # <----
│ │ └── translator.rb
│ └── hola.rb
└── test
└── test_hola.rb
You can see that the ext/hola
directory contains the extconf.rb
and hola.c
files. You can also see that the install has generated a Makefile, a hola.o
object file generated by the compiler, and the finished product, hola.so
. And as mentioned before, notice that hola.so
has been copied to lib/hola
.
When you require the gem, RubyGems adds hola
’s lib
directory to $LOAD_PATH
, which is where require
looks for files:
% irb -rubygems
ruby-1.8.7-p352 :001 > require 'hola'
=> true
ruby-1.8.7-p352 :002 > puts $LOAD_PATH
/home/john/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/lib
... (lots of standard paths) ...
.
=> nil
Finally, you can call the C extension’s bonjour
method from ruby code:
ruby-1.8.7-p352 :003 > Hola.bonjour
=> "bonjour!"
The string "bonjour!"
came from our C extension. Hooray!
Of course, you should also add a test to the test suite:
% cat test/test_hola.rb
require 'test/unit'
require 'hola'
class HolaTest < Test::Unit::TestCase
... (other tests) ...
def test_bonjour
assert_equal "bonjour!", Hola.bonjour
end
end
Add Rake
tasks
Building and installing the gem every time you make a change quickly gets tedious. To speed things up, it helps to add some extra tasks to your Rakefile that automate the build process. The following should work on any unix. For cross-platform Rake
tasks see rake-compiler.
require 'rake/testtask'
require 'rake/clean'
NAME = 'hola'
# rule to build the extension: this says
# that the extension should be rebuilt
# after any change to the files in ext
file "lib/#{NAME}/#{NAME}.so" =>
Dir.glob("ext/#{NAME}/*{.rb,.c}") do
Dir.chdir("ext/#{NAME}") do
# this does essentially the same thing
# as what RubyGems does
ruby "extconf.rb"
sh "make"
end
cp "ext/#{NAME}/#{NAME}.so", "lib/#{NAME}"
end
# make the :test task depend on the shared
# object, so it will be built automatically
# before running the tests
task :test => "lib/#{NAME}/#{NAME}.so"
# use 'rake clean' and 'rake clobber' to
# easily delete generated files
CLEAN.include('ext/**/*{.o,.log,.so}')
CLEAN.include('ext/**/Makefile')
CLOBBER.include('lib/**/*.so')
# the same as before
Rake::TestTask.new do |t|
t.libs << 'test'
end
desc "Run tests"
task :default => :test
Now typing rake
will build (or rebuild) the extension before running the tests, as necessary:
% rake
(in /home/john/rubygems_hola)
/home/john/.rvm/rubies/ruby-1.8.7-p352/bin/ruby extconf.rb
creating Makefile
make
# gcc output ...
cp ext/hola/hola.so lib/hola
Loaded suite /home/john/.rvm/gems/ruby-1.8.7-p352/gems/rake-0.8.7/lib/rake/rake_test_loader
Started
....
Finished in 0.00182 seconds.
4 tests, 4 assertions, 0 failures, 0 errors
If the C source and extconf.rb
build script have not changed, then running rake
a second time runs only the test suite.
The code for this tutorial is available on github.
A good alternative to writing your own Rakefile like the one above is to use the rake compiler, which does the same stuff – and much more – out of the box. It also lets you more easily target multiple platforms and interpreters.
Advice
Extension Naming
To avoid unintended interactions between gems, it’s a good idea for each gem to keep all of its files in a single directory. This is the motivation behind the naming conventions used in the tutorial. To summarize, the suggested conventions for a gem with name $g
are:
-
ext/$g
is the directory that contains the source files andextconf.rb
-
ext/$g/$g.c
is the main source file (there may be others) -
ext/$g/$g.c
contains a functionInit_$g
-
ext/$g/extconf.rb
callscreate_makefile('$g/$g')
- the gemspec sets
extensions = ['ext/$g/extconf.rb']
and setsfiles
to list any C source or header files inext/$g
- the first require in
lib/$g.rb
isrequire '$g/$g'
An alternative is to name the extension like $g_ext
instead of $g/$g
. The result is that the $g_ext.so
file is installed into the gem’s lib
directory, and it can be required from lib/$g.rb
as require '$g_ext'
. This also works, though it is perhaps not as clean as the first convention.
Wrapping Existing Libraries
A common reason for writing a C extension is to wrap an existing C or C++ library. This can be done manually (see this tutorial – part 1 and part 2), but several tools also exist:
- SWIG, the Simplified Wrapper Interface Generator, is mature and probably the most popular
- rb++ is nicer in several ways but is less stable
Multi-Implementation Extensions
There are several ruby implementations. C extensions that use the ruby C API can be loaded by the standard ruby interpreter (the MRI – Matz’s Ruby Interpreter) and other C-based interpreters, but they cannot be loaded into JRuby(ruby on the Java Virtual machine) or IronRuby (ruby on the Common Language Runtime (.NET)), for example.
See ruby-ffi for a way to build extensions that work with other Ruby implementations.
References
This guide is based largely on this excellent two-part tutorial:
The main references for ruby’s C API are: