HOW TO SETUP BUMBLEBEE STEREO CAMERA UNDER LINUX

HOW TO SETUP BUMBLEBEE STEREO CAMERA UNDER LINUX

This article introduces the general steps to make BumbleBee Stereo Camera (Figure 1) work under linux.

Preconditions

The BumbleBee Stereo camera has excellent reputation for its performance on the stereo vision. However, it is very "picky" on its running platform: it only works under 32-bit Linux system due to the 32-bit static library provided byPoint Grey Research, Inc[3].

The preconditions are listed below:

Items	Notes
32-bit Linux System	Here we use Debian-2.6.26-2-686
PC or Laptop with ieee1394 interface	I use Firewire (1394) PCI card
Kernel Module Installed	ieee1394, ohci1394, raw1394
Library Installed	libdc1394 (>=2.0.2), libraw1394 (>=1.2.0)

Technical Notes:

• <1> Configure the 1394 device kernel module for Redhat/Fedora Linux

• Article [2] introduces how to setup the kernel modules of 1394 devices with Redhat 9.0 and Fedora Core 1 & 3
• For Redhat EL5 or Fedora 10 or aboved versions linux, the 1394-kmdl-<version>-<arch>.rpm is available inATrpms
• For other Redhat or Fedora versions of linux, if there is no 1394 device module installed in the kernel, you have to build a kernel with 1394 device support. The [5] lists detailed instruction

• <2> Configure the 1394 device kernel module for Debian/Ubuntu Linux

• To automatically load the kernel modules at booting time, we can add following lines into file "/etc/rc.local"

                # create the 1394 devices
                mknod /dev/raw1394 c 171 0
                chmod a+rw /dev/raw1394
                mkdir /dev/video1394
                mknod /dev/video1394/0 c 171 16
                mknod /dev/video1394/1 c 171 17
                mknod /dev/video1394/2 c 171 18
                mknod /dev/video1394/3 c 171 19
                chmod a+rw /dev/video1394/0
                chmod a+rw /dev/video1394/1
                chmod a+rw /dev/video1394/2
                chmod a+rw /dev/video1394/3
                # install the modules
                modprobe raw1394
                modprobe video1394

• Also you can choose to manually load the kernel modules into the kernel by run commands in the terminal (root permission is needed)

                   modprobe ieee1394
                   modprobe ohci1394
                   modprobe raw1394
                   modprobe video1394

• <3> For libdc1394: you should have a directory called dc1394 in which some headers are contained in path of/usr/include or else.

Install Library

Point Grey Research, Inc. provides the Triclops SDK, which is an area based correlation with SAD (Sum of Absolute Differences), to perform the image rectification and stereo processing.

The algorithm is fairly robust and it has a number of validation steps that reduce the level of noise. The method requires texture and contrast to work correctly. For more detail, please refer [6].

To install the library, first please download the pacakge: Triclops3.2.0.8-FC3.tgz, by registering onPoint Grey Research, Inc's website or directly download from here.

Second, you also need to download the package pgr-stereo-examples-libdc-2.0.2.tar.gzhere. It contains some example programs and a static library libpgrlibdcstereo.a.

1). Uncompress the file Triclops3.2.0.8-FC3.tgz to the directory/usr/local/include first:

            ~$ sudo tar zxvf Triclops3.2.0.8-FC3.tgz -C /usr/local/include

2). The uncompressed folder name is "Triclops3.2.0.8", rename it as "triclops":

             ~$ sudo mv /usr/local/include/Triclops3.2.0.8  /usr/local/include/triclops

3). Uncompress the file pgr-stereo-examples-libdc-2.0.2.tar.gz to some place, (I prefer/opt)

            ~$ sudo tar zxvf pgr-stereo-examples-libdc-2.0.2.tar.gz -C /opt

4). Now the uncompressed files under /opt should be:

              |--- /opt/pgr-stereo-examples-libdc-2.0.2
                  |---  Makefile
                  |--- pgrlibdcstereo
                  |--- simplegrab
                  |--- simplestereo
                  |--- x3bstereo
     

go to the subdirectory pgrlibdcstereo:

           ~$ cd /opt/pgr-stereo-examples-libdc-2.0.2/pgrlibdcstereo

Modify its Makefile with your favorate editor as below:

           ROOT_INCLUDE = /usr/local/include
           USER_INCLUDE = /usr/include
           # compilation flags
           CPPFLAGS += -I.
           # libdc1394 installed in /usr/include location
           CPPFLAGS += -I$(USER_INCLUDE)/dc1394
           CPPFLAGS += -I$(ROOT_INCLUDE)/triclops/include
           CPPFLAGS += -Wall -g
           CPPFLAGS += -DLINUX
           # library that will be generated 
           LIBRARY_NAME    = pgrlibdcstereo
           LIBRARY         = libpgrlibdcstereo.a
           LIBRARY_SRC     = pgr_conversions.cpp \
                             pgr_stereocam.cpp \
                             pgr_registers.cpp
           all:    $(LIBRARY)
           $(LIBRARY): $(LIBRARY_SRC:%.cpp=%.o)
                   $(AR) $(ARFLAGS) $@ $^
           %.o:%.cpp
                   g++ -c $(CXXFLAGS) $(CPPFLAGS) $*.cpp -o $*.o
           clean:
                   rm -f *~ *.o *.d $(EXECS)

Then compile the source codes:

          /opt/pgr-stereo-examples-libdc-2.0.2/pgrlibdcstereo$ sudo make

Copy the generated library to:

         /opt/pgr-stereo-examples-libdc-2.0.2/pgrlibdcstereo$ sudo cp libpgrlibdcstereo.a  /usr/local/include/triclops/lib
         /opt/pgr-stereo-examples-libdc-2.0.2/pgrlibdcstereo$ sudo cp *.h  /usr/local/include/triclops/include

Now we get our libraries for BumbleBee camera ready!

Run Test Program

The other three folders under /opt/pgr-stereo-examples-libdc-2.0.2 are sample codes, to run them we need to compile the source codes to executable files first. For example:

         ~$ cd  /opt/pgr-stereo-examples-libdc-2.0.2/simplestereo

Modify the Makefile here:

         ~/opt/pgr-stereo-examples-libdc-2.0.2/simplestereo$ vim Makefile         

Define ROOT_INCLUDE as the parent directory of triclops; USER_INCLUDE as parent directory of dc1394; USER_LIB as path oftriclops/lib; The modified Makefile looks like:

       ROOT_INCLUDE = /usr/local/include
       USER_INCLUDE = /usr/include
       ROOT_LIB = /usr/local/lib
       USER_LIB = /usr/local/include/triclops/lib
       LOCALLIB = ../pgrlibdcstereo
       # compilation flags
       CPPFLAGS += -I.
       # libdc1394 installed in /usr/local/include location
       CPPFLAGS += -I$(ROOT_INCLUDE)/dc1394
       CPPFLAGS += -I$(LOCALLIB)
       CPPFLAGS += -I$(ROOT_INCLUDE)/triclops/include
       CPPFLAGS += -Wall -g
       CPPFLAGS += -DLINUX
       # libraries flags
       LDFLAGS += -L. -L$(ROOT_LIB)/triclops
       LDFLAGS += -L$(LOCALLIB)
       LDFLAGS += -L$(ROOT_LIB)
       LDFLAGS += -L$(USER_LIB)
       LIBS    += -ldc1394 -lraw1394 -pthread
       LIBS    += -lpgrlibdcstereo -ltriclops -lpnmutils
       # executable name and contents
       EXEC1           = simplestereo
       EXEC1SRC        = $(EXEC1).cpp
       EXEC2           = simplestereo-profile
       EXEC2SRC        = $(EXEC2).cpp
       EXECS           = $(EXEC1) $(EXEC2)
       # compile
       all:    bin
       bin: $(EXECS)
       $(EXEC1): $(EXEC1SRC:%.cpp=%.o)
               $(CXX) $(LDFLAGS) -o $@ $^ $(LIBS)
       $(EXEC2): $(EXEC2SRC:%.cpp=%.o)
               $(CXX) $(LDFLAGS) -o $@ $^ $(LIBS) 
       %.o:%.cpp
               g++ -c $(CXXFLAGS) $(CPPFLAGS) $*.cpp -o $*.o
       clean:
               rm -f *~ *.o *.d $(EXECS) *.pgm *.ppm

Run make to commile:

       ~/opt/pgr-stereo-examples-libdc-2.0.2/simplestereo$ sudo make

Now we can use the sample program to test the BumbleBee stereo camera:

       ~/opt/pgr-stereo-examples-libdc-2.0.2/simplestereo$ ./simplestereo

The captured figures are listed below:

Figure 2: Raw image of TXT's nametag captured by BumbleBee's Left camera

Figure 3: Raw image of TXT's nametag captured by BumbleBee's Right camera

Figure 3: Rectified Image

Figure 4: Disparity Image

References

• [1] Point Grey Research, Inc. Using Point Grey Stereo Vision Products under Linux
• [2] Point Grey Research, Inc. Setting up 1394 devices using the Linux "udev" device manager
• [3] Point Grey Research, Inc. 64-bit Linux support for Digiclops and Triclops libraries
• [4] Point Grey Research, Inc. Source code examples of grabbing images under Linux
• [5] www.FedoraProject.org Building a custom kernel
• [6] Point Grey Research, Inc. Triclops™ SDK
• [7] The Image Group at the Department of Computer Science, University of Copenhagen, How to work with the Bumblebee2 Stereo Camera on Linux
• [8] Point Grey Research, Inc. Technical Application Note (TAN2008005): Stereo Vision Introduction and Applications (pdf)

Author

    Yang Song