野史终归不如官方版本的,转自http://sourceforge.net/apps/trac/kosmosfs/wiki/HowToCompile
Required Packages¶
To compile and run KFS, you need to have the following software packages installed on your machine:
- Boost (preferably, version 1.34 or higher)
- cmake (preferably, version 2.4.6 or higher)
- log4cpp (preferably, version 1.0)
- gcc version 4.1 (or higher)
- xfs devel RPMs on Linux
This document assumes that you have downloaded the source to directory: ~/code/kfs. We assume that you want to build the source in ~/code/kfs/build.
There are few parts to compiling the source:
- Build the C++ side to get the metaserver/chunkserver binaries, tools, and the C++ client library.
- Build the Java side to get a kfs.jar file which contains the wrapper calls to native C++ via JNI; this allows Java apps to access files stored in KFS.
- Building Python extension module for Python support (optional)
Building C++ Components¶
It is preferable to build KFS binaries with debugging symbols. The sequence of steps are:
cd ~/code/kfs
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RelWithDebInfo ~/code/kfs/
gmake
gmake install
The binaries will be installed in:
- Executables will be in: ~/code/kfs/build/bin
- Libraries will be in: ~/code/kfs/build/lib
Building Java Components¶
To build Java support setup:
cd ~/code/kfs ant jar
This will produce the following set of files:
- ~/code/kfs/build/classes --- This will contain the Java class files
- ~/code/kfs/build/kfs-{version}.jar --- The jar file containing the Java classes
Add .jar file to your CLASSPATH environment variable
export CLASSPATH=${CLASSPATH}:~/code/kfs/build/kfs-[version].jar
Building Python Support¶
Build the KFS client library (see above). Let the path to the shared libraries is ~/code/kfs/build/lib. Now, to build the Python extension module:
cd to ~/code/kfs/src/cc/access Edit kfs_setup.py and setup the include path. Specifically, kfsext = Extension('kfs', include_dirs ['kfs/src/cc/', '<path to boost>']) python kfs_setup.py ~/code/kfs/build/lib/ build
This will build a shared object library _kfs.so_. The kfs.so library needs to be installed either in the site-packages for python or in an alternate location. To install in site-packages for python:
python kfs_setup.py ~/code/kfs/build/lib/ install
To install in alternate location such as, ~/code/kfs/build/lib
python kfs_setup.py ~/code/kfs/build/lib install --home=~/code/kfs/build/lib
If you installed in alternate location, update PYTHONPATH environment variables:
export PYTHONPATH=${PYTHONPATH}:~/code/kfs/build/lib/lib64/python
Also, update the LD_LIBRARY_PATH environment variable:
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:~/code/kfs/build/lib
Using Python Client on the Mac (Leopard)¶
For the Mac, update the DYLD_LIBRARY_PATH environment variable (so that libkfsClient.dylib will be found):
export DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}:~/code/kfs/build/lib
Requirements¶
The KFS package comes with scripts for installing/starting/stopping KFS servers on a set of machines. The scripts use ssh to login and execute commands (either on a single node or in a cluster). The scripts require that the user have "no password" ssh access to every affected server. For every host affected, be sure you can "ssh host true" without being prompted for a password.
Defining Machine Configuration for KFS-0.2.3 (or higher)¶
With KFS-0.2.3. (or higher), the system configuration can be defined using two files. One that defines the enviroment (such as, paths, port numbers to etc) for all the chunkservers and another file that lists the nodes on which the chunkservers should be started. The set of machines on which KFS servers have to be deployed is defined in a configuration file that follows the Python config file format. For example, the config file machines.cfg:
[metaserver] node: machine1 clusterkey: kfs-test-cluster rundir: /mnt/kfs/meta baseport: 20000 loglevel: INFO numservers: 2 [chunkserver_defaults] rundir: /mnt/kfs/chunk chunkDir: /mnt/kfs/chunk/bin/kfschunk baseport: 30000 space: 3400 G loglevel: INFONext, assuming that there are 3 nodes in the cluster, their names should be listed in a separate file, machines.txt. The format of this file is one entry per line. For example,
10.2.3.1 10.2.3.2 10.2.3.3Defining Machine Configuration (Old style format)¶
The set of machines on which KFS servers have to be deployed is defined in a configuration file that follows the Python config file format. For example,
[metaserver] node: machine1 clusterkey: kfs-test-cluster rundir: /mnt/kfs/meta baseport: 20000 loglevel: INFO numservers: 2 [chunkserver1] node: machine2 rundir: /mnt/kfs/chunk chunkDir: /mnt/kfs/chunk/bin/kfschunk baseport: 30000 space: 3400 G loglevel: INFO [chunkserver2] node: machine2 rundir: /mnt/kfs/chunk chunkDir: /mnt/kfs/chunk/bin/kfschunk baseport: 30000 space: 3400 G loglevel: INFOThis file defines a configuration for a deployment comprising of a metaserver and two chunkservers. The various fields:
- node: This defines the machine name where the binary should run
- rundir: This defines the directory on the machine where KFS binaries will be installed.
- baseport: This port at which the metaserver/chunkserver process will listen for connection from clients
- loglevel: The level for outputting messages. Since KFS uses log4cpp, the values are INFO/DEBUG
- space: The storage space exported by a chunkserver for storing chunks (units are 'G' for GigaBytes and 'M' for MegaBytes)
- chunkDir : The list of directories used to store chunk files on the chunkserver nodes. For a JBOD configuration, this would be a space separated list of directory names.
- clusterkey : A key that is shared between metaserver/chunkserver. On startup, each chunkserver announces itself to the metaserver and provides the cluster key. This key should match the value that the metaserver; only then is the chunkserver part of the KFS deployment.
Script for install/launching processes¶
Two sets of scripts are checked into the repository:
- Linux: Use ~/code/kfs/scripts
- Solaris, Mac OSX: Use ~/code/kfs/scripts.solaris
The instructions in this section are for Linux platform. Use the corresponding script in scripts.solaris for Solaris/Mac platforms. When you use any script which will transfer files (particularly kfssetup) on the Macintosh, you will need the additional option --tar=tar; the scripts assume that GNU tar is named "gtar" otherwise.
Installing KFS Binaries¶
Define the configuration in machines.cfg. Then, run the install script. The install script runs a set of processes concurrently. This works well when the servers need to be configured for a distributed setting. For a single node setup, the processes need to be launched sequentially:
- When all the servers are on a single host
cd ~/code/kfs/scripts python kfssetup.py -f machines.cfg -m machines.txt -b ../build -w ../webui -s- MacOSX:
cd ~/code/kfs/scripts.solaris python kfssetup.py -f machines.cfg -m machines.txt -b ../build -w ../webui --tar=tar -s- When the servers are on multiple hosts
cd ~/code/kfs/scripts python kfssetup.py -f machines.cfg -m machines.txt -b ../build -w ../webuiNOTE: The "-m" option is supported only with KFS-0.2.3 or higher.
Upgrading from prior KFS 0.2.x releases¶
Stop the servers. Upgrade the binaries and relaunch:
- Stopping the servers:
cd ~/code/kfs/scripts python kfslaunch.py -f machines.cfg -S
- When all the servers are on a single host
cd ~/code/kfs/scripts python kfssetup.py -f machines.cfg -b ../build -w ../webui -u -s
- When the servers are on multiple hosts
cd ~/code/kfs/scripts python kfssetup.py -f machines.cfg -b ../build -w ../webui -uStarting KFS servers¶
To launch the KFS servers, run the launch script:
cd ~/code/kfs/scripts python kfslaunch.py -f machines.cfg -m machines.txt -sChecking System Status¶
To verify that the servers started up and are connected to the metaserver, use the kfsping tool:
cd ~/code/kfs/build/bin/tools kfsping -m -s <metaserver host> -p <metaserver port>If you using KFS-0.2.2 or higher, you can use a Web browser to monitor the servers. The KFS package now includes a simple python-based web server that shows the set of servers that are currently connected to the metaserver. The KFS web server runs on the same machine as the KFS metaserver. The web server's port is metaserver's baseport + 50. For example,
metaserver runs on machine node1 and uses port 20000 the web server runs on machine node1 at port 20050. Point the browser at http://node1:20050The KFS web server uses a file "all-machines.txt" to track where the chunkservers should be running. This file should be manually created and placed in ~/code/kfs/webui before deploying KFS binaries. The format of this file is the name of the chunkserver machines, one entry per line. For example.
10.2.3.1 10.2.3.2 10.2.3.3When you open the page, http://node1:20050/cluster-view, the web server will return a web page that lists:
- where the chunkservers are currently running
- where the chunkservers could not be started (due to ssh failures, etc.)
- where the chunkservers have failed
Stopping KFS servers¶
To stop the KFS servers, run the launch script:
cd ~/code/kfs/scripts python kfslaunch.py -f machines.cfg -m machines.txt -SAdding new chunkservers¶
It is not necessary to stop the KFS servers to add new chunkservers. Simply update the machines.cfg file and start the servers (see Starting KFS servers).
Un-installing KFS Binaries¶
To format the filesystem/uninstall the binaries, run the install script:
cd ~/code/kfs/scripts python kfssetup.py -f machines.cfg -m machines.txt -b ../build/bin -U