network block device(nbd)

网络块设备是一个廉价的共享存储解决方案, 结合分布式文件系统可以构建比较有规模的共享块设备.

例如, 在以下架构里面, 分布式存储提供冗余,共享和扩展性. 

多台NBD提供冗余以及共享网络块设备.

NBD SERVER和分布式存储可以共享主机, 也可以分开主机. 

这个架构需要注意的问题,

例如操作系统层缓存, 当从nbd-server切换到另一个nbd-server时, 如果有缓存为写入镜像文件的话, 会导致数据丢失.

例子 : 

使用nbd构建Oracle-RAC : 

典型的例子, 使用共享存储.

使用NBD : 

NBD在WIKI中的介绍 : 

Network block device

From Wikipedia, the free encyclopedia

In Linux, a network block device is a device node whose content is provided by a remote machine. Typically, network block devices are used to access a storage device that does not physically reside in the local machine but on a remote one. As an example, the local machine can access a fixed disk that is attached to another computer.

Contents

   [hide] 

• 1 Kernel client/userspace server
• 2 Example
• 3 Availability
• 4 See also
• 5 References
• 6 External links

Kernel client/userspace server[edit]

Technically, a network block device is realized by two components. In the client machine, where the device node is to work, a kernel module named nbd controls the device. Whenever a program tries to access the device, this kernel module forwards the request to the server machine, where the data physically resides.

On the server machine, requests from the client are handled by a userspace program called nbd-server. This program is not implemented as a kernel module because all it has to do is to serve network requests, which in turn just requires regular access to the server filesystem.

Example[edit]

If the file /tmp/xxx on ComputerA has to be made accessible on ComputerB, one performs the following steps:

On ComputerA:

nbd-server 2000 /tmp/xxx

On ComputerB:

modprobe nbd
nbd-client ComputerA 2000 /dev/nbd0

The file is now accessible on ComputerB as device /dev/nbd0. If the original file was for example a disk image, it could be mounted for example via mount /dev/nbd0 /mnt/whatever.

The command modprobe nbd is not necessary if module loading is done automatically. Once the module is in the kernel, nbd-client is used to send commands to it, such as associating a given remote file to a given local nb device. To finish using /dev/nbd0, that is, to destroy its association with the file on other computer, one can run nbd-client -d /dev/nbd0 on ComputerB.

In this example, 2000 is the number of the server port through which the file is made accessible. Any available port could be used.

Availability[edit]

The network block device client module is available on Linux and GNU Hurd.

Since the server is a userspace program, it can potentially run on every Unix-like platform. It was ported to Solaris.^[1]

在CentOS或RHEL下可以使用EPEL附加仓库安装nbd :

[root@150 postgresql-9.3.5]# yum install -y nbd
Loaded plugins: fastestmirror, refresh-packagekit, security, versionlock
Loading mirror speeds from cached hostfile
epel/metalink                                                                                                | 5.4 kB     00:00     
 * base: mirrors.skyshe.cn
 * epel: mirrors.ustc.edu.cn
 * extras: mirrors.163.com
 * updates: centos.mirror.cdnetworks.com
base                                                                                                         | 3.7 kB     00:00     
extras                                                                                                       | 3.3 kB     00:00     
updates                                                                                                      | 3.4 kB     00:00     
updates/primary_db                                                                                           | 5.3 MB     00:21     
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package nbd.x86_64 0:2.9.20-7.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================================================
 Package                     Arch                           Version                              Repository                    Size
====================================================================================================================================
Installing:
 nbd                         x86_64                         2.9.20-7.el6                         epel                          43 k

Transaction Summary
====================================================================================================================================
Install       1 Package(s)

Total download size: 43 k
Installed size: 83 k
Downloading Packages:
nbd-2.9.20-7.el6.x86_64.rpm                                                                                  |  43 kB     00:00     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : nbd-2.9.20-7.el6.x86_64                                                                                          1/1 
  Verifying  : nbd-2.9.20-7.el6.x86_64                                                                                          1/1 

Installed:
  nbd.x86_64 0:2.9.20-7.el6                                                                                                         

Complete!

包含的内容 : 

[root@150 postgresql-9.3.5]# rpm -ql nbd
/usr/bin/nbd-server
/usr/sbin/nbd-client
/usr/share/doc/nbd-2.9.20
/usr/share/doc/nbd-2.9.20/README
/usr/share/doc/nbd-2.9.20/cliserv.h
/usr/share/doc/nbd-2.9.20/nbd-tester-client.c
/usr/share/doc/nbd-2.9.20/simple_test
/usr/share/man/man1/nbd-server.1.gz
/usr/share/man/man5/nbd-server.5.gz
/usr/share/man/man8/nbd-client.8.gz

[参考]

1.  http://en.wikipedia.org/wiki/Network_block_device

2.  http://www.fi.muni.cz/~kripac/orac-nbd/

3.  http://www.fi.muni.cz/~kripac/oracle-altix/

4.  http://sourceforge.net/projects/nbd/

5.  https://github.com/yoe/nbd

6.  http://nbd.sourceforge.net/

7.  http://web.freenas.org/images/resources/freenas9.2.1/freenas9.2.1_guide.pdf

8.  http://fedoraproject.org/wiki/EPEL

9. 

man 1 nbd-server

man 5 nbd-server

man 8 nbd-client