分布式协调器ZooKeeper3.4—入门

最新推荐文章于 2024-09-08 06:50:48 发布

guxch

最新推荐文章于 2024-09-08 06:50:48 发布

阅读量2.7k

点赞数

分类专栏：分布式计算 Linux

Linux 同时被 2 个专栏收录

25 篇文章 4 订阅

订阅专栏

分布式计算

24 篇文章 2 订阅

订阅专栏

【ZooKeeper是Apache Hadoop下的开源软件，是一个分布式的协调器，本文来自于Zookeeper的官方网站，地址为：http://zookeeper.apache.org/doc/r3.4.5/ zookeeperStarted.html】

GettingStarted: Coordinating Distributed Applications with ZooKeeper

入门篇：用ZooKeeper协调分布式系统

This document contains information to get you started quickly with ZooKeeper. It is aimedprimarily at developers hoping to try it out, and contains simple installation instructions for a single ZooKeeper server, a few commands to verify that it isrunning, and a simple programming example. Finally, as a convenience, there area few sections regarding more complicated installations, for example running replicated deployments, and optimizing the transaction log. However for thecomplete instructions for commercial deployments, please refer to the ZooKeeper Administrator's Guide.

本文包含使你快速使用ZooKeeper的内容。它的主要目标是为那些希望使用的开发人员，所以包含了单机的简单安装指导，一些验证它是否运行的命令以及一个简单的编程实例，最后，为使用者方便，有一些部分涉及了复杂的安装过程，例如，复制部署、优化事务日志等，但是，实现商业部署的完整部署操作，请参阅“ZooKeeperAdministrator's Guide”。

Pre-requisites

先决准备条件

See System Requirements in the Admin guide.

请参阅管理员手册中的“系统要求”。

Download

下载

To get a ZooKeeper distribution, download a recentstable release from one of the Apache Download Mirrors.

要得到一个ZooKeeper的发行版，请到Apache下载镜像网站之一下载一个最新的稳定版。

Standalone Operation

Standalone安装

Setting up a ZooKeeper server in standalone mode is straight forward. The server is containedin a single JAR file, so installation consists of creating a configuration.

Once you've downloaded a stable ZooKeeper release unpack it and cd to the root

To start ZooKeeper you need a configuration file. Here is a sample, create it inconf/zoo.cfg:

tickTime=2000
dataDir=/var/lib/zookeeper
clientPort=2181

This file can be called anything, but for the sake of this discussion call itconf/zoo.cfg.Change the value of dataDir to specify an existing (empty to start with)directory. Here are the meanings for each of the fields:

tickTime

thebasic time unit in milliseconds used by ZooKeeper. It is used to do heartbeatsand the minimum session timeout will be twice the tickTime.

dataDir

thelocation to store the in-memory database snapshots and, unless specifiedotherwise, the transaction log of updates to the database.

clientPort

the port to listen for client connections

Now that you created the configuration file, you can start ZooKeeper:

bin/zkServer.sh start

ZooKeeper logs messages using log4j -- more detail available in theLogging section of the Programmer's Guide. You will see log messages coming to theconsole (default) and/or a log file depending on the log4j configuration.

The steps outlined here run ZooKeeper in standalone mode. There is no replication, so ifZooKeeper process fails, the service will go down. This is fine for most development situations, but to run ZooKeeper in replicated mode, please see Running ReplicatedZooKeeper.

在一个standalone环境中搭建ZooKeeper，很简明。服务软件包含在一个JAR文件中，安装将包含创建配置文件。

一旦你下载了一个ZooKeeper的稳定版，解压它，转到root目录。要启动ZooKeeper，需要一个配置文件，以下是一个例子，将它创建在conf/zoo.cfg中：

tickTime=2000

dataDir=/var/lib/zookeeper

clientPort=2181

这个文件可以叫任何名字，但为了讨论方便，就叫它conf/zoo.cfg。改变dataDir的值，使之是一个已存在的空目录。以下是每个值的意义。

tickTime

ZooKeeper使用的基本时间单位（毫秒）。它被用来做心跳，并且最小的Session过期时间是两倍的tickTime。

dataDir

存储内存数据库快照的位置，以及对这个数据更新日志的位置（如果不指定的话）。

clientPort

侦听客户端连接的端口。

现在，你创建了这个配置文件，就可以启动ZooKeeper了：

bin/zkServer.sh start

ZooKeeper采用log4j来记录日志信息－更多的关于日志部分的信息，请参考编程手册。由log4j配置所决定，你可以看到日志信息显示在控制台上（缺省情况）并/或者写入日志文件。

这里给出的是standalone情况下的大致运行ZooKeeper的步骤，不涉及复制（Replicated），所以，如果ZooKeeper进程发生错误，ZooKeeper服务就终止了。这适合于大部分开发情况，如果想让ZooKeeper运行于复制情况，请参阅“RunningReplicated ZooKeeper”。

Managing ZooKeeper Storage

ZooKeeper存储管理

For long running production systems ZooKeeper storage must be managed externally(dataDir and logs). See the section onmaintenance formore details.

如果要长期运行，ZooKeeper如何存储必需放到外部管理（dataDir和logs），详情请参阅“maintenance”一节。

Connectingto ZooKeeper

连接到ZooKeeper

Once ZooKeeper is running, you have several options for connection to it:

Java: Use

bin/zkCli.sh -server 127.0.0.1:2181

This lets you perform simple, file-like operations.

C: compile cli_mt (multi-threaded) or cli_st (single-threaded) by running make cli_mt or make cli_st in thesrc/c subdirectory in the ZooKeeper sources. See the README contained withinsrc/c for full details.

You can run the program from src/c using:

LD_LIBRARY_PATH=. cli_mt 127.0.0.1:2181

LD_LIBRARY_PATH=. cli_st 127.0.0.1:2181

This will give you a simple shell to execute file system like operations onZooKeeper.

ZooKeeper运行后，你有几个选择来连接它：

Java: 使用如下方法，这让你采用一种简单的、类似文件的操作方法。

bin/zkCli.sh -server 127.0.0.1:2181

C: 通过运行make cli_mt 或 make cli_st 来编译cli_mt (多线程)或 cli_st (单线程) ，它们在 ZooKeeper源文件的src/c 子目录下，详情请参阅src/c 下的README。

在src/c下，你可以用以下命令运行程序：

LD_LIBRARY_PATH=. cli_mt 127.0.0.1:2181

或

LD_LIBRARY_PATH=. cli_st 127.0.0.1:2181

这个命令会给你提供一个简单的运行壳（shell）：在ZooKeeper上可以运行类似文件系统一样的操作。

Once you have connected, you should see something like:

一旦连接上之后，你将看到如下的东西：

Connecting to localhost:2181
log4j:WARN No appenders could be found for logger (org.apache.zookeeper.ZooKeeper).
log4j:WARN Please initialize the log4j system properly.
Welcome to ZooKeeper!
JLine support is enabled
[zkshell: 0]

From the shell, type help to get a listing of commands that can be executed from the client, as in:

从这个壳（shell）中，键入help，可以得到一系列可以在客户端运行的命令，例如：

[zkshell: 0] help
ZooKeeper host:port cmd args
        get path [watch]
        ls path [watch]
        set path data [version]
        delquota [-n|-b] path
        quit
        printwatches on|off
        createpath data acl
        stat path [watch]
        listquota path
        history
        setAcl path acl
        getAcl path
        sync path
        redo cmdno
        addauth scheme auth
        delete path [version]
        deleteall path
        setquota -n|-b val path

From here,you can try a few simple commands to get a feel for this simple command line interface. First, start by issuing the list command, as in ls, yielding:

现在起，你就能试试一些简单的命令，感觉一下这个命令行界面。首先，试试list命令，如下：

[zkshell: 8] ls /
[zookeeper]

Next, createa new znode by running create /zk_test my_data.This creates a new znode and associates the string "my_data" with thenode. You should see:

下一步，运行create/zk_test my_data来创建一个新的znode，字符串my_data将与这个新的znode关联，运行情况如下：

[zkshell: 9] create /zk_test my_data
Created /zk_test

Issue another ls / command to see what thedirectory looks like:

键入ls/，能看到类似下面的目录

[zkshell: 11] ls /
[zookeeper, zk_test]

Notice that the zk_test directory has now been created.

Next, verify that the data was associated with the znode by running the get command, as in:

注意此时zk_test目录已经被创建了。

下一步，验证与此节点关联的数据，运行get命令，如下：

[zkshell: 12] get /zk_test
my_data
cZxid = 5
ctime = Fri Jun 05 13:57:06 PDT 2009
mZxid = 5
mtime = Fri Jun 05 13:57:06 PDT 2009
pZxid = 5
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0
dataLength = 7
numChildren = 0

We can change the data associated with zk_test by issuing the setcommand, as in:

我们能用set命令修改与zk_test关联的数据，如下：

[zkshell: 14] set /zk_test junk
cZxid = 5
ctime = Fri Jun 05 13:57:06 PDT 2009
mZxid = 6
mtime = Fri Jun 05 14:01:52 PDT 2009
pZxid = 5
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0
dataLength = 4
numChildren = 0
[zkshell: 15] get /zk_test
junk
cZxid = 5
ctime = Fri Jun 05 13:57:06 PDT 2009
mZxid = 6
mtime = Fri Jun 05 14:01:52 PDT 2009
pZxid = 5
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0
dataLength = 4
numChildren = 0

(Notice we did a get after setting the data and it did, indeed, change.

Finally, let's delete the node by issuing:

注意我们在set之后用get查看，它确实变了。

最后，我们用下面的方法删除这个节点。

[zkshell: 16] delete /zk_test
[zkshell: 17] ls /
[zookeeper]
[zkshell: 18]

That's it for now. To explore more, continue with the rest of this document and see theProgrammer'sGuide.

现在，先到这里，如果想试验更多的内容，请继续读下去，并参考Programmer'sGuide。

Programmingto ZooKeeper

ZooKeeper编程

ZooKeeper has a Java bindings and C bindings. They are functionally equivalent. The C bindings exist in two variants: single threaded and multi-threaded. Thesediffer only in how the messaging loop is done. For more information, see the Programming Examples in the ZooKeeper Programmer's Guide for sample code using of the different APIs.

ZooKeeper有一个Java绑定和C绑定。它们在功能上是一样的。C绑定有两个套：单线程的和多线程的，它们的不同仅在消息循环的处理上。更详细的内容，包括不同的API及例子，请参见ProgrammingExamples in the ZooKeeper Programmer's Guide。

Running Replicated ZooKeeper

复制（Replicated）情况下搭建ZooKeeper

Running ZooKeeper in standalone mode is convenient for evaluation, some development,and testing. But in production, you should run ZooKeeper in replicated mode. Areplicated group of servers in the same application is called aquorum,and in replicated mode, all servers in the quorum have copies of the same configuration file. The file is similar to the one used in standalone mode, butwith a few differences. Here is an example:

在standalone情况下运行ZooKeeper，对试着用用、简单编程等来说是合适的。但在正式的生产场合，应该运行Replicated工作方式下的ZooKeeper。属于同一应用的若干ZooKeeper服务器，被叫做quorum，在复制工作模式下，quorum中所有的服务器都有相同的配置文件（不同拷贝），这个文件类似于standalone模式下使用的，但稍有不同，下面是一个例子

tickTime=2000
dataDir=/var/lib/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888

The new entry, initLimit is timeouts ZooKeeper uses to limit the length of time the ZooKeeper servers in quorum have to connect to a leader. The entrysyncLimit limits how far out of date a server can be from a leader.

With both of these timeouts, you specify the unit of time usingtickTime. In this example, the timeout for initLimit is 5 ticks at 2000 milleseconds a tick, or10 seconds.

The entries of the form server.X list the servers that make up the ZooKeeper service.When the server starts up, it knows which server it is by looking for the filemyidin the data directory. That file has the contains the server number, in ASCII.

Finally, notethe two port numbers after each server name: " 2888" and"3888". Peers use the former port to connect to other peers. Such aconnection is necessary so that peers can communicate, for example, to agreeupon the order of updates. More specifically, a ZooKeeper server uses this port to connect followers to the leader. When a new leader arises, a follower opensa TCP connection to the leader using this port. Because the default leaderelection also uses TCP, we currently require another port for leader election.This is the second port in the server entry.

Note

If you want to test multiple servers on a single machine,specify the servername aslocalhost with unique quorum & leaderelection ports (i.e. 2888:3888, 2889:3889, 2890:3890 in the example above) foreach server.X in that server's config file. Of course separatedataDirsand distinct clientPorts are also necessary (in the above replicatedexample, running on a singlelocalhost, you would still have threeconfig files).

新增加的条目，initLimit是quorum中其他服务器用来连接leader服务器的超时值，ZooKeeper服务器用此值限制了它们之间连接的时间。syncLimit是其他服务器能够不与leader通讯的最大允许值。

这两个超时值都是基于基本时间单位tickTime的，在这个例子中，initLimit是5跳，2000ms一跳的话，就是10秒。

下面的条目，类似server.X的列表，是构成ZooKeeper服务的服务器。当服务器启动时，它通过在数据目录中查找myid文件（该文件以ASCII码方式写明了服务器的数字编号），来确定它是哪一台服务器。

最后，请注意每个服务器名字后的两个端口：2888和3888。服务器之间用前一个端口相互连接，这个连接是必要的，因为它们需要通讯，例如，协调更新的顺序。确切来说，ZooKeeper服务器用这个端口来连接follower和leader。当一个新leader产生后，follower用此端口来连接leader。由于leader的选举也是TCP连接，现在我们需要另一个端口来进行选举，这就是服务器条目中的第二个端口。

注：

如果你想在单台机器上试验多个服务器，将servername设置为localhost，每个server.X的配置文件中，设置各端口，如i.e.2888:3888, 2889:3889, 2890:3890。当然，不同的dataDirs和clientPort是必须的（在上面的例子中，在一台机器localhost上运行replicated模式ZooKeeper，你依然需要3个配置文件）。

Other Optimizations

其他的优化

There are a couple of other configuration parameters that can greatly increase performance:

To get low latencies on updates it is important to have a dedicated transaction log directory. By default transaction logs are put in the same directory as the data snapshots andmyid file. The dataLogDir parameters indicates a different directory to use for the transaction logs.
[tbd: what is the other config param?]

有些配置参数会极大的增加运行性能：