使用Zookeeper进行开发 —一个基本教程
介绍
在这个指导中,将会告诉你怎样使用Zookeeper实现屏障和生产者-消费者队列. 我们将使用到两个类: Barrier和Queue. 这些实例假定你至少有一个Zookeeper服务器在运行.
它们都使用下面的代码:
static ZooKeeper zk = null; static Integer mutex; String root; SyncPrimitive(String address) { if(zk == null){ try { System.out.println("Starting ZK:"); zk = new ZooKeeper(address, 3000, this); mutex = new Integer(-1); System.out.println("Finished starting ZK: " + zk); } catch (IOException e) { System.out.println(e.toString()); zk = null; } } } synchronized public void process(WatchedEvent event) { synchronized (mutex) { mutex.notify(); } }
所有类都继承自SyncPrimitive. 通过这个方法,我们执行SyncPrimitive构造方法是使用通用的步骤. 为了保持实例简单,我们创建一个Zookeeper对象,首先我们我们创建一个Barrier和Queue,并且我们创建一个静态对象来引用这个对象. 之后Barrier和Queue实例会检查Zookeeper对象是否存在. 或者,我们能够创建一个Zookeeper对象,并把它传递到Barrier和Queue的构造函数中.
我们使用process()方法去处理监听器触发的通知. 在下面的内容中,我们会讨论watches的代码实现. 一个监听者内部构造使Zookeeper能够通知节点变化到客户端. 例如,如果一个客户端正等待其他客户端离开Barrier,接着他设定一个观察者,等待对于指定节点的修改,它可以表明这是等待的结束. 一旦我们完车这个例子,这个点会变的特别清楚.
阻塞
barrier是一个元件,它能够使用一组进程去同步计算的开始和结束. 这个实现的一般方法是有一个barrier节点为成为各个处理节点的父节点的目的服务。我们假设这个阻塞节点叫做"/b1". 然后每个进程"p"创建一个节点"/b1/p". 一旦足够的进程创建相应的节点, 加入的过程就可以开始计算.
在这个例子,每一个进程都会实例化一个Barrier对象,它的构造函数作为参数:
Zookeeper服务的地址 (例如 "zoo1.foo.com:2181")
Zookeeper阻塞节点的路径(例如 "/b1")
进程组的个数
Barrier的构造方法通过传入Zookeeper服务的地址去构造父类. 如果Zookeeper实例不存在,那么父类就会创建一个Zookeeper实例. 然后Barrier的构造函数会接着在Zookeeper上创建一个阻塞节点,它是所有进程节点的父节点,我们把它称为root(注意:不是Zookeeper root "/"的意思)
/** * Barrier的构造方法 * * @param address * @param root * @param size */ Barrier(String address, String root, int size) { super(address); this.root = root; this.size = size; // Create barrier node if (zk != null) { try { Stat s = zk.exists(root, false); if (s == null) { zk.create(root, new byte[0], Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT); } } catch (KeeperException e) { System.out .println("Keeper exception when instantiating queue: " + e.toString()); } catch (InterruptedException e) { System.out.println("Interrupted exception"); } } // My node name try { name = new String(InetAddress.getLocalHost().getCanonicalHostName().toString()); } catch (UnknownHostException e) { System.out.println(e.toString()); } }
进入barrier, 一个进程调用enter(). 这个进程在root下创建一个叫节点去表示自己,使用它的主机名称来表示节点名称. 然后等待足够的进程进入Barrier. 进程通过"getChildren()"方法获得root节点的子节点数量,并且不到一定的数量时,会一直等待. 当root节点变化时接收通知,一个进程必须设置一个监听者,并通过调用"getChildren()"完成. 在代码里,getChildren()方法有两个参数. 第一个参数是从哪个节点读取,第二个参数是一个Boolean值,指明是否设置监听者. 在代码里这个flag是true.
/** * Join barrier * * @return * @throws KeeperException * @throws InterruptedException */ boolean enter() throws KeeperException, InterruptedException{ zk.create(root + "/" + name, new byte[0], Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL); while (true) { synchronized (mutex) { List<String> list = zk.getChildren(root, true); if (list.size() < size) { mutex.wait(); } else { return true; } } } }
注意enter()方法包含两个异常KeeperException和InterruptedException, 所以需要应用来catch和处理这些异常.
一旦计算完成,一个进程会调用leave()去离开lbarrier. 首先它会删除所对应的节点,然后会获取root节点的孩子节点. 如果至少有一个子节点,接着它会等待通知(obs: 注意调用getChildren()方法的第二个参数是true(),含义是为root节点设置一个监听者). 收到通知后,它会再次检查root节点是否有子节点.
/** * Wait until all reach barrier * * @return * @throws KeeperException * @throws InterruptedException */ boolean leave() throws KeeperException, InterruptedException{ zk.delete(root + "/" + name, 0); while (true) { synchronized (mutex) { List<String> list = zk.getChildren(root, true); if (list.size() > 0) { mutex.wait(); } else { return true; } } } } }
生产者和消费队列
生产者-消费者队列是由一连串的生产和消费的分布式的数据结构组成,生产者创建新元素并添加到队列. 消费者进程从列表中删除元素,并处理他们. 在这个实现中, 元素是简单的Integers. 一个队列由一个根节点表示, 添加一个元素去一个队列,一个生产者创建一个新的节点, root节点的孩子节点.
以下代码片段是对象的构造. 与Barrier对象一样, 它首先调用了父类的构造方法,SyncPrimitive, 如果Zookeeper对象不存在则创建一个Zookeeper对象. 接着会检查队列节点是否存在,若不存在则创建.
/** * Constructor of producer-consumer queue * * @param address * @param name */ Queue(String address, String name) { super(address); this.root = name; // Create ZK node name if (zk != null) { try { Stat s = zk.exists(root, false); if (s == null) { zk.create(root, new byte[0], Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT); } } catch (KeeperException e) { System.out .println("Keeper exception when instantiating queue: " + e.toString()); } catch (InterruptedException e) { System.out.println("Interrupted exception"); } } }
一个生产者调用produce()添加一个元素到队列,并传入一个Interger作为参数. 添加一个元素到队列,这个方法创建使用create()创建一个新的节点, 并使用SEQUENCE标志指示Zookeeper追加根节点顺序计数器的值. 用这个方法, 我利用队列元素上的最终顺序,确保队列最老的元素是下一个消费对象.
/** * Add element to the queue. * * @param i * @return */ boolean produce(int i) throws KeeperException, InterruptedException{ ByteBuffer b = ByteBuffer.allocate(4); byte[] value; // Add child with value i b.putInt(i); value = b.array(); zk.create(root + "/element", value, Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT_SEQUENTIAL); return true; }
消费一个节点,一个消费进程获得根节点的孩子节点, 读取节点计数中最小的值,返回该值对应的元素. 注意如果存在冲突,两个竞争的进程中一个能够删除节点,另一个删除操作会出现异常.
调用getChildren()返回字典顺序的孩子节点集合. 由于字段顺序不需要组训计数器值的顺序, 我们需要确定元素中最小的. 为了确定计数器中最小的值,我们会遍历集合,去除"element"元素获取最小的值.
/** * 从队列里删除第一个元素 * * @return * @throws KeeperException * @throws InterruptedException */ int consume() throws KeeperException, InterruptedException{ int retvalue = -1; Stat stat = null; // Get the first element available while (true) { synchronized (mutex) { List<String> list = zk.getChildren(root, true); if (list.size() == 0) { System.out.println("Going to wait"); mutex.wait(); } else { Integer min = new Integer(list.get(0).substring(7)); for(String s : list){ Integer tempValue = new Integer(s.substring(7)); //System.out.println("Temporary value: " + tempValue); if(tempValue < min) min = tempValue; } System.out.println("Temporary value: " + root + "/element" + min); byte[] b = zk.getData(root + "/element" + min, false, stat); zk.delete(root + "/element" + min, 0); ByteBuffer buffer = ByteBuffer.wrap(b); retvalue = buffer.getInt(); return retvalue; } } } } }
完整例子
在这部分中,你可以完整执行命令行程序去验证上面的内容. 使用下面的命令运行它.
ZOOBINDIR="[zookeeper目录地址]/bin" . "$ZOOBINDIR"/zkEnv.sh java SyncPrimitive [启动类型 qTest(阻塞)|其他(队列)] [服务地址 使用ip:port形式] [测试元素个数] [字符串p(生产者)|其他(消费者)]
队列测试
启动生产者创建100个元素
java SyncPrimitive qTest localhost 100 p
启动消费者消费100个元素
java SyncPrimitive qTest localhost 100 c
阻塞测试
启动对两个参与者启动阻塞(开始时,你想加入的参与者可能很多)
java SyncPrimitive bTest localhost 2
Source Listing
import java.io.IOException; import java.net.InetAddress; import java.net.UnknownHostException; import java.nio.ByteBuffer; import java.util.List; import java.util.Random; import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.KeeperException; import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.Watcher; import org.apache.zookeeper.ZooKeeper; import org.apache.zookeeper.ZooDefs.Ids; import org.apache.zookeeper.data.Stat; public class SyncPrimitive implements Watcher { static ZooKeeper zk = null; static Integer mutex; String root; SyncPrimitive(String address) { if(zk == null){ try { System.out.println("Starting ZK:"); zk = new ZooKeeper(address, 3000, this); mutex = new Integer(-1); System.out.println("Finished starting ZK: " + zk); } catch (IOException e) { System.out.println(e.toString()); zk = null; } } //else mutex = new Integer(-1); } synchronized public void process(WatchedEvent event) { synchronized (mutex) { //System.out.println("Process: " + event.getType()); mutex.notify(); } } /** * Barrier */ static public class Barrier extends SyncPrimitive { int size; String name; /** * Barrier constructor * * @param address * @param root * @param size */ Barrier(String address, String root, int size) { super(address); this.root = root; this.size = size; // Create barrier node if (zk != null) { try { Stat s = zk.exists(root, false); if (s == null) { zk.create(root, new byte[0], Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT); } } catch (KeeperException e) { System.out .println("Keeper exception when instantiating queue: " + e.toString()); } catch (InterruptedException e) { System.out.println("Interrupted exception"); } } // My node name try { name = new String(InetAddress.getLocalHost().getCanonicalHostName().toString()); } catch (UnknownHostException e) { System.out.println(e.toString()); } } /** * Join barrier * * @return * @throws KeeperException * @throws InterruptedException */ boolean enter() throws KeeperException, InterruptedException{ zk.create(root + "/" + name, new byte[0], Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL); while (true) { synchronized (mutex) { List<String> list = zk.getChildren(root, true); if (list.size() < size) { mutex.wait(); } else { return true; } } } } /** * Wait until all reach barrier * * @return * @throws KeeperException * @throws InterruptedException */ boolean leave() throws KeeperException, InterruptedException{ zk.delete(root + "/" + name, 0); while (true) { synchronized (mutex) { List<String> list = zk.getChildren(root, true); if (list.size() > 0) { mutex.wait(); } else { return true; } } } } } /** * Producer-Consumer queue */ static public class Queue extends SyncPrimitive { /** * Constructor of producer-consumer queue * * @param address * @param name */ Queue(String address, String name) { super(address); this.root = name; // Create ZK node name if (zk != null) { try { Stat s = zk.exists(root, false); if (s == null) { zk.create(root, new byte[0], Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT); } } catch (KeeperException e) { System.out .println("Keeper exception when instantiating queue: " + e.toString()); } catch (InterruptedException e) { System.out.println("Interrupted exception"); } } } /** * Add element to the queue. * * @param i * @return */ boolean produce(int i) throws KeeperException, InterruptedException{ ByteBuffer b = ByteBuffer.allocate(4); byte[] value; // Add child with value i b.putInt(i); value = b.array(); zk.create(root + "/element", value, Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT_SEQUENTIAL); return true; } /** * Remove first element from the queue. * * @return * @throws KeeperException * @throws InterruptedException */ int consume() throws KeeperException, InterruptedException{ int retvalue = -1; Stat stat = null; // Get the first element available while (true) { synchronized (mutex) { List<String> list = zk.getChildren(root, true); if (list.size() == 0) { System.out.println("Going to wait"); mutex.wait(); } else { Integer min = new Integer(list.get(0).substring(7)); String minNode = list.get(0); for(String s : list){ Integer tempValue = new Integer(s.substring(7)); //System.out.println("Temporary value: " + tempValue); if(tempValue < min) { min = tempValue; minNode = s; } } System.out.println("Temporary value: " + root + "/" + minNode); byte[] b = zk.getData(root + "/" + minNode, false, stat); zk.delete(root + "/" + minNode, 0); ByteBuffer buffer = ByteBuffer.wrap(b); retvalue = buffer.getInt(); return retvalue; } } } } } public static void main(String args[]) { if (args[0].equals("qTest")) queueTest(args); else barrierTest(args); } public static void queueTest(String args[]) { Queue q = new Queue(args[1], "/app1"); System.out.println("Input: " + args[1]); int i; Integer max = new Integer(args[2]); if (args[3].equals("p")) { System.out.println("Producer"); for (i = 0; i < max; i++) try{ q.produce(10 + i); } catch (KeeperException e){ } catch (InterruptedException e){ } } else { System.out.println("Consumer"); for (i = 0; i < max; i++) { try{ int r = q.consume(); System.out.println("Item: " + r); } catch (KeeperException e){ i--; } catch (InterruptedException e){ } } } } public static void barrierTest(String args[]) { Barrier b = new Barrier(args[1], "/b1", new Integer(args[2])); try{ boolean flag = b.enter(); System.out.println("Entered barrier: " + args[2]); if(!flag) System.out.println("Error when entering the barrier"); } catch (KeeperException e){ } catch (InterruptedException e){ } // Generate random integer Random rand = new Random(); int r = rand.nextInt(100); // Loop for rand iterations for (int i = 0; i < r; i++) { try { Thread.sleep(100); } catch (InterruptedException e) { } } try{ b.leave(); } catch (KeeperException e){ } catch (InterruptedException e){ } System.out.println("Left barrier"); } }