[Curator] Tree Cache 的使用与分析

Tree Cache

使用zk路径下所有节点数据作为缓存。这个类会监控ZK的一个路径,路径下所有的节点变动,数据变动都会被响应。 还可以通过注册自定义监听器来更细节的控制这些数据变动操作。

1. 关键 API

org.apache.curator.framework.recipes.cache.TreeCache

org.apache.curator.framework.recipes.cache.TreeCacheListener

org.apache.curator.framework.recipes.cache.TreeCacheEvent

org.apache.curator.framework.recipes.cache.ChildData

2. 机制说明

Path Cache类似

  • Builder模式
    • TreeCache属性较多,所以提供了一个TreeCache.Builder
  • Path Cache是一层结构
    • 按cache的name,挂载到path下
  • TreeCache
    • 顾名思义,内部是一个树结构
    • 所以有一个内部类TreeCache.TreeNode封装树节点信息

TreeCache使用一个内部类TreeNode来维护这个一个树结构。并将这个树结构与ZK节点进行了映射。

3. 用法

3.1 创建

public TreeCache(CuratorFramework client,
                         String path,
                         boolean cacheData)
  • cacheData
    • 如果设置true,是否需要缓存数据

3.2 使用

还是一样的套路,在使用前需要调用start();用完之后需要调用close()方法。

随时都可以调用getCurrentData()获取当前缓存的状态和数据。

也可以通过getListenable()获取监听器容器,并在此基础上增加自定义监听器:

public void addListener(NodeCacheListener listener)

不过Path Cache,以及Node Cache不一样的是:

  • 多了一个getCurrentChildren()方法
    • 返回path下多个子节点的缓存数据
    • 封装成一个Map<String,ChildData>返回
    • 没有很精准的进行数据同步
      • 可以当作一份快照使用

4. 错误处理

TreeCache实例自带一个ConnectionStateListener处理链接状态的变化,并处理数据变动情况

5. 源码分析

5.1 类定义

public class TreeCache implements Closeable{}

  • 实现了java.io.Closeable

5.1.1 TreeCache.Builder

org.apache.curator.framework.recipes.cache.TreeCache.Builder

public static final class Builder{}

5.1.2 TreeCache.TreeNode

org.apache.curator.framework.recipes.cache.TreeCache.TreeNode

private final class TreeNode implements Watcher, BackgroundCallback{}

5.2 成员变量

public class TreeCache implements Closeable
{
    private static final Logger LOG = LoggerFactory.getLogger(TreeCache.class);
    private final boolean createParentNodes;
    private final TreeCacheSelector selector;
    
    private static final AtomicReferenceFieldUpdater<TreeNode, NodeState> nodeStateUpdater =
            AtomicReferenceFieldUpdater.newUpdater(TreeNode.class, NodeState.class, "nodeState");

    private static final AtomicReferenceFieldUpdater<TreeNode, ChildData> childDataUpdater =
            AtomicReferenceFieldUpdater.newUpdater(TreeNode.class, ChildData.class, "childData");

    private static final AtomicReferenceFieldUpdater<TreeNode, ConcurrentMap> childrenUpdater =
            AtomicReferenceFieldUpdater.newUpdater(TreeNode.class, ConcurrentMap.class, "children");
            
    private final AtomicLong outstandingOps = new AtomicLong(0);

    private final AtomicBoolean isInitialized = new AtomicBoolean(false);

    private final TreeNode root;
    private final CuratorFramework client;
    private final ExecutorService executorService;
    private final boolean cacheData;
    private final boolean dataIsCompressed;
    private final int maxDepth;
    private final ListenerContainer<TreeCacheListener> listeners = new ListenerContainer<TreeCacheListener>();
    private final ListenerContainer<UnhandledErrorListener> errorListeners = new ListenerContainer<UnhandledErrorListener>();
    private final AtomicReference<TreeState> treeState = new AtomicReference<TreeState>(TreeState.LATENT);

    private final ConnectionStateListener connectionStateListener = new ConnectionStateListener()
    {
        @Override
        public void stateChanged(CuratorFramework client, ConnectionState newState)
        {
            handleStateChange(newState);
        }
    };

    static final ThreadFactory defaultThreadFactory = ThreadUtils.newThreadFactory("TreeCache");
}

可以看出,定义了很多变量。(所以额外的使用了Builder模式)

  • LOG
    • 私有常量
    • 静态内部类也要使用日志
  • createParentNodes
    • 是否需要创建父节点
    • 默认情况下,不会自动创建path的父节点
  • selector
    • org.apache.curator.framework.recipes.cache.TreeCacheSelector
    • 默认使用org.apache.curator.framework.recipes.cache.DefaultTreeCacheSelector
    • 用于区分哪些节点作为缓存使用
  • nodeStateUpdater
    • java.util.concurrent.atomic.AtomicReferenceFieldUpdater
    • 为TreeNode的nodeState属性提供原子操作
  • childDataUpdater
    • java.util.concurrent.atomic.AtomicReferenceFieldUpdater
    • 为TreeNode的childData属性提供原子操作
  • childrenUpdater
    • java.util.concurrent.atomic.AtomicReferenceFieldUpdater
    • 为TreeNode的children属性提供原子操作
  • outstandingOps
    • 记录未完成回调处理的任务数量
  • isInitialized
    • 是否已初始化
    • 是否已发出INITIALIZED事件
  • root
    • 树结构的根节点
    • 也可看作一个哨兵
  • client
  • executorService
    • 线程池
    • 老套路,操作异步处理。类似Path Cache
  • cacheData
    • 是否缓存数据
  • dataIsCompressed
    • 数据是否压缩
  • maxDepth
    • 树结构的最大深度
  • listeners
    • 监听器容器
    • TreeCacheListener缓存监听器
  • errorListeners
    • 监听器容器
    • UnhandledErrorListener监听器单独一个容器
    • TreeCacheListener监听器分开,利于对不同种类的监听器单独管理
  • treeState
    • AtomicReference
    • 内部枚举
      • LATENT
      • STARTED
      • CLOSED
  • defaultThreadFactory
  • connectionStateListener
    • zk链接状态监听器
    • 老套路,回调状态机,异步处理事件
5.2.1 TreeNode
private final class TreeNode implements Watcher, BackgroundCallback
{

    volatile NodeState nodeState = NodeState.PENDING;
    volatile ChildData childData;
    final TreeNode parent;
    final String path;
    volatile ConcurrentMap<String, TreeNode> children;
    final int depth;

    TreeNode(String path, TreeNode parent)
    {
        this.path = path;
        this.parent = parent;
        this.depth = parent == null ? 0 : parent.depth + 1;
    }
}
  • 实现了org.apache.zookeeper.Watcher接口
  • 实现了org.apache.curator.framework.api.BackgroundCallback接口
  • nodeState
    • org.apache.curator.framework.recipes.cache.TreeCache.NodeState
    • 节点状态
      • PENDING 等待处理
      • LIVE 有效
      • DEAD 无效
  • childData
    • org.apache.curator.framework.recipes.cache.ChildData
    • 节点数据
  • parent
    • 父节点
  • path
    • 对应的zk节点路径
  • children
    • java.util.concurrent.ConcurrentMap
    • 子节点列表
  • depth
    • 当前节点所处深度(层级)

5.3 构造器

public TreeCache(CuratorFramework client, String path)
{
    this(client, path, true, false, Integer.MAX_VALUE, Executors.newSingleThreadExecutor(defaultThreadFactory), false, new DefaultTreeCacheSelector());
}

TreeCache(CuratorFramework client, String path, boolean cacheData, boolean dataIsCompressed, int maxDepth, final ExecutorService executorService, boolean createParentNodes, TreeCacheSelector selector)
{
    this.createParentNodes = createParentNodes;
    this.selector = Preconditions.checkNotNull(selector, "selector cannot be null");
    this.root = new TreeNode(validatePath(path), null);
    this.client = Preconditions.checkNotNull(client, "client cannot be null");
    this.cacheData = cacheData;
    this.dataIsCompressed = dataIsCompressed;
    this.maxDepth = maxDepth;
    this.executorService = Preconditions.checkNotNull(executorService, "executorService cannot be null");
}

可以发现:

  • 默认处理线程池是一个单线程线程池
  • 默认缓存节点数据
  • 默认不对数据进行压缩
  • 默认不控制树的深度
  • 默认不创建父节点
  • 使用默认的DefaultTreeCacheSelector
  • 初始化了root节点信息

除了两个构造器,还有Builderbuild方法来创建TreeCache

private final CuratorFramework client;
private final String path;
private boolean cacheData = true;
private boolean dataIsCompressed = false;
private ExecutorService executorService = null;
private int maxDepth = Integer.MAX_VALUE;
private boolean createParentNodes = false;
private TreeCacheSelector selector = new DefaultTreeCacheSelector();

public TreeCache build()
{
    ExecutorService executor = executorService;
    if ( executor == null )
    {
        executor = Executors.newSingleThreadExecutor(defaultThreadFactory);
    }
    return new TreeCache(client, path, cacheData, dataIsCompressed, maxDepth, executor, createParentNodes, selector);
}

创建过程也没有特殊的设定,各属性默认值也都一致

5.4 启动

使用之前需要调用start()方法

public TreeCache start() throws Exception
{
    Preconditions.checkState(treeState.compareAndSet(TreeState.LATENT, TreeState.STARTED), "already started");
    if ( createParentNodes )
    {
        client.createContainers(root.path);
    }
    client.getConnectionStateListenable().addListener(connectionStateListener);
    if ( client.getZookeeperClient().isConnected() )
    {
        root.wasCreated();
    }
    return this;
}
  1. 原子更新状态
  2. 如果需要创建父节点,则创建
  3. 在链接上增加监听器
  4. 如果链接是已连接,则调用root.wasCreated()root节点开始逐个同步数据

org.apache.curator.framework.recipes.cache.TreeCache.TreeNode#wasCreated:

void wasCreated() throws Exception
{
    refresh();
}

private void refresh() throws Exception
{
    if ((depth < maxDepth) && selector.traverseChildren(path))
    {
        outstandingOps.addAndGet(2);
        doRefreshData();
        doRefreshChildren();
    } else {
        refreshData();
    }
}
private void refreshData() throws Exception
{
    outstandingOps.incrementAndGet();
    doRefreshData();
}

  1. 如果深度(树层级)允许,则下探(开始两个任务)
    1. 刷新当前节点数据
    2. 刷新下层节点列表
  2. 如果无需下探,则仅刷新当前结点的数据
  • 每次调用doXxxxx方法,则创建一个任务
    • 使用outstandingOps跟踪任务数量
刷新节点数据
private void doRefreshData() throws Exception
{
    if ( dataIsCompressed )
    {
        client.getData().decompressed().usingWatcher(this).inBackground(this).forPath(path);
    }
    else
    {
        client.getData().usingWatcher(this).inBackground(this).forPath(path);
    }
}
  • 根据是否使用数据压缩,调用不同的获取数据的方法
  • 都使用了回掉处理
    • TreeNode本身已经实现了Watcher, BackgroundCallback两个接口,所以回调/监听对象都是this

回调: org.apache.curator.framework.recipes.cache.TreeCache.TreeNode#processResult:

public void processResult(CuratorFramework client, CuratorEvent event) throws Exception
    {
        LOG.debug("processResult: {}", event);
        Stat newStat = event.getStat();
        switch ( event.getType() )
        {
        case EXISTS:
            Preconditions.checkState(parent == null, "unexpected EXISTS on non-root node");
            if ( event.getResultCode() == KeeperException.Code.OK.intValue() )
            {
                nodeStateUpdater.compareAndSet(this, NodeState.DEAD, NodeState.PENDING);
                wasCreated();
            }
            break;
        case CHILDREN:
            if ( event.getResultCode() == KeeperException.Code.OK.intValue() )
            {
                ChildData oldChildData = childData;
                if ( oldChildData != null && oldChildData.getStat().getMzxid() == newStat.getMzxid() )
                {
                    // Only update stat if mzxid is same, otherwise we might obscure
                    // GET_DATA event updates.
                    childDataUpdater.compareAndSet(this, oldChildData, new ChildData(oldChildData.getPath(), newStat, oldChildData.getData()));
                }

                if ( event.getChildren().isEmpty() )
                {
                    break;
                }

                ConcurrentMap<String, TreeNode> childMap = children;
                if ( childMap == null )
                {
                    childMap = Maps.newConcurrentMap();
                    if ( !childrenUpdater.compareAndSet(this, null, childMap) )
                    {
                        childMap = children;
                    }
                }

                // Present new children in sorted order for test determinism.
                List<String> newChildren = new ArrayList<String>();
                for ( String child : event.getChildren() )
                {
                    if ( !childMap.containsKey(child) && selector.acceptChild(ZKPaths.makePath(path, child)) )
                    {
                        newChildren.add(child);
                    }
                }

                Collections.sort(newChildren);
                for ( String child : newChildren )
                {
                    String fullPath = ZKPaths.makePath(path, child);
                    TreeNode node = new TreeNode(fullPath, this);
                    if ( childMap.putIfAbsent(child, node) == null )
                    {
                        node.wasCreated();
                    }
                }
            }
            else if ( event.getResultCode() == KeeperException.Code.NONODE.intValue() )
            {
                wasDeleted();
            }
            break;
        case GET_DATA:
            if ( event.getResultCode() == KeeperException.Code.OK.intValue() )
            {
                ChildData toPublish = new ChildData(event.getPath(), newStat, event.getData());
                ChildData oldChildData;
                if ( cacheData )
                {
                    oldChildData = childDataUpdater.getAndSet(this, toPublish);
                }
                else
                {
                    oldChildData = childDataUpdater.getAndSet(this, new ChildData(event.getPath(), newStat, null));
                }

                boolean added;
                if (parent == null) {
                    // We're the singleton root.
                    added = nodeStateUpdater.getAndSet(this, NodeState.LIVE) != NodeState.LIVE;
                } else {
                    added = nodeStateUpdater.compareAndSet(this, NodeState.PENDING, NodeState.LIVE);
                    if (!added) {
                        // Ordinary nodes are not allowed to transition from dead -> live;
                        // make sure this isn't a delayed response that came in after death.
                        if (nodeState != NodeState.LIVE) {
                            return;
                        }
                    }
                }

                if ( added )
                {
                    publishEvent(TreeCacheEvent.Type.NODE_ADDED, toPublish);
                }
                else
                {
                    if ( oldChildData == null || oldChildData.getStat().getMzxid() != newStat.getMzxid() )
                    {
                        publishEvent(TreeCacheEvent.Type.NODE_UPDATED, toPublish);
                    }
                }
            }
            else if ( event.getResultCode() == KeeperException.Code.NONODE.intValue() )
            {
                wasDeleted();
            }
            break;
        default:
            // An unknown event, probably an error of some sort like connection loss.
            LOG.info(String.format("Unknown event %s", event));
            // Don't produce an initialized event on error; reconnect can fix this.
            outstandingOps.decrementAndGet();
            return;
        }

        if ( outstandingOps.decrementAndGet() == 0 )
        {
            if ( isInitialized.compareAndSet(false, true) )
            {
                publishEvent(TreeCacheEvent.Type.INITIALIZED);
            }
        }
    }
}
  • 状态机 这里先来看看在启动时获取数据的逻辑:

  • GET_DATA

    1. 如果成功获取了zk节点数据
      1. 构建新的ChildData
      2. 如果需要对数据进行缓存
        1. 使用AtomicReferenceFieldUpdater原子化操作覆盖当前结点的数据
      3. 如果不需要对数据进行缓存
        1. 则使用一个不含数据的ChildData覆盖当前结点的数据
      4. 检查当前结点是否已经加入到缓存中
        • 通过判断节点状态来实现
      5. 如果发现为节点已经DEAD无效了,则只是覆盖更新数据,而再不处理,直接返回
      6. 如果当前结点是第一次被添加到树中
        1. 触发NODE_ADDED事件
      7. 如果已经在树中,则判断数据是否有更新
        1. 如果数据有更新,则触发NODE_UPDATED事件
    2. 如果发现zk节点已经不存在了
      1. 则进行删除书节点操作wasDeleted()
  • 当任务计数器outstandingOps为0时,说明同步任务全部完成

    • 更新isInitialized状态
    • 触发了INITIALIZED事件

org.apache.curator.framework.recipes.cache.TreeCache.TreeNode#wasDeleted:

void wasDeleted() throws Exception
{
    ChildData oldChildData = childDataUpdater.getAndSet(this, null);
    client.clearWatcherReferences(this);
    ConcurrentMap<String, TreeNode> childMap = childrenUpdater.getAndSet(this,null);
    if ( childMap != null )
    {
        ArrayList<TreeNode> childCopy = new ArrayList<TreeNode>(childMap.values());
        childMap.clear();
        for ( TreeNode child : childCopy )
        {
            child.wasDeleted();
        }
    }

    if ( treeState.get() == TreeState.CLOSED )
    {
        return;
    }

    NodeState oldState = nodeStateUpdater.getAndSet(this, NodeState.DEAD);
    if ( oldState == NodeState.LIVE )
    {
        publishEvent(TreeCacheEvent.Type.NODE_REMOVED, oldChildData);
    }

    if ( parent == null )
    {
        // Root node; use an exist query to watch for existence.
        client.checkExists().usingWatcher(this).inBackground(this).forPath(path);
    }
    else
    {
        // Remove from parent if we're currently a child
        ConcurrentMap<String, TreeNode> parentChildMap = parent.children;
        if ( parentChildMap != null )
        {
            parentChildMap.remove(ZKPaths.getNodeFromPath(path), this);
        }
    }
}
  1. 将当前节点数据清空
  2. 清除节点上的监听器
  3. 制空子节点列表
    1. 调用每一个节点的wasDeleted()
    • 树结构从上到下的清理动作
  4. 如果整个树状态都已经关闭,则不需要在处理了
  5. 如果树状态正常,只是清理某个分支则:
    1. 则将当前结点状态更新为DEAD无效
    2. 触发NODE_REMOVED事件
    3. 如果父节点为空(自身就是根节点)
      1. 则对zk节点进行检查(检查树对应的zk节点根节点是否还存在)
    4. 如果父节点不为空(树中的某个分支节点)
      1. 则将自己从父节点的子节点列表中移除
刷新子节点列表
private void doRefreshChildren() throws Exception
{
    client.getChildren().usingWatcher(this).inBackground(this).forPath(path);
}

仍然是一个回调自身processResult状态机的方式。

  • 这里看看CHILDREN的逻辑
    1. 如果以及成功获取了ZK子节点列表
      1. 如果子节点列表没有变化(ZK节点没变化,不是说本地列表没变化)
        1. 覆盖更新当前结点的子节点列表
        • 只是更新了Stat信息,后续逻辑会判断data的相等性,如果这里也覆盖了data部分,则会导致后续逻辑任务数据发生了更新,从而导致不必要的GET_DATA事件
      2. 如果子节点列表是空的,自然不需要再做什么处理了
      3. 开始构建children
        • 考虑到了children的并发
          • 先用原子操作
          • 如果失败,说明有其他线程已经更新了,则使用新值
      4. 对节点列表进行梳理排序后,加入子节点列表
        1. 逐个调用子节点的wasCreated
          • 从上到下的加入一个树分支
    2. 如果发现ZK节点不存在了
      1. 则进行删除书节点操作wasDeleted()
        • 参见上一节

5.5 获取缓存

有两个方式来获取缓存

5.5.1 获取指定节点缓存数据

调用:ChildData getCurrentData(String fullPath)

看看源码:

public ChildData getCurrentData(String fullPath)
{
    TreeNode node = find(fullPath);
    if ( node == null || node.nodeState != NodeState.LIVE )
    {
        return null;
    }
    ChildData result = node.childData;
    // Double-check liveness after retreiving data.
    return node.nodeState == NodeState.LIVE ? result : null;
}
  1. 查找到对应的节点
  2. 获取节点数据
  3. Double check

所以,重点在于节点的查找过程:

private TreeNode find(String findPath)
{
    PathUtils.validatePath(findPath);
    LinkedList<String> rootElements = new LinkedList<String>(ZKPaths.split(root.path));
    LinkedList<String> findElements = new LinkedList<String>(ZKPaths.split(findPath));
    while (!rootElements.isEmpty()) {
        if (findElements.isEmpty()) {
            // Target path shorter than root path
            return null;
        }
        String nextRoot = rootElements.removeFirst();
        String nextFind = findElements.removeFirst();
        if (!nextFind.equals(nextRoot)) {
            // Initial root path does not match
            return null;
        }
    }

    TreeNode current = root;
    while (!findElements.isEmpty()) {
        String nextFind = findElements.removeFirst();
        ConcurrentMap<String, TreeNode> map = current.children;
        if ( map == null )
        {
            return null;
        }
        current = map.get(nextFind);
        if ( current == null )
        {
            return null;
        }
    }
    return current;
}
  1. 分离出缓存节点对应路径
    1. root.pathfindPath的每一层(/分隔)逐一比较
    2. findPath中,去掉root.path的部分
  2. 从root节点开始,按剩余的findPath部分逐一查找
  3. 直到最后匹配的节点
5.5.2 获取一组子节点缓存数据

调用:Map<String, ChildData> getCurrentChildren(String fullPath)

public Map<String, ChildData> getCurrentChildren(String fullPath)
{
    TreeNode node = find(fullPath);
    if ( node == null || node.nodeState != NodeState.LIVE )
    {
        return null;
    }
    ConcurrentMap<String, TreeNode> map = node.children;
    Map<String, ChildData> result;
    if ( map == null )
    {
        result = ImmutableMap.of();
    }
    else
    {
        ImmutableMap.Builder<String, ChildData> builder = ImmutableMap.builder();
        for ( Map.Entry<String, TreeNode> entry : map.entrySet() )
        {
            TreeNode childNode = entry.getValue();
            ChildData childData = childNode.childData;
            // Double-check liveness after retreiving data.
            if ( childData != null && childNode.nodeState == NodeState.LIVE )
            {
                builder.put(entry.getKey(), childData);
            }
        }
        result = builder.build();
    }

    // Double-check liveness after retreiving children.
    return node.nodeState == NodeState.LIVE ? result : null;
}
  1. 查找对应的节点
  2. 逐个获取子节点数据
  • 返回的是一个不可变Map

5.6 关闭缓存

使用完缓存需要调用close()

public void close()
{
    if ( treeState.compareAndSet(TreeState.STARTED, TreeState.CLOSED) )
    {
        client.getConnectionStateListenable().removeListener(connectionStateListener);
        listeners.clear();
        executorService.shutdown();
        try
        {
            root.wasDeleted();
        }
        catch ( Exception e )
        {
            ThreadUtils.checkInterrupted(e);
            handleException(e);
        }
    }
}
  1. 原子化更新状态
  2. 移除链接状态监听器
  3. 清理监听器容器
  4. 关闭线程池
  5. 删除root节点

6. 小结

对比项Path CacheNode CacheTree Cache
缓存数量多个单个多个
使用管理着一组缓存,类似一个CacheManager单个name的缓存有层级关系的CacheManager
机制状态机,事件驱动(对重复操作进行过滤),单线程,任务队列状态机,回调驱动状态机,单线程,事件驱动,使用任务计数器跟踪

转载于:https://my.oschina.net/roccn/blog/956654

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值