文章目录
一、注册中心的作用与模块划分
注册中心有什么作用?同上我们使用的微服务框架需要通过注册中心来管理各个微服务,通过注册中心我们可以实现服务的注册与发现等功能,服务提供者Provider会往注册中心注册服务,而消费者Consumer会从注册中心中订阅相关的服务,并不会订阅全部的服务,同时对于Provider的增加或者下线都可以通过注册中心动态的通知到消费者,整体流程如图所示:
1.1、dubbo-registry模块
在dubbo中,有个主要的模块叫做dubbo-registry。这个模块集成了很多子moudle:
(1)dubbo-registry-api:各种注册中心的抽象,用于其他模块继承实现自己的逻辑,这是最核心的模块。
(2)dubbo-registry-xxx:其他对应的注册中心的实现,如zookeeper、nacos、Redis、etcd等。
二、dubbo-registry-api包源码分析
目录结构:
2.1、RegistryFactory
注册中心接口工厂,采用SPI机制动态拓展
org.apache.dubbo.registry.RegistryFactory 是注册中心工厂类,可以通过SPI机制进行动态扩展。通过源码可以看到getRegistry是通过不同的protocol参数,来选择不同的协议进行注册,例如url.protocol = zookeeper 时,可以获得 ZookeeperRegistryFactory 实现类。
@SPI("dubbo")
public interface RegistryFactory {
/**
* 连接注册中心.
*
* 连接注册中心需处理契约:<br>
* 1. 当设置check=false时表示不检查连接,否则在连接不上时抛出异常。<br>
* 2. 支持URL上的username:password权限认证。<br>
* 3. 支持backup=10.20.153.10备选注册中心集群地址。<br>
* 4. 支持file=registry.cache本地磁盘文件缓存。<br>
* 5. 支持timeout=1000请求超时设置。<br>
* 6. 支持session=60000会话超时或过期设置。<br>
*
* @param url 注册中心地址,不允许为空
* @return 注册中心引用,总不返回空
*/
@Adaptive({"protocol"})
Registry getRegistry(URL url);
}
UML图:
通过uml图我们可以发现,在每个具体的实现类的上层还有个AbstractRegistryFactory类。
com.alibaba.dubbo.registry.support.AbstractRegistryFactory是RegistryFactory的抽象实现类,提供了对Registry的管理工作。
public abstract class AbstractRegistryFactory implements RegistryFactory {
private static final Logger LOGGER = LoggerFactory.getLogger(AbstractRegistryFactory.class);
// 获取注册中心过程中使用的互斥锁
private static final ReentrantLock LOCK = new ReentrantLock();
// 注册中心映射Map<key=注册名称、value=注册中心>
private static final Map<String, Registry> REGISTRIES = new ConcurrentHashMap<String, Registry>();
/**
* 获取到所有Registry
*/
public static Collection<Registry> getRegistries() {
return Collections.unmodifiableCollection(REGISTRIES.values());
}
/**
* 删除所有Registry
*/
public static void destroyAll() {
if (LOGGER.isInfoEnabled()) {
LOGGER.info("Close all registries " + getRegistries());
}
// 删除的时候需要先进行加锁操作
LOCK.lock();
try {
for (Registry registry : getRegistries()) {
try {
// 调用销毁方法
registry.destroy();
} catch (Throwable e) {
LOGGER.error(e.getMessage(), e);
}
}
// 销毁后清空缓存
REGISTRIES.clear();
} finally {
LOCK.unlock();
}
}
/*
* 解析URL获取具体实现的Registry
*/
public Registry getRegistry(URL url) {
url = url.setPath(RegistryService.class.getName())
.addParameter(Constants.INTERFACE_KEY, RegistryService.class.getName())
.removeParameters(Constants.EXPORT_KEY, Constants.REFER_KEY);
String key = url.toServiceString();
// 加锁确保只生成一个Registry实例
LOCK.lock();
try {
// Registry已存在的情况
Registry registry = REGISTRIES.get(key);
if (registry != null) {
return registry;
}
// 不存在则新增
registry = createRegistry(url);
if (registry == null) {
throw new IllegalStateException("Can not create registry " + url);
}
// 将创建的Registry放到缓存中
REGISTRIES.put(key, registry);
return registry;
} finally {
LOCK.unlock();
}
}
/*
* 创建注册中心的模板方法模式,留给子类做实现
*/
protected abstract Registry createRegistry(URL url);
}
2.2、RegistryService
org.apache.dubbo.registry.RegistryService 接口是注册中心模块的服务接口,提供了注册、取消注册、订阅、取消订阅以及查询符合条件的已注册数据,也是注册中心中最为基础的类。
AbstractRegistry 是对注册中心的封装,主要是做了一个缓存的作用,会通过持久化操作缓存注册中心的信息。在远程注册中心不可用的时候,可以采用本地的注册中心来使用
FailbackRegistry 与AbstracrReistry相比,多了几个容器,在执行注册、注销注册、订阅、取消订阅、通知这些操作时,如果操作失败,会将URL和对应的信息加入对应的容器,增加了重试机制。
uml:
主要的几个方法,寻址,注册,订阅,取消注册,取消订阅
public interface RegistryService {
/**
* 注册数据,比如:提供者地址,消费者地址,路由规则,覆盖规则,等数据。
* <p>
* 注册需处理契约:<br>
* 1. 当URL设置了check=false时,注册失败后不报错,在后台定时重试,否则抛出异常。<br>
* 2. 当URL设置了dynamic=false参数,则需持久存储,否则,当注册者出现断电等情况异常退出时,需自动删除。<br>
* 3. 当URL设置了category=routers时,表示分类存储,缺省类别为providers,可按分类部分通知数据。<br>
* 4. 当注册中心重启,网络抖动,不能丢失数据,包括断线自动删除数据。<br>
* 5. 允许URI相同但参数不同的URL并存,不能覆盖。<br>
*
* @param url 注册信息,不允许为空,如:dubbo://10.20.153.10/com.alibaba.foo.BarService?version=1.0.0&application=kylin
*/
void register(URL url);
/**
* 取消注册.
* <p>
* 取消注册需处理契约:<br>
* 1. 如果是dynamic=false的持久存储数据,找不到注册数据,则抛IllegalStateException,否则忽略。<br>
* 2. 按全URL匹配取消注册。<br>
*
* @param url 注册信息,不允许为空,如:dubbo://10.20.153.10/com.alibaba.foo.BarService?version=1.0.0&application=kylin
*/
void unregister(URL url);
/**
* 订阅符合条件的已注册数据,当有注册数据变更时自动推送.
* <p>
* 订阅需处理契约:<br>
* 1. 当URL设置了check=false时,订阅失败后不报错,在后台定时重试。<br>
* 2. 当URL设置了category=routers,只通知指定分类的数据,多个分类用逗号分隔,并允许星号通配,表示订阅所有分类数据。<br>
* 3. 允许以interface,group,version,classifier作为条件查询,如:interface=com.alibaba.foo.BarService&version=1.0.0<br>
* 4. 并且查询条件允许星号通配,订阅所有接口的所有分组的所有版本,或:interface=*&group=*&version=*&classifier=*<br>
* 5. 当注册中心重启,网络抖动,需自动恢复订阅请求。<br>
* 6. 允许URI相同但参数不同的URL并存,不能覆盖。<br>
* 7. 必须阻塞订阅过程,等第一次通知完后再返回。<br>
*
* @param url 订阅条件,不允许为空,如:consumer://10.20.153.10/com.alibaba.foo.BarService?version=1.0.0&application=kylin
* @param listener 变更事件监听器,不允许为空
*/
void subscribe(URL url, NotifyListener listener);
/**
* 取消订阅.
* <p>
* 取消订阅需处理契约:<br>
* 1. 如果没有订阅,直接忽略。<br>
* 2. 按全URL匹配取消订阅。<br>
*
* @param url 订阅条件,不允许为空,如:consumer://10.20.153.10/com.alibaba.foo.BarService?version=1.0.0&application=kylin
* @param listener 变更事件监听器,不允许为空
*/
void unsubscribe(URL url, NotifyListener listener);
/**
* 查询符合条件的已注册数据,与订阅的推模式相对应,这里为拉模式,只返回一次结果。
*
* @param url 查询条件,不允许为空,如:consumer://10.20.153.10/com.alibaba.foo.BarService?version=1.0.0&application=kylin
* @return 已注册信息列表,可能为空,含义同{@link com.alibaba.dubbo.registry.NotifyListener#notify(List<URL>)}的参数。
* @see com.alibaba.dubbo.registry.NotifyListener#notify(List)
*/
List<URL> lookup(URL url);
}
2.3、Registry
每个注册中心都对应一个Registry实例,包括dubbo,zookeeper,redis, nacos等
public interface Registry extends Node, RegistryService {
default void reExportRegister(URL url) {
register(url);
}
default void reExportUnregister(URL url) {
unregister(url);
}
}
UML:
结合上一节介绍过,AbstractRegistry 实现缓存机制,FailbackRegistry 实现失败重试功能。
底层 ZookeeperRegistry等为具体实现类,实现与 ZooKeeper 等注册中心交互的逻辑。
接下去我们具体分析 AbstractRegistry 与 FailbackRegistry 逻辑。
2.3.1、AbstractRegistry
// URL地址分隔符,用于文件缓存中,服务提供者URL分隔
private static final char URL_SEPARATOR = ' ';
// URL地址分隔正则表达式,用于解析文件缓存中服务提供者URL列表
private static final String URL_SPLIT = "\\s+";
protected final Logger logger = LoggerFactory.getLogger(getClass());
// 本地配置缓存
private final Properties properties = new Properties();
// 注册中心缓存写入执行器,固定线程数1(ali不推荐这么写)
private final ExecutorService registryCacheExecutor = Executors.newFixedThreadPool(1, new NamedThreadFactory("DubboSaveRegistryCache", true));
// 是否同步保存文件
private final boolean syncSaveFile;
// 上一次缓存改变数据版本号(原子类)
private final AtomicLong lastCacheChanged = new AtomicLong();
// 已注册的URL集合
private final Set<URL> registered = new ConcurrentHashSet<URL>();
// 已订阅的URL及监听器映射集合
private final ConcurrentMap<URL, Set<NotifyListener>> subscribed = new ConcurrentHashMap<URL, Set<NotifyListener>>();
// 被通知的URL集合
private final ConcurrentMap<URL, Map<String, List<URL>>> notified = new ConcurrentHashMap<URL, Map<String, List<URL>>>();
// 该注册中心的URL
private URL registryUrl;
// 本地缓存文件
private File file;
/*
* 构造方法,注册URL
*/
public AbstractRegistry(URL url) {
// 设置registryUrl
setUrl(url);
// 获取是否同步保存缓存
syncSaveFile = url.getParameter(Constants.REGISTRY_FILESAVE_SYNC_KEY, false);
// 缓存文件路径名称
String filename = url.getParameter(Constants.FILE_KEY, System.getProperty("user.home") + "/.dubbo/dubbo-registry-" + url.getParameter(Constants.APPLICATION_KEY) + "-" + url.getAddress() + ".cache");
File file = null;
// 创建缓存文件
if (ConfigUtils.isNotEmpty(filename)) {
file = new File(filename);
if (!file.exists() && file.getParentFile() != null && !file.getParentFile().exists()) {
if (!file.getParentFile().mkdirs()) {
throw new IllegalArgumentException("Invalid registry store file " + file + ", cause: Failed to create directory " + file.getParentFile() + "!");
}
}
}
this.file = file;
// 加载配置到properties中
loadProperties();
// 通知
notify(url.getBackupUrls());
}
/*
* 查询符合条件的已注册数据,与订阅的推模式相对应,这里为拉模式,只返回一次结果
* RegistryService的override方法
*/
public List<URL> lookup(URL url) {
// 订阅列表
List<URL> result = new ArrayList<URL>();
// 获取通知列表
Map<String, List<URL>> notifiedUrls = getNotified().get(url);
if (notifiedUrls != null && notifiedUrls.size() > 0) {
for (List<URL> urls : notifiedUrls.values()) {
for (URL u : urls) {
if (!Constants.EMPTY_PROTOCOL.equals(u.getProtocol())) {
// 添加到result中
result.add(u);
}
}
}
} else {
// 通知列表为空,创建监听者做一次订阅操作,再获取到订阅列表
final AtomicReference<List<URL>> reference = new AtomicReference<List<URL>>();
NotifyListener listener = new NotifyListener() {
public void notify(List<URL> urls) {
reference.set(urls);
}
};
subscribe(url, listener);
List<URL> urls = reference.get();
if (urls != null && !urls.isEmpty()) {
for (URL u : urls) {
if (!Constants.EMPTY_PROTOCOL.equals(u.getProtocol())) {
result.add(u);
}
}
}
}
return result;
}
/*
* 注册数据,比如:提供者地址,消费者地址,路由规则,覆盖规则,等数据
* RegistryService的override方法
*/
public void register(URL url) {
if (url == null) {
throw new IllegalArgumentException("register url == null");
}
if (logger.isInfoEnabled()) {
logger.info("Register: " + url);
}
// 将URL添加缓存中
registered.add(url);
}
/*
* 取消注册URL
* RegistryService的override方法
*/
public void unregister(URL url) {
if (url == null) {
throw new IllegalArgumentException("unregister url == null");
}
if (logger.isInfoEnabled()) {
logger.info("Unregister: " + url);
}
// 将URL从缓存中移除
registered.remove(url);
}
/*
* 订阅符合条件的已注册数据,当有注册数据变更时自动推送.
*/
public void subscribe(URL url, NotifyListener listener) {
if (url == null) {
throw new IllegalArgumentException("subscribe url == null");
}
if (listener == null) {
throw new IllegalArgumentException("subscribe listener == null");
}
if (logger.isInfoEnabled()) {
logger.info("Subscribe: " + url);
}
Set<NotifyListener> listeners = subscribed.get(url);
if (listeners == null) {
subscribed.putIfAbsent(url, new ConcurrentHashSet<NotifyListener>());
listeners = subscribed.get(url);
}
//添加监听,服务有变化后主动推送
listeners.add(listener);
}
/*
* 取消订阅
*/
public void unsubscribe(URL url, NotifyListener listener) {
if (url == null) {
throw new IllegalArgumentException("unsubscribe url == null");
}
if (listener == null) {
throw new IllegalArgumentException("unsubscribe listener == null");
}
if (logger.isInfoEnabled()) {
logger.info("Unsubscribe: " + url);
}
Set<NotifyListener> listeners = subscribed.get(url);
if (listeners != null) {
//删除监听
listeners.remove(listener);
}
}
/*
* 断线重连恢复
*/
protected void recover() throws Exception {
// 重新建立URL注册关系
Set<URL> recoverRegistered = new HashSet<URL>(getRegistered());
if (!recoverRegistered.isEmpty()) {
if (logger.isInfoEnabled()) {
logger.info("Recover register url " + recoverRegistered);
}
for (URL url : recoverRegistered) {
register(url);
}
}
// 重新建立订阅关系
Map<URL, Set<NotifyListener>> recoverSubscribed = new HashMap<URL, Set<NotifyListener>>(getSubscribed());
if (!recoverSubscribed.isEmpty()) {
if (logger.isInfoEnabled()) {
logger.info("Recover subscribe url " + recoverSubscribed.keySet());
}
for (Map.Entry<URL, Set<NotifyListener>> entry : recoverSubscribed.entrySet()) {
URL url = entry.getKey();
for (NotifyListener listener : entry.getValue()) {
subscribe(url, listener);
}
}
}
}
/*
* 通知所有URL的所有监听者出现变更
*/
protected void notify(List<URL> urls) {
if (urls == null || urls.isEmpty()) return;
for (Map.Entry<URL, Set<NotifyListener>> entry : getSubscribed().entrySet()) {
URL url = entry.getKey();
if (!UrlUtils.isMatch(url, urls.get(0))) {
continue;
}
// 获取到监听者集合
Set<NotifyListener> listeners = entry.getValue();
if (listeners != null) {
for (NotifyListener listener : listeners) {
try {
// 通知该url的该listener下urls的变更
notify(url, listener, filterEmpty(url, urls));
} catch (Throwable t) {
logger.error("Failed to notify registry event, urls: " + urls + ", cause: " + t.getMessage(), t);
}
}
}
}
}
// 对特定的url做全量的变更通知
protected void notify(URL url, NotifyListener listener, List<URL> urls) {
if (url == null) {
throw new IllegalArgumentException("notify url == null");
}
if (listener == null) {
throw new IllegalArgumentException("notify listener == null");
}
if ((urls == null || urls.isEmpty())
&& !Constants.ANY_VALUE.equals(url.getServiceInterface())) {
logger.warn("Ignore empty notify urls for subscribe url " + url);
return;
}
if (logger.isInfoEnabled()) {
logger.info("Notify urls for subscribe url " + url + ", urls: " + urls);
}
Map<String, List<URL>> result = new HashMap<String, List<URL>>();
for (URL u : urls) {
if (UrlUtils.isMatch(url, u)) {
String category = u.getParameter(Constants.CATEGORY_KEY, Constants.DEFAULT_CATEGORY);
List<URL> categoryList = result.get(category);
if (categoryList == null) {
categoryList = new ArrayList<URL>();
result.put(category, categoryList);
}
categoryList.add(u);
}
}
if (result.size() == 0) {
return;
}
Map<String, List<URL>> categoryNotified = notified.get(url);
if (categoryNotified == null) {
notified.putIfAbsent(url, new ConcurrentHashMap<String, List<URL>>());
categoryNotified = notified.get(url);
}
for (Map.Entry<String, List<URL>> entry : result.entrySet()) {
String category = entry.getKey();
List<URL> categoryList = entry.getValue();
categoryNotified.put(category, categoryList);
saveProperties(url);
listener.notify(categoryList);
}
}
/**
* 保存配置
*/
private void saveProperties(URL url) {
if (file == null) {
return;
}
try {
StringBuilder buf = new StringBuilder();
Map<String, List<URL>> categoryNotified = notified.get(url);
if (categoryNotified != null) {
for (List<URL> us : categoryNotified.values()) {
for (URL u : us) {
if (buf.length() > 0) {
buf.append(URL_SEPARATOR);
}
buf.append(u.toFullString());
}
}
}
properties.setProperty(url.getServiceKey(), buf.toString());
long version = lastCacheChanged.incrementAndGet();
if (syncSaveFile) {
doSaveProperties(version);
} else {
registryCacheExecutor.execute(new SaveProperties(version));
}
} catch (Throwable t) {
logger.warn(t.getMessage(), t);
}
}
/*
* 关闭dubbo时做销毁
*/
public void destroy() {
if (logger.isInfoEnabled()) {
logger.info("Destroy registry:" + getUrl());
}
// 取消全部登记URL的登记
Set<URL> destroyRegistered = new HashSet<URL>(getRegistered());
if (!destroyRegistered.isEmpty()) {
for (URL url : new HashSet<URL>(getRegistered())) {
if (url.getParameter(Constants.DYNAMIC_KEY, true)) {
try {
unregister(url);
if (logger.isInfoEnabled()) {
logger.info("Destroy unregister url " + url);
}
} catch (Throwable t) {
logger.warn("Failed to unregister url " + url + " to registry " + getUrl() + " on destroy, cause: " + t.getMessage(), t);
}
}
}
}
// 取消全部订阅者的订阅
Map<URL, Set<NotifyListener>> destroySubscribed = new HashMap<URL, Set<NotifyListener>>(getSubscribed());
if (!destroySubscribed.isEmpty()) {
for (Map.Entry<URL, Set<NotifyListener>> entry : destroySubscribed.entrySet()) {
URL url = entry.getKey();
for (NotifyListener listener : entry.getValue()) {
try {
unsubscribe(url, listener);
if (logger.isInfoEnabled()) {
logger.info("Destroy unsubscribe url " + url);
}
} catch (Throwable t) {
logger.warn("Failed to unsubscribe url " + url + " to registry " + getUrl() + " on destroy, cause: " + t.getMessage(), t);
}
}
}
}
}
2.3.2、FailbackRegistry
// Scheduled executor service
private final ScheduledExecutorService retryExecutor = Executors.newScheduledThreadPool(1, new NamedThreadFactory("DubboRegistryFailedRetryTimer", true));
/**
* 失败重试定时器,定时检查是否有请求失败,如有,无限次重试
*/
private final ScheduledFuture<?> retryFuture;
/**
* 失败发起注册失败的 URL 集合
*/
private final Set<URL> failedRegistered = new ConcurrentHashSet<URL>();
/**
* 失败取消注册失败的 URL 集合
*/
private final Set<URL> failedUnregistered = new ConcurrentHashSet<URL>();
/**
* 失败发起订阅失败的监听器集合
*/
private final ConcurrentMap<URL, Set<NotifyListener>> failedSubscribed = new ConcurrentHashMap<URL, Set<NotifyListener>>();
/**
* 失败取消订阅失败的监听器集合
*/
private final ConcurrentMap<URL, Set<NotifyListener>> failedUnsubscribed = new ConcurrentHashMap<URL, Set<NotifyListener>>();
/**
* 失败通知通知的 URL 集合
*/
private final ConcurrentMap<URL, Map<NotifyListener, List<URL>>> failedNotified = new ConcurrentHashMap<URL, Map<NotifyListener, List<URL>>>();
/**
* 是否销毁
*/
private AtomicBoolean destroyed = new AtomicBoolean(false);
public FailbackRegistry(URL url) {
super(url);
// 重试频率,单位:毫秒
int retryPeriod = url.getParameter(Constants.REGISTRY_RETRY_PERIOD_KEY, Constants.DEFAULT_REGISTRY_RETRY_PERIOD);
// 创建失败重试定时器
this.retryFuture = retryExecutor.scheduleWithFixedDelay(new Runnable() {
public void run() {
// Check and connect to the registry
try {
retry();
} catch (Throwable t) { // Defensive fault tolerance
logger.error("Unexpected error occur at failed retry, cause: " + t.getMessage(), t);
}
}
}, retryPeriod, retryPeriod, TimeUnit.MILLISECONDS);
}
/**
* 添加到 `failedSubscribed`
*
* @param url
* @param listener
*/
private void addFailedSubscribed(URL url, NotifyListener listener) {
Set<NotifyListener> listeners = failedSubscribed.get(url);
if (listeners == null) {
failedSubscribed.putIfAbsent(url, new ConcurrentHashSet<NotifyListener>());
listeners = failedSubscribed.get(url);
}
listeners.add(listener);
}
/**
* 移除出 `failedSubscribed` `failedUnsubscribed` `failedNotified`
*
* @param url URL
* @param listener 监听器
*/
private void removeFailedSubscribed(URL url, NotifyListener listener) {
// 移除出 `failedSubscribed`
Set<NotifyListener> listeners = failedSubscribed.get(url);
if (listeners != null) {
listeners.remove(listener);
}
// 移除出 `failedUnsubscribed`
listeners = failedUnsubscribed.get(url);
if (listeners != null) {
listeners.remove(listener);
}
// 移除出 `failedNotified`
Map<NotifyListener, List<URL>> notified = failedNotified.get(url);
if (notified != null) {
notified.remove(listener);
}
}
@Override
public void register(URL url) {
// 已销毁,跳过
if (destroyed.get()){
return;
}
// 添加到 `registered` 变量
super.register(url);
// 移除出 `failedRegistered` `failedUnregistered` 变量
failedRegistered.remove(url);
failedUnregistered.remove(url);
// 向注册中心发送注册请求
try {
// Sending a registration request to the server side
doRegister(url);
} catch (Exception e) {
Throwable t = e;
// 如果开启了启动时检测,则直接抛出异常
// If the startup detection is opened, the Exception is thrown directly.
boolean check = getUrl().getParameter(Constants.CHECK_KEY, true)
&& url.getParameter(Constants.CHECK_KEY, true)
&& !Constants.CONSUMER_PROTOCOL.equals(url.getProtocol()); // 非消费者。消费者会在 `ReferenceConfig#createProxy(...)` 方法中,调用 `Invoker#avalible()` 方法,进行检查。
boolean skipFailback = t instanceof SkipFailbackWrapperException;
if (check || skipFailback) {
if (skipFailback) {
t = t.getCause();
}
throw new IllegalStateException("Failed to register " + url + " to registry " + getUrl().getAddress() + ", cause: " + t.getMessage(), t);
} else {
logger.error("Failed to register " + url + ", waiting for retry, cause: " + t.getMessage(), t);
}
// 将失败的注册请求记录到 `failedRegistered`,定时重试
// Record a failed registration request to a failed list, retry regularly
failedRegistered.add(url);
}
}
@Override
public void unregister(URL url) {
// 已销毁,跳过
if (destroyed.get()){
return;
}
// 移除出 `registered` 变量
super.unregister(url);
// 移除出 `failedRegistered` `failedUnregistered` 变量
failedRegistered.remove(url);
failedUnregistered.remove(url);
// 向注册中心发送取消注册请求
try {
// Sending a cancellation request to the server side
doUnregister(url);
} catch (Exception e) {
Throwable t = e;
// 如果开启了启动时检测,则直接抛出异常
// If the startup detection is opened, the Exception is thrown directly.
boolean check = getUrl().getParameter(Constants.CHECK_KEY, true)
&& url.getParameter(Constants.CHECK_KEY, true)
&& !Constants.CONSUMER_PROTOCOL.equals(url.getProtocol());
boolean skipFailback = t instanceof SkipFailbackWrapperException;
if (check || skipFailback) {
if (skipFailback) {
t = t.getCause();
}
throw new IllegalStateException("Failed to unregister " + url + " to registry " + getUrl().getAddress() + ", cause: " + t.getMessage(), t);
} else {
logger.error("Failed to uregister " + url + ", waiting for retry, cause: " + t.getMessage(), t);
}
// 将失败的取消注册请求记录到 `failedUnregistered`,定时重试
// Record a failed registration request to a failed list, retry regularly
failedUnregistered.add(url);
}
}
@Override
public void subscribe(URL url, NotifyListener listener) {
// 已销毁,跳过
if (destroyed.get()){
return;
}
// 移除出 `subscribed` 变量
super.subscribe(url, listener);
// 移除出 `failedSubscribed` `failedUnsubscribed` `failedNotified`
removeFailedSubscribed(url, listener);
// 向注册中心发送订阅请求
try {
// Sending a subscription request to the server side
doSubscribe(url, listener);
} catch (Exception e) {
Throwable t = e;
// 如果有缓存的 URL 集合,进行通知。后续订阅成功后,会使用最新的 URL 集合,进行通知。
List<URL> urls = getCacheUrls(url);
if (urls != null && !urls.isEmpty()) {
notify(url, listener, urls);
logger.error("Failed to subscribe " + url + ", Using cached list: " + urls + " from cache file: " + getUrl().getParameter(Constants.FILE_KEY, System.getProperty("user.home") + "/dubbo-registry-" + url.getHost() + ".cache") + ", cause: " + t.getMessage(), t);
} else {
// 如果开启了启动时检测,则直接抛出异常
// If the startup detection is opened, the Exception is thrown directly.
boolean check = getUrl().getParameter(Constants.CHECK_KEY, true)
&& url.getParameter(Constants.CHECK_KEY, true);
boolean skipFailback = t instanceof SkipFailbackWrapperException;
if (check || skipFailback) {
if (skipFailback) {
t = t.getCause();
}
throw new IllegalStateException("Failed to subscribe " + url + ", cause: " + t.getMessage(), t);
} else {
logger.error("Failed to subscribe " + url + ", waiting for retry, cause: " + t.getMessage(), t);
}
}
// 将失败的订阅请求记录到 `failedSubscribed`,定时重试
// Record a failed registration request to a failed list, retry regularly
addFailedSubscribed(url, listener);
}
}
@Override
public void unsubscribe(URL url, NotifyListener listener) {
// 已销毁,跳过
if (destroyed.get()){
return;
}
// 移除出 `unsubscribed` 变量
super.unsubscribe(url, listener);
// 移除出 `failedSubscribed` `failedUnsubscribed` `failedNotified`
removeFailedSubscribed(url, listener);
// 向注册中心发送取消订阅请求
try {
// Sending a canceling subscription request to the server side
doUnsubscribe(url, listener);
} catch (Exception e) {
Throwable t = e;
// 如果开启了启动时检测,则直接抛出异常
// If the startup detection is opened, the Exception is thrown directly.
boolean check = getUrl().getParameter(Constants.CHECK_KEY, true)
&& url.getParameter(Constants.CHECK_KEY, true);
boolean skipFailback = t instanceof SkipFailbackWrapperException;
if (check || skipFailback) {
if (skipFailback) {
t = t.getCause();
}
throw new IllegalStateException("Failed to unsubscribe " + url + " to registry " + getUrl().getAddress() + ", cause: " + t.getMessage(), t);
} else {
logger.error("Failed to unsubscribe " + url + ", waiting for retry, cause: " + t.getMessage(), t);
}
// 将失败的订阅请求记录到 `failedUnsubscribed`,定时重试
// Record a failed registration request to a failed list, retry regularly
Set<NotifyListener> listeners = failedUnsubscribed.get(url);
if (listeners == null) {
failedUnsubscribed.putIfAbsent(url, new ConcurrentHashSet<NotifyListener>());
listeners = failedUnsubscribed.get(url);
}
listeners.add(listener);
}
}
@Override
protected void notify(URL url, NotifyListener listener, List<URL> urls) {
if (url == null) {
throw new IllegalArgumentException("notify url == null");
}
if (listener == null) {
throw new IllegalArgumentException("notify listener == null");
}
// 通知监听器
try {
doNotify(url, listener, urls);
} catch (Exception t) {
// 将失败的通知记录到 `failedNotified`,定时重试
// Record a failed registration request to a failed list, retry regularly
Map<NotifyListener, List<URL>> listeners = failedNotified.get(url);
if (listeners == null) {
failedNotified.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, List<URL>>());
listeners = failedNotified.get(url);
}
listeners.put(listener, urls);
logger.error("Failed to notify for subscribe " + url + ", waiting for retry, cause: " + t.getMessage(), t);
}
}
protected void doNotify(URL url, NotifyListener listener, List<URL> urls) {
super.notify(url, listener, urls);
}
@Override
protected void recover() throws Exception {
// register 恢复注册,添加到 `failedRegistered` ,定时重试
Set<URL> recoverRegistered = new HashSet<URL>(getRegistered());
if (!recoverRegistered.isEmpty()) {
if (logger.isInfoEnabled()) {
logger.info("Recover register url " + recoverRegistered);
}
for (URL url : recoverRegistered) {
failedRegistered.add(url);
}
}
// subscribe 恢复订阅,添加到 `failedSubscribed` ,定时重试
Map<URL, Set<NotifyListener>> recoverSubscribed = new HashMap<URL, Set<NotifyListener>>(getSubscribed());
if (!recoverSubscribed.isEmpty()) {
if (logger.isInfoEnabled()) {
logger.info("Recover subscribe url " + recoverSubscribed.keySet());
}
for (Map.Entry<URL, Set<NotifyListener>> entry : recoverSubscribed.entrySet()) {
URL url = entry.getKey();
for (NotifyListener listener : entry.getValue()) {
addFailedSubscribed(url, listener);
}
}
}
}
/**
* 重试
*/
protected void retry() {
// 重试执行注册
if (!failedRegistered.isEmpty()) {
Set<URL> failed = new HashSet<URL>(failedRegistered); // 避免并发冲突
if (failed.size() > 0) {
if (logger.isInfoEnabled()) {
logger.info("Retry register " + failed);
}
try {
for (URL url : failed) {
try {
// 执行注册
doRegister(url);
// 移除出 `failedRegistered`
failedRegistered.remove(url);
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to retry register " + failed + ", waiting for again, cause: " + t.getMessage(), t);
}
}
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to retry register " + failed + ", waiting for again, cause: " + t.getMessage(), t);
}
}
}
// 重试执行取消注册
if (!failedUnregistered.isEmpty()) {
Set<URL> failed = new HashSet<URL>(failedUnregistered); // 避免并发冲突
if (!failed.isEmpty()) {
if (logger.isInfoEnabled()) {
logger.info("Retry unregister " + failed);
}
try {
for (URL url : failed) {
try {
// 执行取消注册
doUnregister(url);
// 移除出 `failedUnregistered`
failedUnregistered.remove(url);
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to retry unregister " + failed + ", waiting for again, cause: " + t.getMessage(), t);
}
}
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to retry unregister " + failed + ", waiting for again, cause: " + t.getMessage(), t);
}
}
}
// 重试执行注册
if (!failedSubscribed.isEmpty()) {
Map<URL, Set<NotifyListener>> failed = new HashMap<URL, Set<NotifyListener>>(failedSubscribed); // 避免并发冲突
for (Map.Entry<URL, Set<NotifyListener>> entry : new HashMap<URL, Set<NotifyListener>>(failed).entrySet()) {
if (entry.getValue() == null || entry.getValue().size() == 0) {
failed.remove(entry.getKey());
}
}
if (failed.size() > 0) {
if (logger.isInfoEnabled()) {
logger.info("Retry subscribe " + failed);
}
try {
for (Map.Entry<URL, Set<NotifyListener>> entry : failed.entrySet()) {
URL url = entry.getKey();
Set<NotifyListener> listeners = entry.getValue();
for (NotifyListener listener : listeners) {
try {
// 执行注册
doSubscribe(url, listener);
// 移除出监听器
listeners.remove(listener);
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to retry subscribe " + failed + ", waiting for again, cause: " + t.getMessage(), t);
}
}
}
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to retry subscribe " + failed + ", waiting for again, cause: " + t.getMessage(), t);
}
}
}
// 重试执行取消注册
if (!failedUnsubscribed.isEmpty()) {
Map<URL, Set<NotifyListener>> failed = new HashMap<URL, Set<NotifyListener>>(failedUnsubscribed);
for (Map.Entry<URL, Set<NotifyListener>> entry : new HashMap<URL, Set<NotifyListener>>(failed).entrySet()) {
if (entry.getValue() == null || entry.getValue().isEmpty()) {
failed.remove(entry.getKey());
}
}
if (failed.size() > 0) {
if (logger.isInfoEnabled()) {
logger.info("Retry unsubscribe " + failed);
}
try {
for (Map.Entry<URL, Set<NotifyListener>> entry : failed.entrySet()) {
URL url = entry.getKey();
Set<NotifyListener> listeners = entry.getValue();
for (NotifyListener listener : listeners) {
try {
// 执行取消注册
doUnsubscribe(url, listener);
// 移除出监听器
listeners.remove(listener);
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to retry unsubscribe " + failed + ", waiting for again, cause: " + t.getMessage(), t);
}
}
}
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to retry unsubscribe " + failed + ", waiting for again, cause: " + t.getMessage(), t);
}
}
}
// 重试执行通知监听器
if (!failedNotified.isEmpty()) {
Map<URL, Map<NotifyListener, List<URL>>> failed = new HashMap<URL, Map<NotifyListener, List<URL>>>(failedNotified);
for (Map.Entry<URL, Map<NotifyListener, List<URL>>> entry : new HashMap<URL, Map<NotifyListener, List<URL>>>(failed).entrySet()) {
if (entry.getValue() == null || entry.getValue().size() == 0) {
failed.remove(entry.getKey());
}
}
if (failed.size() > 0) {
if (logger.isInfoEnabled()) {
logger.info("Retry notify " + failed);
}
try {
for (Map<NotifyListener, List<URL>> values : failed.values()) {
for (Map.Entry<NotifyListener, List<URL>> entry : values.entrySet()) {
try {
NotifyListener listener = entry.getKey();
List<URL> urls = entry.getValue();
// 通知监听器
listener.notify(urls);
// 移除出监听器
values.remove(listener);
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to retry notify " + failed + ", waiting for again, cause: " + t.getMessage(), t);
}
}
}
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to retry notify " + failed + ", waiting for again, cause: " + t.getMessage(), t);
}
}
}
}
三、zk注册中心
Zookeeper注册中心的实现位于dubbo-registry模块下的dubbo-registry-zookeeper子模块:
首先要从工厂创建Registry:
ZookeeperRegistryFactory
public class ZookeeperRegistryFactory extends AbstractRegistryFactory {
private ZookeeperTransporter zookeeperTransporter;
/**
* Invisible injection of zookeeper client via IOC/SPI
* @param zookeeperTransporter
*/
public void setZookeeperTransporter(ZookeeperTransporter zookeeperTransporter) {
this.zookeeperTransporter = zookeeperTransporter;
}
/**
* 创建ZookeeperRegistry
*/
@Override
public Registry createRegistry(URL url) {
return new ZookeeperRegistry(url, zookeeperTransporter);
}
}
3.1、ZookeeperRegistry
com.alibaba.dubbo.registry.zookeeper.ZookeeperRegistry也就是真正的zk注册中心实现类,继承于FailbackRegistry
// logger
private final static Logger logger = LoggerFactory.getLogger(ZookeeperRegistry.class);
// 默认zk端口号
private final static int DEFAULT_ZOOKEEPER_PORT = 2181;
// 默认zk根节点
private final static String DEFAULT_ROOT = "dubbo";
// zk根节点
private final String root;
// Service接口全名集合
private final Set<String> anyServices = new ConcurrentHashSet<String>();
//监听器集合
private final ConcurrentMap<URL, ConcurrentMap<NotifyListener, ChildListener>> zkListeners = new ConcurrentHashMap<URL, ConcurrentMap<NotifyListener, ChildListener>>();
// zk客户端
private final ZookeeperClient zkClient;
/*
* 构造器
*/
public ZookeeperRegistry(URL url, ZookeeperTransporter zookeeperTransporter) {
//调用父类构造器,详情查看2.3节
super(url);
if (url.isAnyHost()) {
throw new IllegalStateException("registry address == null");
}
// zk根结点
String group = url.getParameter(Constants.GROUP_KEY, DEFAULT_ROOT);
if (!group.startsWith(Constants.PATH_SEPARATOR)) {
group = Constants.PATH_SEPARATOR + group;
}
// 设置zk根结点
this.root = group;
// 初始化zkClient
zkClient = zookeeperTransporter.connect(url);
zkClient.addStateListener(new StateListener() {
public void stateChanged(int state) {
if (state == RECONNECTED) {
try {
// 断线重连zk时恢复自身数据
recover();
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
}
}
});
}
/*
* 通过url注册zk
*/
protected void doRegister(URL url) {
try {
// 创建Root+Service+Type+Url的zk节点
zkClient.create(toUrlPath(url), url.getParameter(Constants.DYNAMIC_KEY, true));
} catch (Throwable e) {
throw new RpcException("Failed to register " + url + " to zookeeper " + getUrl() + ", cause: " + e.getMessage(), e);
}
}
/*
* 获取到Root + Service + Type + URL
*/
private String toUrlPath(URL url) {
return toCategoryPath(url) + Constants.PATH_SEPARATOR + URL.encode(url.toFullString());
}
/*
* 取消注册,并去zk删除节点
*/
protected void doUnregister(URL url) {
try {
zkClient.delete(toUrlPath(url));
} catch (Throwable e) {
throw new RpcException("Failed to unregister " + url + " to zookeeper " + getUrl() + ", cause: " + e.getMessage(), e);
}
}
/**
* 订阅服务
*/
@Override
protected void doSubscribe(final URL url, final NotifyListener listener) {
try {
if (Constants.ANY_VALUE.equals(url.getServiceInterface())) {
String root = toRootPath();
// 获得url对应的监听器集合
ConcurrentMap<NotifyListener, ChildListener> listeners = zkListeners.get(url);
//创建监听器
if (listeners == null) {
zkListeners.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, ChildListener>());
listeners = zkListeners.get(url);
listeners.putIfAbsent(listener, new ChildListener() public void childChanged(String parentPath, List<String> currentChilds) { for (String child : currentChilds) {
child = URL.decode(child);
if (!anyServices.contains(child)) {
anyServices.add(child);
subscribe(url.setPath(child).addParameters(Constants.INTERFACE_KEY, child,
Constants.CHECK_KEY, String.valueOf(false)), listener);
}
}
}
});
zkListener = listeners.get(listener);
}
// 创建 root 永久节点。
zkClient.create(root, false);
// 订阅child节点
List<String> services = zkClient.addChildListener(root, zkListener);
// 订阅所有service的节点
if (services != null && !services.isEmpty()) {
for (String service : services) {
service = URL.decode(service);
anyServices.add(service);
subscribe(url.setPath(service).addParameters(Constants.INTERFACE_KEY, service,
Constants.CHECK_KEY, String.valueOf(false)), listener);
}
}
// 处理指定 Service 层的发起订阅,例如服务消费者的订阅
} else {
// 子节点数据数组
List<URL> urls = new ArrayList<URL>();
for (String path : toCategoriesPath(url)) {
// 获得 url 对应的监听器集合
ConcurrentMap<NotifyListener, ChildListener> listeners = zkListeners.get(url);
//创建监听器
if (listeners == null) {
zkListeners.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, ChildListener>());
listeners = zkListeners.get(url);
}
// 获取当前监听的子节点
ChildListener zkListener = listeners.get(listener);
if (zkListener == null) {
listeners.putIfAbsent(listener, new ChildListener() {
public void childChanged(String parentPath, List<String> currentChilds) {
// 如果子节点有变化则通过notify通知订阅的目标
ZookeeperRegistry.this.notify(url, listener, toUrlsWithEmpty(url, parentPath, currentChilds));
}
});
zkListener = listeners.get(listener);
}
//创建节点
zkClient.create(path, false);
// 添加监听器
List<String> children = zkClient.addChildListener(path, zkListener);
// 添加到urls 中
if (children != null) {
urls.addAll(toUrlsWithEmpty(url, path, children));
}
}
// 调用notify方法
notify(url, listener, urls);
}
} catch (Throwable e) {
throw new RpcException("Failed to subscribe " + url + " to zookeeper " + getUrl() + ", cause: " + e.getMessage(), e);
}
}
/**
* 取消订阅
*/
@Override
protected void doUnsubscribe(URL url, NotifyListener listener) {
ConcurrentMap<NotifyListener, ChildListener> listeners = zkListeners.get(url);
if (listeners != null) {
ChildListener zkListener = listeners.get(listener);
if (zkListener != null) {
// 删除订阅的监听
zkClient.removeChildListener(toUrlPath(url), zkListener);
}
}
}