1.注册中心作用
从官网摘的图,我们可以简单了解其流程;
- 服务提供者在注册中心进行注册(本质是存放一些关键数据:提供者IP,Port,serviceKey,method,version,group等等信息);
- 服务消费者进行订阅(消费者获取提供者的关键数据);
- 消费者与注册中心通过监听器对数据进行同步(如果服务提供者的信息修改,销毁,新增,监听器来同步);
- 服务消费者调用服务提供者(根据获取的信息,进行调用,简单理解为:给服务提供者Socket端口发送数据,等待响应);
- 调用服务的状态在监控中心进行记录,检测服务质量(记录服务调用信息,多少次,成功,失败等次数,用于其他操作);
如果对Dubbo熟悉的同学,肯定知道经常使用zookeeper,nacos等来作为注册中心;
2.Dubbo-Register-API源码解读
我们看下主要类结构,可以清楚看到Dubbo支持的注册Zookeeper,Redis,Nacos,Etcd等
我们针对每一个类进行分析与导读,API大多作为抽象层,具体的ZookeeperRegister,RedisZookeeper等后续文章导读;主要关心RegisterService,AbstractRegister,FackRegister等
RegisterService
/**
* 注册中心的注册,取消注册,订阅,取消订阅,查询订阅列表等动作定义
*/
public interface RegistryService {
/**
注册URL,注意URL肯定是唯一的,对应存放数据到注册中心
* @param url Registration information , is not allowed to be empty, e.g: dubbo://10.20.153.10/org.apache.dubbo.foo.BarService?version=1.0.0&application=kylin
*/
void register(URL url);
/**
* Unregister
取消注册,对于删除对于的数据
*
* @param url Registration information , is not allowed to be empty, e.g: dubbo://10.20.153.10/org.apache.dubbo.foo.BarService?version=1.0.0&application=kylin
*/
void unregister(URL url);
/**
* Subscribe to eligible registered data and automatically push when the registered data is changed.
订阅注册的数据,如果数据变更,进行通知,Listener就是用来通知数据变更
* @param url Subscription condition, not allowed to be empty, e.g. consumer://10.20.153.10/org.apache.dubbo.foo.BarService?version=1.0.0&application=kylin
* @param listener A listener of the change event, not allowed to be empty
*/
void subscribe(URL url, NotifyListener listener);
/**
* Unsubscribe
取消订阅
* @param url Subscription condition, not allowed to be empty, e.g. consumer://10.20.153.10/org.apache.dubbo.foo.BarService?version=1.0.0&application=kylin
* @param listener A listener of the change event, not allowed to be empty
*/
void unsubscribe(URL url, NotifyListener listener);
/**
查询注册列表,通过url进行条件查询所匹配的所有URL集合。
*/
List<URL> lookup(URL url);
}
Register&Node比较简单不描述
public interface Registry extends Node, RegistryService {
}
public interface Node {
/**
* get url.
*
* @return url.
*/
URL getUrl();
/**
* is available.
*
* @return available.
*/
boolean isAvailable();
/**
* destroy.
*/
void destroy();
}
AbstractRegistry
abstractRegister实现了对基本方法的实现,导读之前我们先了解下一些Dubbo的特点;
- Dubbo会在本地文件存储一些服务注册中心的数据,减少对注册中心的访问压力;
- 我们知道注册的本质是存放数据到注册中心,因此真正的创建数据的逻辑在对应的实现类中如ZookeeperRegister;
基本属性
public abstract class AbstractRegistry implements Registry {
// URL的地址分隔符,在缓存文件中使用,服务提供者的URL分隔
private static final char URL_SEPARATOR = ' ';
// URL地址分隔正则表达式,用于解析文件缓存中服务提供者URL列表
private static final String URL_SPLIT = "\\s+";
// 存储数据到文件的最大尝试次数
private static final int MAX_RETRY_TIMES_SAVE_PROPERTIES = 3;
// Log记录
protected final Logger logger = LoggerFactory.getLogger(getClass());
// 存放配置信息
private final Properties properties = new Properties();
// 缓存调度器
// File cache timing writing
private final ExecutorService registryCacheExecutor = Executors.newFixedThreadPool(1, new NamedThreadFactory("DubboSaveRegistryCache", true));
// 是否异步存放数据
private final boolean syncSaveFile;
// 数据版本号
private final AtomicLong lastCacheChanged = new AtomicLong();
private final AtomicInteger savePropertiesRetryTimes = new AtomicInteger();
// 存放已经注册的URL
private final Set<URL> registered = new ConcurrentHashSet<>();
// 存放已经订阅的URL
private final ConcurrentMap<URL, Set<NotifyListener>> subscribed = new ConcurrentHashMap<>();
// 某个消费者被通知的某一类型的 URL 集合
// 第一个key是消费者的URL,对应的就是哪个消费者。
// value是一个map集合,该map集合的key是分类的意思,例如providers、routes等,value就是被通知的URL集合
private final ConcurrentMap<URL, Map<String, List<URL>>> notified = new ConcurrentHashMap<>();
// 注册中心URL
private URL registryUrl;
//本地存放文件对象
private File file;
*
*
*
}
public AbstractRegistry(URL url)初始化方法;
public AbstractRegistry(URL url) {
// 存储注册中心地址
setUrl(url);
// 获取syncSave值,默认false
syncSaveFile = url.getParameter(REGISTRY_FILESAVE_SYNC_KEY, false);
// 文件的默认路径 user.home+/.dubbo/dubbo-register&application&address 唯一区分
String defaultFilename = System.getProperty("user.home") + "/.dubbo/dubbo-registry-" + url.getParameter(APPLICATION_KEY) + "-" + url.getAddress().replaceAll(":", "-") + ".cache";
String filename = url.getParameter(FILE_KEY, defaultFilename);
File file = null;
// 如果是此一次需要创建文件
if (ConfigUtils.isNotEmpty(filename)) {
file = new File(filename);
if (!file.exists() && file.getParentFile() != null && !file.getParentFile().exists()){
if (!file.getParentFile().mkdirs()) {
throw new IllegalArgumentException("Invalid register cache file");
}
}
}
this.file = file;
// When starting the subscription center,
// we need to read the local cache file for future Registry fault tolerance processing.
// 加载配置
loadProperties();
// 监听数据变化
notify(url.getBackupUrls());
}
LoadProperties()方法
// 加载文件的配置到properties
private void loadProperties() {
if (file != null && file.exists()) {
InputStream in = null;
try {
// 获取文件流
in = new FileInputStream(file);
// 核心方法,加载配置到properties
properties.load(in);
// 如果日志级别可以输出info
if (logger.isInfoEnabled()) {
logger.info("Load registry cache file " + file + ", data: " + properties);
}
} catch (Throwable e) {
logger.warn("Failed to load registry cache file " + file, e);
} finally {
// 关闭流
if (in != null) {
try {
in.close();
} catch (IOException e) {
logger.warn(e.getMessage(), e);
}
}
}
}
}
protected static List<URL> filterEmpty(URL url, List<URL> urls) ;保证
protected static List<URL> filterEmpty(URL url, List<URL> urls) {
// 如果urls为空
if (CollectionUtils.isEmpty(urls)) {
List<URL> result = new ArrayList<>(1);
// 添加一个空协议进去
result.add(url.setProtocol(EMPTY_PROTOCOL));
return result;
}
return urls;
}
public void doSaveProperties(long version);存放数据到本地文件中与loadProperties相反
public void doSaveProperties(long version) {
// 确定版本号最后
if (version < lastCacheChanged.get()) {
return;
}
if (file == null) {
return;
}
// Save
try {
// 获取文件绝对路径.lock文件
File lockfile = new File(file.getAbsolutePath() + ".lock");
// 不存在则创建
if (!lockfile.exists()) {
lockfile.createNewFile();
}
// 获取文件rw权限
try (RandomAccessFile raf = new RandomAccessFile(lockfile, "rw");
// 获取管道
FileChannel channel = raf.getChannel()) {
// 锁住
FileLock lock = channel.tryLock();
if (lock == null) {
throw new IOException("Can not lock the registry cache file " + file.getAbsolutePath() + ", ignore and retry later, maybe multi java process use the file, please config: dubbo.registry.file=xxx.properties");
}
// Save
try {
if (!file.exists()) {
file.createNewFile();
}
try (FileOutputStream outputFile = new FileOutputStream(file)) {
//写入配置
properties.store(outputFile, "Dubbo Registry Cache");
}
} finally {
lock.release();
}
}
} catch (Throwable e) {
savePropertiesRetryTimes.incrementAndGet();
if (savePropertiesRetryTimes.get() >= MAX_RETRY_TIMES_SAVE_PROPERTIES) {
logger.warn("Failed to save registry cache file after retrying " + MAX_RETRY_TIMES_SAVE_PROPERTIES + " times, cause: " + e.getMessage(), e);
savePropertiesRetryTimes.set(0);
return;
}
if (version < lastCacheChanged.get()) {
savePropertiesRetryTimes.set(0);
return;
} else {
registryCacheExecutor.execute(new SaveProperties(lastCacheChanged.incrementAndGet()));
}
logger.warn("Failed to save registry cache file, will retry, cause: " + e.getMessage(), e);
}
}
我们可以看到对应的file和file.lock文件哈
public List<URL> getCacheUrls(URL url) :获取内存缓存的URL
public List<URL> getCacheUrls(URL url) {
for (Map.Entry<Object, Object> entry : properties.entrySet()) {
// 获取分类
String key = (String) entry.getKey();
// 分类value
String value = (String) entry.getValue();
if (key != null && key.length() > 0 && key.equals(url.getServiceKey())
&& (Character.isLetter(key.charAt(0)) || key.charAt(0) == '_')
&& value != null && value.length() > 0) {
// 分割
String[] arr = value.trim().split(URL_SPLIT);
List<URL> urls = new ArrayList<>();
for (String u : arr) {
urls.add(URL.valueOf(u));
}
// 返回截取后的数据
return urls;
}
}
return null;
}
public List<URL> lookup(URL url) 获取消费者订阅的URL列表
@Override
public List<URL> lookup(URL url) {
List<URL> result = new ArrayList<>();
Map<String, List<URL>> notifiedUrls = getNotified().get(url);
// 如果监听的URL列表不为控
if (notifiedUrls != null && notifiedUrls.size() > 0) {
for (List<URL> urls : notifiedUrls.values()) {
for (URL u : urls) {
// 防止加入空协议
if (!EMPTY_PROTOCOL.equals(u.getProtocol())) {
result.add(u);
}
}
}
} else {
// 原子类 避免在获取注册在注册中心的服务url时能够保证是最新的url集合
final AtomicReference<List<URL>> reference = new AtomicReference<>();
NotifyListener listener = reference::set;
subscribe(url, listener); // Subscribe logic guarantees the first notify to return
List<URL> urls = reference.get();
if (CollectionUtils.isNotEmpty(urls)) {
for (URL u : urls) {
if (!EMPTY_PROTOCOL.equals(u.getProtocol())) {
result.add(u);
}
}
}
}
return result;
}
register,unregister,subscribe,unsubscribe比较简单,都是对变量的增加修改,子类会对其重写,调用;
@Override
public void register(URL url) {
if (url == null) {
throw new IllegalArgumentException("register url == null");
}
if (logger.isInfoEnabled()) {
logger.info("Register: " + url);
}
registered.add(url);
}
@Override
public void unregister(URL url) {
if (url == null) {
throw new IllegalArgumentException("unregister url == null");
}
if (logger.isInfoEnabled()) {
logger.info("Unregister: " + url);
}
registered.remove(url);
}
@Override
public void subscribe(URL url, NotifyListener listener) {
if (url == null) {
throw new IllegalArgumentException("subscribe url == null");
}
if (listener == null) {
throw new IllegalArgumentException("subscribe listener == null");
}
if (logger.isInfoEnabled()) {
logger.info("Subscribe: " + url);
}
Set<NotifyListener> listeners = subscribed.computeIfAbsent(url, n -> new ConcurrentHashSet<>());
listeners.add(listener);
}
@Override
public void unsubscribe(URL url, NotifyListener listener) {
if (url == null) {
throw new IllegalArgumentException("unsubscribe url == null");
}
if (listener == null) {
throw new IllegalArgumentException("unsubscribe listener == null");
}
if (logger.isInfoEnabled()) {
logger.info("Unsubscribe: " + url);
}
Set<NotifyListener> listeners = subscribed.get(url);
if (listeners != null) {
listeners.remove(listener);
}
}
recover:注册中心重连,恢复操作;对已经注册,订阅的URL再来一次;
protected void recover() throws Exception {
// register
Set<URL> recoverRegistered = new HashSet<>(getRegistered());
if (!recoverRegistered.isEmpty()) {
if (logger.isInfoEnabled()) {
logger.info("Recover register url " + recoverRegistered);
}
for (URL url : recoverRegistered) {
register(url);
}
}
// subscribe
Map<URL, Set<NotifyListener>> recoverSubscribed = new HashMap<>(getSubscribed());
if (!recoverSubscribed.isEmpty()) {
if (logger.isInfoEnabled()) {
logger.info("Recover subscribe url " + recoverSubscribed.keySet());
}
for (Map.Entry<URL, Set<NotifyListener>> entry : recoverSubscribed.entrySet()) {
URL url = entry.getKey();
for (NotifyListener listener : entry.getValue()) {
subscribe(url, listener);
}
}
}
}
protected void notify(List<URL> urls)
- 发起订阅后,会获取全量数据,此时会调用notify方法。即Registry 获取到了全量数据
- 每次注册中心发生变更时会调用notify方法虽然变化是增量,调用这个方法的调用方,已经进行处理,传入的urls依然是全量的。
- listener.notify,通知监听器,例如,有新的服务提供者启动时,被通知,创建新的 Invoker 对象。
protected void notify(List<URL> urls) {
if (urls == null || urls.isEmpty()) return;
// 遍历订阅URL的监听器集合,通知他们
for (Map.Entry<URL, Set<NotifyListener>> entry : getSubscribed().entrySet()) {
URL url = entry.getKey();
// 匹配
if (!UrlUtils.isMatch(url, urls.get(0))) {
continue;
}
// 遍历监听器集合,通知他们
Set<NotifyListener> listeners = entry.getValue();
if (listeners != null) {
for (NotifyListener listener : listeners) {
try {
notify(url, listener, filterEmpty(url, urls));
} catch (Throwable t) {
logger.error("Failed to notify registry event, urls: " + urls + ", cause: " + t.getMessage(), t);
}
}
}
}
}
protected void notify(URL url, NotifyListener listener, List<URL> urls) {
if (url == null) {
throw new IllegalArgumentException("notify url == null");
}
if (listener == null) {
throw new IllegalArgumentException("notify listener == null");
}
if ((urls == null || urls.isEmpty())
&& !Constants.ANY_VALUE.equals(url.getServiceInterface())) {
logger.warn("Ignore empty notify urls for subscribe url " + url);
return;
}
if (logger.isInfoEnabled()) {
logger.info("Notify urls for subscribe url " + url + ", urls: " + urls);
}
Map<String, List<URL>> result = new HashMap<String, List<URL>>();
// 将urls进行分类
for (URL u : urls) {
if (UrlUtils.isMatch(url, u)) {
// 按照url中key为category对应的值进行分类,如果没有该值,就找key为providers的值进行分类
String category = u.getParameter(Constants.CATEGORY_KEY, Constants.DEFAULT_CATEGORY);
List<URL> categoryList = result.get(category);
if (categoryList == null) {
categoryList = new ArrayList<URL>();
// 分类结果放入result
result.put(category, categoryList);
}
categoryList.add(u);
}
}
if (result.size() == 0) {
return;
}
// 获得某一个消费者被通知的url集合(通知的 URL 变化结果)
Map<String, List<URL>> categoryNotified = notified.get(url);
if (categoryNotified == null) {
// 添加该消费者对应的url
notified.putIfAbsent(url, new ConcurrentHashMap<String, List<URL>>());
categoryNotified = notified.get(url);
}
// 处理通知监听器URL 变化结果
for (Map.Entry<String, List<URL>> entry : result.entrySet()) {
String category = entry.getKey();
List<URL> categoryList = entry.getValue();
categoryNotified.put(category, categoryList);
// 保存到文件
saveProperties(url);
//通知监听器
listener.notify(categoryList);
}
}
saveProperties(),存放数据的方法
private void saveProperties(URL url) {
if (file == null) {
return;
}
try {
StringBuilder buf = new StringBuilder();
// key value形式,key:provider,config value 一长串
Map<String, List<URL>> categoryNotified = notified.get(url);
if (categoryNotified != null) {
for (List<URL> us : categoryNotified.values()) {
for (URL u : us) {
// 用分隔符割开
if (buf.length() > 0) {
buf.append(URL_SEPARATOR);
}
buf.append(u.toFullString());
}
}
}
// properties存放对应的key aluev,后续存入文件用
properties.setProperty(url.getServiceKey(), buf.toString());
// 获取版本号
long version = lastCacheChanged.incrementAndGet();
if (syncSaveFile) {
// 调用核心逻辑,前面的方法
doSaveProperties(version);
} else {
// 如果异步,交给调度器执行一个任务
registryCacheExecutor.execute(new SaveProperties(version));
}
} catch (Throwable t) {
logger.warn(t.getMessage(), t);
}
}
销毁方法:去笑注册和订阅就好了
// 取消所有注册和订阅
@Override
public void destroy() {
if (logger.isInfoEnabled()) {
logger.info("Destroy registry:" + getUrl());
}
Set<URL> destroyRegistered = new HashSet<>(getRegistered());
if (!destroyRegistered.isEmpty()) {
for (URL url : new HashSet<>(getRegistered())) {
if (url.getParameter(DYNAMIC_KEY, true)) {
try {
unregister(url);
if (logger.isInfoEnabled()) {
logger.info("Destroy unregister url " + url);
}
} catch (Throwable t) {
logger.warn("Failed to unregister url " + url + " to registry " + getUrl() + " on destroy, cause: " + t.getMessage(), t);
}
}
}
}
Map<URL, Set<NotifyListener>> destroySubscribed = new HashMap<>(getSubscribed());
if (!destroySubscribed.isEmpty()) {
for (Map.Entry<URL, Set<NotifyListener>> entry : destroySubscribed.entrySet()) {
URL url = entry.getKey();
for (NotifyListener listener : entry.getValue()) {
try {
unsubscribe(url, listener);
if (logger.isInfoEnabled()) {
logger.info("Destroy unsubscribe url " + url);
}
} catch (Throwable t) {
logger.warn("Failed to unsubscribe url " + url + " to registry " + getUrl() + " on destroy, cause: " + t.getMessage(), t);
}
}
}
}
}
异步保存数据到文件的任务类
// 异步保存文件方式
private class SaveProperties implements Runnable {
private long version;
private SaveProperties(long version) {
this.version = version;
}
@Override
public void run() {
doSaveProperties(version);
}
}
好,到此就分析完了,abstractRegister;具体如何存储还得看后续文章,大概抽象导读完毕;
FailbackRegistry
这个类,实现了对我们注册,订阅,取消注册,取消订阅的失败尝试机制实现;
基本属性:失败集合维护,间隔时间等;
public abstract class FailbackRegistry extends AbstractRegistry {
/* retry task map */
private final ConcurrentMap<URL, FailedRegisteredTask> failedRegistered = new ConcurrentHashMap<URL, FailedRegisteredTask>();
private final ConcurrentMap<URL, FailedUnregisteredTask> failedUnregistered = new ConcurrentHashMap<URL, FailedUnregisteredTask>();
private final ConcurrentMap<Holder, FailedSubscribedTask> failedSubscribed = new ConcurrentHashMap<Holder, FailedSubscribedTask>();
private final ConcurrentMap<Holder, FailedUnsubscribedTask> failedUnsubscribed = new ConcurrentHashMap<Holder, FailedUnsubscribedTask>();
private final ConcurrentMap<Holder, FailedNotifiedTask> failedNotified = new ConcurrentHashMap<Holder, FailedNotifiedTask>();
/**
* The time in milliseconds the retryExecutor will wait
*/
private final int retryPeriod;
// Timer for failure retry, regular check if there is a request for failure, and if there is, an unlimited retry
private final HashedWheelTimer retryTimer;
Register方法:我们看下他是如何对父类abstractRegister进行扩展的
// 注册方法,真实
@Override
public void register(URL url) {
if (!acceptable(url)) {
logger.info("URL " + url + " will not be registered to Registry. Registry " + url + " does not accept service of this protocol type.");
return;
}
// 调用父类接口
super.register(url);
/**
* 移除失败注册,失败取消注册
*/
removeFailedRegistered(url);
removeFailedUnregistered(url);
try {
// 核心的交给子类去实现,zookeeper,redis等实现方法不同
doRegister(url);
} catch (Exception e) {
Throwable t = e;
// If the startup detection is opened, the Exception is thrown directly.
boolean check = getUrl().getParameter(Constants.CHECK_KEY, true)
&& url.getParameter(Constants.CHECK_KEY, true)
&& !CONSUMER_PROTOCOL.equals(url.getProtocol());
boolean skipFailback = t instanceof SkipFailbackWrapperException;
if (check || skipFailback) {
if (skipFailback) {
t = t.getCause();
}
throw new IllegalStateException("Failed to register " + url + " to registry " + getUrl().getAddress() + ", cause: " + t.getMessage(), t);
} else {
logger.error("Failed to register " + url + ", waiting for retry, cause: " + t.getMessage(), t);
}
// 执行重试逻辑
addFailedRegistered(url);
}
}
AddFailedRegister(Url)方法:重试注册
private void addFailedRegistered(URL url) {
FailedRegisteredTask oldOne = failedRegistered.get(url);
if (oldOne != null) {
return;
}
// 创建一个任务task,我们关心run就好
FailedRegisteredTask newTask = new FailedRegisteredTask(url, this);
oldOne = failedRegistered.putIfAbsent(url, newTask);
if (oldOne == null) {
// never has a retry task. then start a new task for retry.
retryTimer.newTimeout(newTask, retryPeriod, TimeUnit.MILLISECONDS);
}
}
public final class FailedRegisteredTask extends AbstractRetryTask {
private static final String NAME = "retry register";
public FailedRegisteredTask(URL url, FailbackRegistry registry) {
super(url, registry, NAME);
}
// 重试的本质
@Override
protected void doRetry(URL url, FailbackRegistry registry, Timeout timeout) {
registry.doRegister(url);
registry.removeFailedRegisteredTask(url);
}
}
// AbstractRetryTask的方法,run
@Override
public void run(Timeout timeout) throws Exception {
if (timeout.isCancelled() || timeout.timer().isStop() || isCancel()) {
// other thread cancel this timeout or stop the timer.
return;
}
if (times > retryTimes) {
// reach the most times of retry.
logger.warn("Final failed to execute task " + taskName + ", url: " + url + ", retry " + retryTimes + " times.");
return;
}
if (logger.isInfoEnabled()) {
logger.info(taskName + " : " + url);
}
try {
doRetry(url, registry, timeout);
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to execute task " + taskName + ", url: " + url + ", waiting for again, cause:" + t.getMessage(), t);
// reput this task when catch exception.
reput(timeout, retryPeriod);
}
}
其余比如unregister,subscribe,unsubscribe都是一个套路;
同时我们也看到核心的逻辑交给不同的注册中心实现,留给子类实现;
到此Dubbo-Register-API就分析完了,后续展开ZookeeperRegister等源码,探索真正的注册实现;
总结:
Dubbo-Register-API实现了对基本流程方法的抽象,以及高可用的重试机制;
个人觉得源码值得仔细品味,有很多设计思想,没有讲,可以多探索下;