1 服务端
1.1 数据初始化
在初始化PersistService后,会对数据源进行初始化:
@PostConstruct
public void initDataSource() throws Exception {
// 读取jdbc.properties配置, 加载数据源
Properties props = ResourceUtils.getResourceAsProperties("jdbc.properties");
BasicDataSource ds = new BasicDataSource();
ds.setDriverClassName(JDBC_DRIVER_NAME);
ds.setUrl(ensurePropValueNotNull(props.getProperty("db.url")));
ds.setUsername(ensurePropValueNotNull(props.getProperty("db.user")));
ds.setPassword(ensurePropValueNotNull(props.getProperty("db.password")));
ds.setInitialSize(Integer.parseInt(ensurePropValueNotNull(props.getProperty("db.initialSize"))));
ds.setMaxActive(Integer.parseInt(ensurePropValueNotNull(props.getProperty("db.maxActive"))));
ds.setMaxIdle(Integer.parseInt(ensurePropValueNotNull(props.getProperty("db.maxIdle"))));
ds.setMaxWait(Long.parseLong(ensurePropValueNotNull(props.getProperty("db.maxWait"))));
ds.setPoolPreparedStatements(Boolean.parseBoolean(ensurePropValueNotNull(props
.getProperty("db.poolPreparedStatements"))));
this.jt = new JdbcTemplate();
this.jt.setDataSource(ds);
// 设置最大记录数,防止内存膨胀
this.jt.setMaxRows(MAX_ROWS);
// 设置JDBC执行超时时间
this.jt.setQueryTimeout(QUERY_TIMEOUT);
}
在TimerTaskService中,会初始化一个定时任务,用于每隔600秒从数据库查询最新数据,查询数据后,首先更新缓存,然后写入磁盘:
@PostConstruct
public void init() {
this.scheduledExecutorService = Executors.newSingleThreadScheduledExecutor(new ThreadFactory() {
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setName(THREAD_NAME);
t.setDaemon(true);
return t;
}
});
DumpConfigInfoTask dumpTask = new DumpConfigInfoTask(this);
dumpTask.run();
this.scheduledExecutorService.scheduleWithFixedDelay(dumpTask, SystemConfig.getDumpConfigInterval(),
SystemConfig.getDumpConfigInterval(), TimeUnit.SECONDS);
}
在NotifyService中,会读取配置文件加载服务端节点:
@PostConstruct
public void loadNodes() {
InputStream in = null;
try {
in = ResourceUtils.getResourceAsStream("node.properties");
nodeProperties.load(in);
}
catch (IOException e) {
log.error("加载节点配置文件失败");
}
finally {
try {
if (in != null)
in.close();
}
catch (IOException e) {
log.error("关闭node.properties失败", e);
}
}
log.info("节点列表:" + nodeProperties);
}
1.2 更新配置
登录过程省略,提交数据进入AdminController类的postConfig方法。
先对参数进行检验处理,在进入具体的处理操作。
源码如下:
@RequestMapping(params = "method=postConfig", method = RequestMethod.POST)
public String postConfig(HttpServletRequest request, HttpServletResponse response,
@RequestParam("dataId") String dataId, @RequestParam("group") String group,
@RequestParam("content") String content, ModelMap modelMap) {
response.setCharacterEncoding("GBK");
boolean checkSuccess = true;
String errorMessage = "参数错误";
if (StringUtils.isBlank(dataId) || DiamondUtils.hasInvalidChar(dataId.trim())) {
checkSuccess = false;
errorMessage = "无效的DataId";
}
if (StringUtils.isBlank(group) || DiamondUtils.hasInvalidChar(group.trim())) {
checkSuccess = false;
errorMessage = "无效的分组";
}
if (StringUtils.isBlank(content)) {
checkSuccess = false;
errorMessage = "无效的内容";
}
if (!checkSuccess) {
modelMap.addAttribute("message", errorMessage);
return "/admin/config/new";
}
dataId = dataId.trim();
group = group.trim();
this.configService.addConfigInfo(dataId, group, content);
modelMap.addAttribute("message", "提交成功!");
return listConfig(request, response, dataId, group, 1, 20, modelMap);
}
进入ConfigService类的addConfigInfo方法。
先将数据保存到数据库,然后更新缓存,再保存到本地磁盘。
最后还需要通知其他服务节点同步更改。
源码如下:
public void addConfigInfo(String dataId, String group, String content) {
checkParameter(dataId, group, content);
ConfigInfo configInfo = new ConfigInfo(dataId, group, content);
// 保存顺序:先数据库,再磁盘
try {
persistService.addConfigInfo(configInfo);
// 切记更新缓存
this.contentMD5Cache.put(generateMD5CacheKey(dataId, group), configInfo.getMd5());
diskService.saveToDisk(configInfo);
// 通知其他节点
this.notifyOtherNodes(dataId, group);
}
catch (Exception e) {
log.error("保存ConfigInfo失败", e);
throw new ConfigServiceException(e);
}
}
1.3 节点通知
在ConfigService类中调用Service类的notifyConfigInfoChange方法。
在更新配置后,还需要通知其他节点重新从数据库上同步配置。
源码如下:
private void notifyOtherNodes(String dataId, String group) {
this.notifyService.notifyConfigInfoChange(dataId, group);
}
进入NotifyService类的notifyConfigInfoChange方法。
通过URL调用通知节点的接口。
源码如下:
public void notifyConfigInfoChange(String dataId, String group) {
Enumeration<?> enu = nodeProperties.propertyNames();
while (enu.hasMoreElements()) {
String address = (String) enu.nextElement();
if (address.contains(SystemConfig.LOCAL_IP)) {
continue;
}
String urlString = generateNotifyConfigInfoPath(dataId, group, address);
final String result = invokeURL(urlString);
log.info("通知节点" + address + "分组信息改变:" + result);
}
}
进入NotifyController类的notifyConfigInfo方法。
执行同步逻辑。
源码如下:
@RequestMapping(method = RequestMethod.GET, params = "method=notifyConfigInfo")
public String notifyConfigInfo(@RequestParam("dataId") String dataId, @RequestParam("group") String group) {
dataId = dataId.trim();
group = group.trim();
this.configService.loadConfigInfoToDisk(dataId, group);
return "200";
}
进入ConfigService类的loadConfigInfoDisk方法。
先更新缓存,然后保存到本地磁盘。
源码如下:
public void loadConfigInfoToDisk(String dataId, String group) {
try {
ConfigInfo configInfo = this.persistService.findConfigInfo(dataId, group);
if (configInfo != null) {
this.contentMD5Cache.put(generateMD5CacheKey(dataId, group), configInfo.getMd5());
this.diskService.saveToDisk(configInfo);
}
else {
// 删除文件
this.contentMD5Cache.remove(generateMD5CacheKey(dataId, group));
this.diskService.removeConfigInfo(dataId, group);
}
}
catch (Exception e) {
log.error("保存ConfigInfo到磁盘失败", e);
throw new ConfigServiceException(e);
}
}
2 客户端
2.1 DiamondManager
创建DiamondManager,通过DefaultDiamondManager构造器创建。
通过DiamondManager的getAvailableConfigureInfomation方法获取配置内容。
源码如下:
@Test
void testClient() {
DiamondManager diamondManager = new DefaultDiamondManager("cnpark", "cnpark-station", new ManagerListener() {
public void receiveConfigInfo(String configInfo) {
System.out.println("receiveConfigInfo" + configInfo);
}
public Executor getExecutor() {
return null;
}
});
diamondManager.getDiamondConfigure().setPort(8080);
String availableConfigureInfomation = diamondManager.getAvailableConfigureInfomation(5000);
System.out.println("content" + availableConfigureInfomation);
}
构造DefaultDiamondManager对象。
创建并启动DiamondSubscriber。
源码如下:
public DefaultDiamondManager(String group, String dataId, ManagerListener managerListener) {
this.dataId = dataId;
this.group = group;
diamondSubscriber = DiamondClientFactory.getSingletonDiamondSubscriber();
this.managerListeners.add(managerListener);
((DefaultSubscriberListener) diamondSubscriber.getSubscriberListener()).addManagerListeners(this.dataId,
this.group, this.managerListeners);
diamondSubscriber.addDataId(this.dataId, this.group);
diamondSubscriber.start();
}
2.2 ManagerListener
ManagerListener是一个接口,需要实现并重写其方法。
receiveConfigInfo方法用于接收配置信息,当配置信息发生改动时,可以最新配置信息。
源码如下:
public interface ManagerListener {
public Executor getExecutor();
public void receiveConfigInfo(final String configInfo);
}
2.3 DiamondSubscriber
通过DiamondClientFactory创建DiamondSubscriber,实际上创建的是DefaultDiamondSubscriber。
DiamondSubscriber用于订阅持久的文本配置信息。
源码如下:
private static DiamondSubscriber diamondSubscriber = new DefaultDiamondSubscriber(new DefaultSubscriberListener());
public static DiamondSubscriber getSingletonDiamondSubscriber() {
return diamondSubscriber;
}
创建DefaultDiamondSubscriber。
传入SubscriberListener,创建DiamondConfigure。
源码如下:
public DefaultDiamondSubscriber(SubscriberListener subscriberListener) {
this.subscriberListener = subscriberListener;
this.diamondConfigure = new DiamondConfigure();
}
2.4 SubscriberListener
创建SubscriberListener,实际上创建的是DefaultSubscriberListener。
源码如下:
public Executor getExecutor();
public void receiveConfigInfo(final ConfigureInfomation configureInfomation);
DefaultSubscriberListener中维护了一个ManagerListener的Map集合,Key是dataId和group的组合,Value是ManagerListener的List集合。
重写了receiveConfigInfo方法,配置改动时会遍历ManagerListener并执行其receiveConfigInfo方法。
源码如下:
public void receiveConfigInfo(final ConfigureInfomation configureInfomation) {
String dataId = configureInfomation.getDataId();
String group = configureInfomation.getGroup();
if (null == dataId) {
dataLog.error("[receiveConfigInfo] dataId is null");
return;
}
String key = makeKey(dataId, group);
CopyOnWriteArrayList<ManagerListener> listeners = allListeners.get(key);
if (listeners == null || listeners.isEmpty()) {
dataLog.warn("[notify-listener] no listener for dataId=" + dataId + ", group=" + group);
return;
}
for (ManagerListener listener : listeners) {
try {
notifyListener(configureInfomation, listener);
}
catch (Throwable t) {
dataLog.error("call listener error, dataId=" + dataId + ", group=" + group, t);
}
}
}
private void notifyListener(final ConfigureInfomation configureInfomation, final ManagerListener listener) {
if (listener == null) {
return;
}
final String dataId = configureInfomation.getDataId();
final String group = configureInfomation.getGroup();
final String content = configureInfomation.getConfigureInfomation();
dataLog.info("[notify-listener] call listener " + listener.getClass().getName() + ", for " + dataId + ", "
+ group + ", " + content);
Runnable job = new Runnable() {
public void run() {
try {
listener.receiveConfigInfo(content);
}
catch (Throwable t) {
dataLog.error(t.getMessage(), t);
}
}
};
if (null != listener.getExecutor()) {
listener.getExecutor().execute(job);
}
else {
job.run();
}
}
2.5 DiamondConfigure
创建DiamondConfigure。
DiamondConfigure封装了客户端的一些配置信息,并且会在磁盘上创建一个配置目录。
源码如下:
public DiamondConfigure() {
filePath = System.getProperty("user.home") + "/diamond";
File dir = new File(filePath);
dir.mkdirs();
if (!dir.exists()) {
throw new RuntimeException("创建diamond目录失败:" + filePath);
}
}
2.6 启动DiamondSubscriber
在方法中依次启动LocalConfigInfoProcessor和ServerAddressProcessor,最后轮询获取配置。
源码如下:
public synchronized void start() {
if (isRun) {
return;
}
if (null == scheduledExecutor || scheduledExecutor.isTerminated()) {
scheduledExecutor = Executors.newSingleThreadScheduledExecutor();
}
localConfigInfoProcessor.start(this.diamondConfigure.getFilePath() + "/" + DATA_DIR);
serverAddressProcessor = new ServerAddressProcessor(this.diamondConfigure, this.scheduledExecutor);
serverAddressProcessor.start();
this.snapshotConfigInfoProcessor =
new SnapshotConfigInfoProcessor(this.diamondConfigure.getFilePath() + "/" + SNAPSHOT_DIR);
// 设置domainNamePos值
randomDomainNamePos();
initHttpClient();
// 初始化完毕
isRun = true;
if (log.isInfoEnabled()) {
log.info("当前使用的域名有:" + this.diamondConfigure.getDomainNameList());
}
if (MockServer.isTestMode()) {
bFirstCheck = false;
}
else {
// 设置轮询间隔时间
this.diamondConfigure.setPollingIntervalTime(Constants.POLLING_INTERVAL_TIME);
}
// 轮询
rotateCheckConfigInfo();
addShutdownHook();
}
2.7 启动LocalConfigInfoProcessor
通过LocalConfigInfoProcessor的start方法创建数据目录。
源码如下:
public synchronized void start(String rootPath) {
if (this.isRun) {
return;
}
this.rootPath = rootPath;
this.isRun = true;
if (this.singleExecutor == null || singleExecutor.isTerminated()) {
singleExecutor = Executors.newSingleThreadScheduledExecutor();
}
initDataDir(rootPath);
startCheckLocalDir(rootPath);
}
2.8 启动ServerAddressProcessor
如果是本地优先,通过本地文件获取ServerAddress,默认非本地优先。
如果非本地优先,首次同步获取ServerAddress,然后定时异步获取ServerAddress,默认执行此步骤。
源码如下:
public synchronized void start() {
if (isRun) {
return;
}
isRun = true;
initHttpClient();
if (this.diamondConfigure.isLocalFirst()) {
acquireServerAddressFromLocal();
}
else {
synAcquireServerAddress();
asynAcquireServerAddress();
}
}
2.8.1 同步获取ServerAddress
调用acquireServerAddressOnce方法,先从线上服务器获取,再从日常服务器获取,最后从本地文件获取。
如果从服务器获取成功,则更新本地文件。
源码如下:
protected void synAcquireServerAddress() {
if (!isRun) {
throw new RuntimeException("ServerAddressProcessor不在运行状态,无法同步获取服务器地址列表");
}
if (MockServer.isTestMode()) {
diamondConfigure.addDomainName("测试模式,没有使用的真实服务器");
return;
}
int acquireCount = 0;
if (diamondConfigure.getDomainNameList().size() == 0) {
if (!acquireServerAddressOnce(acquireCount)) {
acquireCount++;
if (acquireServerAddressOnce(acquireCount)) {
// 存入本地文件
storeServerAddressesToLocal();
log.info("在同步获取服务器列表时,向日常ConfigServer服务器获取到了服务器列表");
}
else {
log.info("从本地获取Diamond地址列表");
reloadServerAddresses();
if (diamondConfigure.getDomainNameList().size() == 0)
throw new RuntimeException("当前没有可用的服务器列表");
}
}
else {
log.info("在同步获取服务器列表时,向线上ConfigServer服务器获取到了服务器列表");
// 存入本地文件
storeServerAddressesToLocal();
}
}
}
2.8.2 定时异步获取ServiceAddress
调用acquireServerAddressOnce方法,先从线上服务器获取,再从日常服务器获取。
如果从服务器获取成功,则更新本地文件。
每300秒更新一次。
源码如下:
protected void asynAcquireServerAddress() {
if (MockServer.isTestMode()) {
return;
}
this.scheduledExecutor.schedule(new Runnable() {
public void run() {
if (!isRun) {
log.warn("ServerAddressProcessor不在运行状态,无法异步获取服务器地址列表");
return;
}
int acquireCount = 0;
if (!acquireServerAddressOnce(acquireCount)) {
acquireCount++;
if (acquireServerAddressOnce(acquireCount)) {
// 存入本地文件
storeServerAddressesToLocal();
}
}
else {
// 存入本地文件
storeServerAddressesToLocal();
}
asynAcquireServerAddress();
}
}, asynAcquireIntervalInSec, TimeUnit.SECONDS);
}
2.8.3 从服务器获取ServiceAddress
获取服务器地址和端口,以及文件名称,获取文件中保存的ServiceAddress。
源码如下:
private boolean acquireServerAddressOnce(int acquireCount) {
HostConfiguration hostConfiguration = configHttpClient.getHostConfiguration();
String configServerAddress;
int port;
if (null != diamondConfigure.getConfigServerAddress()) {
configServerAddress = diamondConfigure.getConfigServerAddress();
port = diamondConfigure.getConfigServerPort();
}
else {
if (acquireCount == 0) {
configServerAddress = Constants.DEFAULT_DOMAINNAME;
port = Constants.DEFAULT_PORT;
}
else {
configServerAddress = Constants.DAILY_DOMAINNAME;
port = Constants.DEFAULT_PORT;
}
}
hostConfiguration.setHost(configServerAddress, port);
String serverAddressUrl = Constants.CONFIG_HTTP_URI_FILE;
HttpMethod httpMethod = new GetMethod(serverAddressUrl);
// 设置HttpMethod的参数
HttpMethodParams params = new HttpMethodParams();
params.setSoTimeout(diamondConfigure.getOnceTimeout());
// ///
httpMethod.setParams(params);
try {
if (SC_OK == configHttpClient.executeMethod(httpMethod)) {
InputStreamReader reader = new InputStreamReader(httpMethod.getResponseBodyAsStream());
BufferedReader bufferedReader = new BufferedReader(reader);
String address = null;
List<String> newDomainNameList = new LinkedList<String>();
while ((address = bufferedReader.readLine()) != null) {
address = address.trim();
if (StringUtils.isNotBlank(address)) {
newDomainNameList.add(address);
}
}
if (newDomainNameList.size() > 0) {
log.debug("更新使用的服务器列表");
this.diamondConfigure.setDomainNameList(newDomainNameList);
return true;
}
}
else {
log.warn("没有可用的新服务器列表");
}
}
catch (HttpException e) {
log.error(getErrorMessage(configServerAddress) + ", " + e);
}
catch (IOException e) {
log.error(getErrorMessage(configServerAddress) + ", " + e);
}
catch (Exception e) {
log.error(getErrorMessage(configServerAddress) + ", " + e);
}
finally {
httpMethod.releaseConnection();
}
return false;
}
2.8.4 保存到本地文件
如果是从服务器获取的ServiceAddress,还需要将读取的地址保存到ServiceAddress文件。
源码如下:
void storeServerAddressesToLocal() {
List<String> domainNameList = new ArrayList<String>(diamondConfigure.getDomainNameList());
PrintWriter printWriter = null;
BufferedWriter bufferedWriter = null;
try {
File serverAddressFile =
new File(generateLocalFilePath(this.diamondConfigure.getFilePath(), "ServerAddress"));
if (!serverAddressFile.exists()) {
serverAddressFile.createNewFile();
}
printWriter = new PrintWriter(serverAddressFile);
bufferedWriter = new BufferedWriter(printWriter);
for (String serveraddress : domainNameList) {
bufferedWriter.write(serveraddress);
bufferedWriter.newLine();
}
bufferedWriter.flush();
}
catch (Exception e) {
log.error("存储服务器地址到本地文件失败", e);
}
finally {
if (bufferedWriter != null) {
try {
bufferedWriter.close();
}
catch (IOException e) {
// ignore
}
}
if (printWriter != null) {
printWriter.close();
}
}
}
2.8.5 从本地文件读取ServerAddress
如果没有从服务器中获取,只能从文件中读取。
源码如下:
void reloadServerAddresses() {
FileInputStream fis = null;
InputStreamReader reader = null;
BufferedReader bufferedReader = null;
try {
File serverAddressFile =
new File(generateLocalFilePath(this.diamondConfigure.getFilePath(), "ServerAddress"));
if (!serverAddressFile.exists()) {
return;
}
fis = new FileInputStream(serverAddressFile);
reader = new InputStreamReader(fis);
bufferedReader = new BufferedReader(reader);
String address = null;
while ((address = bufferedReader.readLine()) != null) {
address = address.trim();
if (StringUtils.isNotBlank(address)) {
diamondConfigure.getDomainNameList().add(address);
}
}
bufferedReader.close();
reader.close();
fis.close();
}
catch (Exception e) {
log.error("从本地文件取服务器地址失败", e);
}
finally {
if (bufferedReader != null) {
try {
bufferedReader.close();
}
catch (Exception e) {
}
}
if (reader != null) {
try {
reader.close();
}
catch (Exception e) {
}
}
if (fis != null) {
try {
fis.close();
}
catch (Exception e) {
}
}
}
}
2.9 轮询获取配置
循环探测配置信息是否发生变化,通过三种途径:
1)检查本地配置文件,用缓存中的配置和本地配置文件中的配置比较,如果发生改变,则更新缓存,然后调用popConfigInfo方法。
2)检查服务器配置文件,用缓存中的配置和服务器配置文件中的配置比较,如果发生改变,则更新缓存,然后调用popConfigInfo方法。
3)检查本地磁盘快照,如果没有从本地配置文件中读取配置,也没有从服务器配置文件中读取配置,则获取本地磁盘快照中的配置,并更新缓存,然后调用popConfigInfo方法。
调用popConfigInfo方法后,会推送配置改动的订阅信息到监听器,然后调用saveSnapshot方法更新磁盘快照。
源码如下:
private void rotateCheckConfigInfo() {
scheduledExecutor.schedule(new Runnable() {
public void run() {
if (!isRun) {
log.warn("DiamondSubscriber不在运行状态中,退出查询循环");
return;
}
try {
checkLocalConfigInfo();
checkDiamondServerConfigInfo();
checkSnapshot();
}
catch (Exception e) {
e.printStackTrace();
log.error("循环探测发生异常", e);
}
finally {
rotateCheckConfigInfo();
}
}
}, bFirstCheck ? 60 : diamondConfigure.getPollingIntervalTime(), TimeUnit.SECONDS);
bFirstCheck = false;
}
2.10 获取配置
通过DiamondManager的getAvailableConfigureInfomation方法获取服务器上的配置。
实际上调用的是DefaultDiamondManager的getAvailableConfigureInfomation方法。
源码如下:
public String getAvailableConfigureInfomation(long timeout) {
return diamondSubscriber.getAvailableConfigureInfomation(dataId, group, timeout);
}
继续调用DiamondSubscriber的getAvailableConfigureInfomation方法。
实际上调用的是DefaultDiamondSubscriber的getAvailableConfigureInfomation方法。
源码如下:
public String getAvailableConfigureInfomation(String dataId, String group, long timeout) {
// 尝试先从本地和网络获取配置信息
try {
String result = getConfigureInfomation(dataId, group, timeout);
if (result != null && result.length() > 0) {
return result;
}
}
catch (Throwable t) {
log.error(t.getMessage(), t);
}
// 测试模式不使用本地dump
if (MockServer.isTestMode()) {
return null;
}
return getSnapshotConfiginfomation(dataId, group);
}
2.10.1 从本地和服务器获取配置
调用getConfigureInfomation方法。
尝试从本地配置文件获取配置,获取成功则更新缓存,并更新本地磁盘快照。
尝试从服务器配置文件获取配置,获取成功则更新缓存,并更新本地磁盘快照。
源码如下:
public String getConfigureInfomation(String dataId, String group, long timeout) {
// 同步接口流控
// flowControl();
if (null == group) {
group = Constants.DEFAULT_GROUP;
}
CacheData cacheData = getCacheData(dataId, group);
// 优先使用本地配置
try {
String localConfig = localConfigInfoProcessor.getLocalConfigureInfomation(cacheData, true);
if (localConfig != null) {
cacheData.incrementFetchCountAndGet();
saveSnapshot(dataId, group, localConfig);
return localConfig;
}
}
catch (IOException e) {
log.error("获取本地配置文件出错", e);
}
// 获取本地配置失败,从网络取
String result = getConfigureInfomation(dataId, group, timeout, false);
if (result != null) {
saveSnapshot(dataId, group, result);
cacheData.incrementFetchCountAndGet();
}
return result;
}
2.10.2 从本地磁盘快照获取配置
调用getSnapshotConfiginfomation方法。
源码如下:
private String getSnapshotConfiginfomation(String dataId, String group) {
if (group == null) {
group = Constants.DEFAULT_GROUP;
}
try {
CacheData cacheData = getCacheData(dataId, group);
String config = this.snapshotConfigInfoProcessor.getConfigInfomation(dataId, group);
if (config != null && cacheData != null) {
cacheData.incrementFetchCountAndGet();
}
return config;
}
catch (Exception e) {
log.error("获取snapshot出错, dataId=" + dataId + ",group=" + group, e);
return null;
}
}