- 动态调整日志的目的和意义
- 灵活方便,对于线上遇到疑难问题需要排查时,可以灵活的对某台服务的日志做升降级,不需要重新发布
- 集中管理,通过日志管理组件,实现所有服务的日志级别统一控制
- 实现方案总结
系统分为三部分组成,逻辑图如下:
- 日志监听组件
1.1 业务系统注册至zookeeper@PostConstruct public void initLoggerZookeeper() { try { loggerContext = (LoggerContext) LogManager.getContext(false); tempZkPath = zkPath + getHostIp(); initConnection(); createNodeAddWatch(); } catch (Exception e) { logger.error("init logger error: {}", e); } } @PreDestroy public void destroyZookeeperConn() { CloseableUtils.closeQuietly(node); CloseableUtils.closeQuietly(client); } /** * * @MethodName: initLoggerConfig * @Description: * @param 初始化zk连接 * @return void * @throws */ private void initConnection() { client = CuratorFrameworkFactory.newClient(zkConnectString, sessionTimeoutMs, connectionTimeoutMs, new ExponentialBackoffRetry(1000, 3)); client.getConnectionStateListenable().addListener(getConnectionStateListener()); client.start(); try { countDownLatch.await(3, TimeUnit.SECONDS);//不能因为zk注册影响核心业务,最长等待3秒 } catch (InterruptedException e) { e.printStackTrace(); } } /** * * @MethodName: getConnectionStateListener * @Description: 产生zookeeper的连接状态监听 * @param @return * @return ConnectionStateListener * @throws */ private ConnectionStateListener getConnectionStateListener() { ConnectionStateListener connListenner = new ConnectionStateListener() { @Override public void stateChanged(CuratorFramework client, ConnectionState newState) { if (newState == ConnectionState.CONNECTED) { countDownLatch.countDown(); }else if (newState == ConnectionState.LOST) { CloseableUtils.closeQuietly(client); countDownLatch = new CountDownLatch(1); initLoggerZookeeper(); }else if(newState == ConnectionState.RECONNECTED) { CloseableUtils.closeQuietly(client); countDownLatch = new CountDownLatch(1); initLoggerZookeeper(); } } }; return connListenner; }
1.2 增加节点监听/** * * @MethodName: createNodeAddWatch * @Description: 创建临时节点,增加节点监听 * @param @throws Exception * @return void * @throws */ private void createNodeAddWatch() throws Exception { Map<String, LoggerConfig> loggerMap = loggerContext.getConfiguration().getLoggers(); LoggerConfig config = loggerMap.get(""); String defaultLevel = "init"; if(config != null) { Level level = config.getLevel(); defaultLevel = level.name(); } node = new PersistentEphemeralNode(client, Mode.EPHEMERAL, tempZkPath, defaultLevel.getBytes()); node.start(); node.waitForInitialCreate(2, TimeUnit.SECONDS); addWatcherForNode(); } /** * * @MethodName: addWatcherForNode * @Description: 增加watcher * @param @throws Exception * @return void * @throws */ private void addWatcherForNode() throws Exception{ client.getData().usingWatcher(getNodeWatcher()).inBackground().forPath(node.getActualPath()); } /** * * @MethodName: getNodeWatcher * @Description: 返回节点的监听器,监听数据变化 * @param @param node * @param @return * @return Watcher * @throws */ private Watcher getNodeWatcher() { Watcher watcher = new Watcher() { @Override public void process(WatchedEvent event) { if(event.getType() == EventType.NodeDataChanged) { updateLogLevel(); try { addWatcherForNode(); } catch (Exception e) { logger.error("add watcher error: {}", e); } } } }; return watcher; }
1.3 根据节点内容变化,更新loggerContext/** * * @MethodName: updateLogLevel * @Description: 更新日志等级 * @param * @return void * @throws */ private void updateLogLevel() { String newLevel = null; try { newLevel = new String(client.getData().forPath(tempZkPath), "utf-8"); logger.info("newLevel: {}", newLevel); } catch (Exception e) { e.printStackTrace(); logger.error("nodeWatcher: ", e); } Map<String, LoggerConfig> loggerMap = loggerContext.getConfiguration().getLoggers(); for(String loggerName : loggerMap.keySet() ) { LoggerConfig loggerConfig = loggerMap.get(loggerName); loggerConfig.setLevel(getMatchLevel(newLevel)); } loggerContext.updateLoggers(); } /** * * @MethodName: getMatchLevel * @Description: 根据日志字符串获取Level,如果字符串匹配不到,则返回默认info等级 * @param @param logLevel * @param @return * @return Level * @throws */ private Level getMatchLevel(String logLevel) { Level matchLevel = Level.toLevel(logLevel, Level.INFO); return matchLevel; } /** * * @MethodName: getHostIp * @Description: 获取服务的IP * @param @return * @param @throws Exception * @return String * @throws */ private String getHostIp() throws Exception { InetAddress inetAddress = InetAddress.getLocalHost(); return inetAddress.getHostAddress(); }
- 日志管理平台
核心代码如下:/** * 获取所有的日志服务列表 */ @Override public List<LogInfo> getLogList(String parentPath) { List<LogInfo> logInfoList = new ArrayList<>(); try { List<String> machineList = client.getChildren().forPath(parentPath); for(String machine : machineList) { logger.info(parentPath + "/" + machine); String logLevel = new String(client.getData().forPath(parentPath + "/" + machine), "utf-8"); logger.info(logLevel); LogInfo logInfo = new LogInfo(); logInfo.setIpAddress(machine); logInfo.setLogLevel(logLevel); logInfoList.add(logInfo); } } catch (Exception e) { e.printStackTrace(); logger.error("getLogList error", e); } return logInfoList; } @PostConstruct public void initZKclient() { client = CuratorFrameworkFactory.newClient(zkConnectString, sessionTimeoutMs, connectionTimeoutMs, new ExponentialBackoffRetry(1000, 3)); client.start(); } @PreDestroy public void destroyZookeeperConn() { CloseableUtils.closeQuietly(client); } /* * * * @param zkPath * @param level * @see com.xiaojukeji.ad.admin.service.interfaces.LogMachineService#updateLogLevel(java.lang.String, java.lang.String) */ /** * 更新日志 */ @Override public void updateLogLevel(String zkPath, String level, String hostName) { // TODO Auto-generated method stub try { System.out.println(zkPath + "/" + hostName); client.setData().forPath(zkPath + "/" + hostName, level.getBytes("utf-8")); } catch (UnsupportedEncodingException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (Exception e) { // TODO Auto-generated catch block e.printStackTrace(); } }
- zookeeper服务
大名鼎鼎的分布式服务组件,这里就不做介绍了