最近在做一些通用的监控埋点,在做到 redis 时遇到了一些问题,搞了半天,有点麻烦,但总算是搞定了。先拿简单的 mysql 举个例子,由于 mybatis 本身提供了一个拦截器,所以可以很轻松的在拦截器中对sql进行监控。
@Component
@Intercepts({
@Signature(type = StatementHandler.class, method = "prepare", args = { Connection.class, Integer.class })
})
public class MybatisMonitorInterceptor implements Interceptor {
private static final String CUSTOM_MYSQL_COUNTER = "custom.mysql.counter";
private static final String CUSTOM_MYSQL_TIMER = "custom.mysql.timer";
//本机ip
private final String HOST_IP;
//本机名字
private final String HOST_NAME;
@Resource
private MeterRegistry registry;
public MybatisMonitorInterceptor() throws UnknownHostException {
InetAddress localhost = InetAddress.getLocalHost();
this.HOST_IP = localhost.getHostAddress();
this.HOST_NAME = localhost.getHostName();
}
@Override
public Object intercept(Invocation invocation) throws Throwable {
//记录开始时间
long start = System.currentTimeMillis();
//执行真正的方法
Object result = invocation.proceed();
//记录sql执行结束时间
long end = System.currentTimeMillis();
long cost = end - start;
//获取sql语句
StatementHandler statementHandler = (StatementHandler) invocation.getTarget();
String sql = statementHandler.getBoundSql().getSql();
String type = getSqlStart(sql);
//获取数据库连接
Connection connection = (Connection) invocation.getArgs()[0];
String url = connection.getMetaData().getURL();
//获取数据库的信息
String regex = "\\b(?:[0-9]{1,3}\\.){3}[0-9]{1,3}:\\d{1,5}\\b";
Pattern pattern = Pattern.compile(regex);
Matcher matcher = pattern.matcher(url);
String mysqlAddress = "unknown";
if(matcher.find()){
mysqlAddress = matcher.group();
}
String scheme = connection.getCatalog();
//统计调用次数
registry.counter(CUSTOM_MYSQL_COUNTER, "type", type,
"datasource", mysqlAddress, "scheme", scheme,
"hostIp", HOST_IP, "hostName", HOST_NAME).increment();
//统计总时间
registry.timer(CUSTOM_MYSQL_TIMER, "type", type,
"datasource", mysqlAddress, "scheme", scheme,
"hostIp", HOST_IP, "hostName", HOST_NAME).record(cost, TimeUnit.MILLISECONDS);
return result;
}
@Override
public Object plugin(Object target) {
return Plugin.wrap(target, this);
}
@Override
public void setProperties(Properties properties) {
// 这个方法可以用来初始化拦截器相关的设置,这里不需要实现
}
private String getSqlStart(String sql){
String upperCaseSql = sql.toUpperCase();
return upperCaseSql.substring(0,6);
}
}
可以看到,埋点监控 mysql 查询性能的指标很好实现,只要实现 mybatis 框架自带的 Interceptor 类就可以了。
但是在监控到 redis 的时候,我却遇到了点问题,实际的项目都是用的 spring-boot-starter-data-redis 依赖,然后封装了工具类,使用的时候可以通过注解或者调用工具类中的静态方法。我最初的思路是通过切面来进行埋点,但是在写的时候遇到了几个问题:
- 切 @Cacheable 这种注解的时候只能切到一次,因为后面数据已经存进了缓存里,再次调用不会进入方法。
- 由于自己封装的工具类都是静态方法,没有交给 springboot 管理bean,所以切不到工具类中的方法。
- 可以切到 redisTemplate 类,但是实际上get、set等方法都不在这个类中,而是通过 redisTemplate 类代理的 ValueOperations 等操作类实现的,而这些操作类,不出意外的,还是切不到。
于是我只能往更底层去切,参考了这篇文章 通用的底层埋点都是怎么做的?,实现的思路如下:
通过切 RedisConnectionFactory ,把 Connection 对象替换成自己写的带有埋点功能的 Connection 类。
public class RedisMonitorAspect {
@Resource
private MeterRegistry registry;
@Pointcut("target(org.springframework.data.redis.connection.RedisConnectionFactory)")
public void connectionFactory() {}
@Pointcut("execution(org.springframework.data.redis.connection.RedisConnection *.getConnection(..))")
public void getConnection() {}
@Pointcut("execution(org.springframework.data.redis.connection.RedisClusterConnection *.getClusterConnection(..))")
public void getClusterConnection() {}
@Around("getConnection() && connectionFactory()")
public Object aroundGetConnection(final ProceedingJoinPoint pjp) throws Throwable {
RedisConnection connection = (RedisConnection) pjp.proceed();
return new CatMonitorRedisConnection(connection, registry);
}
@Around("getClusterConnection() && connectionFactory()")
public Object aroundGetClusterConnection(final ProceedingJoinPoint pjp) throws Throwable {
RedisClusterConnection clusterConnection = (RedisClusterConnection) pjp.proceed();
return new CatMonitorRedisClusterConnection(clusterConnection, registry);
}
}
CatMonitorRedisConnection 中对原生的 RedisConnection 做了增强,也不会影响原有的 RedisConnection 的功能。
public class CatMonitorRedisConnection implements RedisConnection {
private final RedisConnection connection;
private CatMonitorHelper catMonitorHelper;
public CatMonitorRedisConnection(RedisConnection connection, MeterRegistry registry) {
this.connection = connection;
this.catMonitorHelper = new CatMonitorHelper(registry);
}
@Override
public byte[] get(byte[] key) {
return catMonitorHelper.get(RedisCommand.GET, () -> connection.get(key), key);
}
}
要实现的方法有很多,所以很麻烦,这里作为例子就只写一个get方法。
CatMonitorHelper 是用来埋点的工具类,我是这么写的:
public class CatMonitorHelper {
private static final String CUSTOM_REDIS_COUNTER = "custom.redis.counter";
private static final String CUSTOM_REDIS_TIMER = "custom.redis.timer";
private static final String CUSTOM_REDIS_HIT_COUNTER = "custom.redis.hit.counter";
//本机ip
private final String HOST_IP;
//本机名字
private final String HOST_NAME;
private MeterRegistry registry;
public CatMonitorHelper(MeterRegistry registry) {
try {
InetAddress localhost = InetAddress.getLocalHost();
this.HOST_IP = localhost.getHostAddress();
this.HOST_NAME = localhost.getHostName();
} catch (UnknownHostException e) {
throw new RuntimeException(e);
}
this.registry = registry;
}
public byte[] get(RedisCommand command, Supplier<byte[]> supplier, byte[] key){
//记录开始时间
long start = System.currentTimeMillis();
//执行真正的方法
byte[] result = supplier.get();
//记录sql执行结束时间
long end = System.currentTimeMillis();
long cost = end - start;
//注册指标
registMetrics(command.name(), new String(key), cost, result!=null);
return result;
}
private void registMetrics(String command, String key, long cost, boolean hit){
//统计调用次数
registry.counter(CUSTOM_REDIS_COUNTER, "key", key, "method", command,
"hostIp", HOST_IP, "hostName", HOST_NAME).increment();
//统计总时间
registry.timer(CUSTOM_REDIS_TIMER, "key", key, "method", command,
"hostIp", HOST_IP, "hostName", HOST_NAME).record(cost, TimeUnit.MILLISECONDS);
if(hit){
//记录命中的次数
//统计调用次数
registry.counter(CUSTOM_REDIS_HIT_COUNTER, "key", key, "method", command,
"hostIp", HOST_IP, "hostName", HOST_NAME).increment();
}
}
}
这样埋点成功了。