metrics指标采集

官方参考资料:

http://metrics.dropwizard.io/4.0.0/

http://metrics.dropwizard.io/4.0.0/getting-started.html

http://metrics.dropwizard.io/4.0.0/manual/index.html


Metrics,英文就是度量、指标的意思,当我们需要为某个系统、服务做监控、做统计,就需要用到Metrics

举个例子,一个图片压缩服务:

  1. 每秒钟的请求数是多少(TPS TransactionPerSecond)
  2. 平均每个请求处理的时间
  3. 请求处理的最长耗时
  4. 等待处理的请求队列长度

又或者一个缓存服务:

  1. 缓存的命中率
  2. 平均查询缓存的时间

基本上每一个服务、应用都需要做一个监控系统,这需要尽量以少量的代码,实现统计某类数据的功能

以 Java 为例,目前最为流行的 metrics 库是来自 Coda Hale 的 dropwizard/metrics ,该库被广泛地应用于各个知名的开源项目中。例如 Hadoop,Kafka,Spark,JStorm 中。


添加Maven依赖

        <dependency>
            <groupId>io.dropwizard.metrics</groupId>
            <artifactId>metrics-core</artifactId>
            <version>4.0.0</version>
        </dependency>

Metric Registries

MetricRegistry类是Metrics的核心,它是存放应用中所有metrics的容器,也是我们使用 Metrics 库的起点
MetricRegistry registry = new MetricRegistry();

每一个 metric 都有它独一无二的名字,Metrics 中使用句点名字,如 com.example.Queue.size。当你在 com.example.Queue 下有两个 metric 实例,如request和response,可以命名为:com.example.Queue.request.size 和 com.example.Queue.response.size 。使用MetricRegistry类的静态方法name(),可以非常方便地生成这种格式的名字,例如:

MetricRegistry.name(Queue.class,"request","size");
MetricRegistry.name(Queue.class,"response","size");

Metrics 数据展示

Metircs 提供了 Report 接口,用于展示 metrics 获取到的统计数据。metrics-core中主要实现了四种 reporter:JMXconsoleSLF4J, 和 CSV

1、console

可看下面“Metrics类型”的例子

2、JMX

看官方文档

3、SLF4J

看官方文档

4、CSV

看官方文档

5、除了metrics-core提供的4种数据输出外,metrics-servlets还提供了通过HTTP的输出方式

看官方文档

五种 Metrics 类型

1、Gauges

最简单的度量指标,只有一个简单的返回值,例如,我们想衡量一个待处理队列中任务的个数,代码如下:

package com.metrics;

import com.codahale.metrics.*;
import java.util.LinkedList;
import java.util.Queue;
import java.util.concurrent.TimeUnit;

public class GaugeTest {
    public static void main(String[] args) throws InterruptedException {
    	final Queue<String> q = new LinkedList<String>();
        MetricRegistry registry = new MetricRegistry();
        ConsoleReporter reporter = ConsoleReporter.forRegistry(registry).build();
        reporter.start(1, TimeUnit.SECONDS);
		registry.register(
				MetricRegistry.name(GaugeTest.class, "queue", "size"),
				new Gauge<Integer>() {
					public Integer getValue() {
						return q.size();
					}
				});
        while(true){
            Thread.sleep(200);
            q.add("Job-xxx");
        }
    }
}

reporter.start(1, TimeUnit.SECONDS);用于启动ConsoleReporter(启动时会新建一个线程放到线程池中执行,线程在线程池中被执行会有延迟,所以与main函数后面的while部分没有顺序关系),且每秒钟从中读取一次数据输出到控制台,输出结果如下:

18-1-8 11:26:10 ================================================================

-- Gauges ----------------------------------------------------------------------
com.metrics.GaugeTest.queue.size
             value = 5


18-1-8 11:26:11 ================================================================

-- Gauges ----------------------------------------------------------------------
com.metrics.GaugeTest.queue.size
             value = 9


18-1-8 11:26:12 ================================================================

-- Gauges ----------------------------------------------------------------------
com.metrics.GaugeTest.queue.size
             value = 14

但是对于大多数队列数据结构,我们并不想简单地返回q.size(),因为java.utiljava.util.concurrent中实现的size()方法很多都是O(n) 的复杂度,这会影响 Gauge 的性能。

2、Counters

Counter 就是计数器,Counter 只是用 Gauge 封装了 AtomicLong 。我们可以使用如下的方法,使得获得队列大小更加高效。

package com.metrics;

import com.codahale.metrics.*;
import java.util.Queue;
import java.util.Random;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;

public class CounterTest {

    public static Queue<String> q = new LinkedBlockingQueue<String>();

    public static Counter pendingJobs;

    public static Random random = new Random();

    public static void addJob(String job) {
        pendingJobs.inc();
        q.offer(job);
    }

    public static String takeJob() {
        String result =  q.poll();
        if(result != null)
        	pendingJobs.dec();
        return result;
    }

    public static void main(String[] args) throws InterruptedException {
        MetricRegistry registry = new MetricRegistry();
        ConsoleReporter reporter = ConsoleReporter.forRegistry(registry).build();
        reporter.start(1, TimeUnit.SECONDS);

        pendingJobs = registry.counter(MetricRegistry.name(Queue.class,"pending-jobs","size"));

        int num = 0;
        while(true){
            Thread.sleep(200);
            if (random.nextDouble() > 0.7){
                String job = takeJob();
                System.out.println("take job : "+job);
            }else{
            	num++;
                String job = "Job-"+num;
                addJob(job);
                System.out.println("add job : "+job);
            }
        }
    }
}

输出结果如下:

take job : null
take job : null
add job : Job-1
add job : Job-2
add job : Job-3
18-1-8 14:25:03 ================================================================

-- Counters --------------------------------------------------------------------
java.util.Queue.pending-jobs.size
             count = 3


take job : Job-1
add job : Job-4
add job : Job-5
add job : Job-6
18-1-8 14:25:04 ================================================================

-- Counters --------------------------------------------------------------------
java.util.Queue.pending-jobs.size
             count = 5

3、Meters

Meter度量一系列事件发生的速率(rate),例如TPS。Meters会统计每秒,最近1分钟,5分钟,15分钟,还有全部时间的速率。

package com.metrics;

import com.codahale.metrics.ConsoleReporter;
import com.codahale.metrics.Meter;
import com.codahale.metrics.MetricRegistry;
import java.util.Random;
import java.util.concurrent.TimeUnit;

public class MeterTest {
    public static Random random = new Random();
    public static void request(Meter meter){
        System.out.println("request");
        meter.mark();
    }
    public static void request(Meter meter, int n){
        while(n > 0){
            request(meter);
            n--;
        }
    }
    public static void main(String[] args) throws InterruptedException {
        MetricRegistry registry = new MetricRegistry();
        ConsoleReporter reporter = ConsoleReporter.forRegistry(registry).build();
        reporter.start(1, TimeUnit.SECONDS);
        Meter meterTps = registry.meter(MetricRegistry.name(MeterTest.class,"request","tps"));
        while(true){
            request(meterTps,random.nextInt(5));
            Thread.sleep(1000);
        }
    }
}

输出结果:

request
18-1-8 15:09:56 ================================================================

-- Meters ----------------------------------------------------------------------
com.metrics.MeterTest.request.tps
             count = 1
         mean rate = 0.97 events/second
     1-minute rate = 0.00 events/second
     5-minute rate = 0.00 events/second
    15-minute rate = 0.00 events/second


request
request
request
18-1-8 15:09:57 ================================================================

-- Meters ----------------------------------------------------------------------
com.metrics.MeterTest.request.tps
             count = 4
         mean rate = 1.99 events/second
     1-minute rate = 0.00 events/second
     5-minute rate = 0.00 events/second
    15-minute rate = 0.00 events/second


18-1-8 15:09:58 ================================================================

-- Meters ----------------------------------------------------------------------
com.metrics.MeterTest.request.tps
             count = 4
         mean rate = 1.33 events/second
     1-minute rate = 0.00 events/second
     5-minute rate = 0.00 events/second
    15-minute rate = 0.00 events/second


18-1-8 15:09:59 ================================================================

-- Meters ----------------------------------------------------------------------
com.metrics.MeterTest.request.tps
             count = 4
         mean rate = 1.00 events/second
     1-minute rate = 0.00 events/second
     5-minute rate = 0.00 events/second
    15-minute rate = 0.00 events/second


request
request
request
request
18-1-8 15:10:00 ================================================================

-- Meters ----------------------------------------------------------------------
com.metrics.MeterTest.request.tps
             count = 8
         mean rate = 1.60 events/second
     1-minute rate = 1.60 events/second
     5-minute rate = 1.60 events/second
    15-minute rate = 1.60 events/second


request
request
18-1-8 15:10:01 ================================================================

-- Meters ----------------------------------------------------------------------
com.metrics.MeterTest.request.tps
             count = 10
         mean rate = 1.66 events/second
     1-minute rate = 1.60 events/second
     5-minute rate = 1.60 events/second
    15-minute rate = 1.60 events/second

4、Histograms

Histogram统计数据的分布情况。比如最小值,最大值,中间值,还有中位数,75百分位, 90百分位, 95百分位, 98百分位, 99百分位, 和 99.9百分位的值(percentiles)。

package com.metrics;

import com.codahale.metrics.ConsoleReporter;
import com.codahale.metrics.ExponentiallyDecayingReservoir;
import com.codahale.metrics.Histogram;
import com.codahale.metrics.MetricRegistry;
import java.util.Random;
import java.util.concurrent.TimeUnit;
public class HistogramTest {
    public static Random random = new Random();
    public static void main(String[] args) throws InterruptedException {
        MetricRegistry registry = new MetricRegistry();
        ConsoleReporter reporter = ConsoleReporter.forRegistry(registry).build();
        reporter.start(1, TimeUnit.SECONDS);
        Histogram histogram = new Histogram(new ExponentiallyDecayingReservoir());
        registry.register(MetricRegistry.name(HistogramTest.class, "request", "histogram"), histogram);
        while(true){
            Thread.sleep(1000);
            histogram.update(random.nextInt(100000));
        }
    }
}

输出结果:

18-1-8 15:33:20 ================================================================

-- Histograms ------------------------------------------------------------------
com.metrics.HistogramTest.request.histogram
             count = 1
               min = 0
               max = 0
              mean = 0.00
            stddev = 0.00
            median = 0.00
              75% <= 0.00
              95% <= 0.00
              98% <= 0.00
              99% <= 0.00
            99.9% <= 0.00


18-1-8 15:33:21 ================================================================

-- Histograms ------------------------------------------------------------------
com.metrics.HistogramTest.request.histogram
             count = 1
               min = 23833
               max = 23833
              mean = 23833.00
            stddev = 0.00
            median = 23833.00
              75% <= 23833.00
              95% <= 23833.00
              98% <= 23833.00
              99% <= 23833.00
            99.9% <= 23833.00


18-1-8 15:33:22 ================================================================

-- Histograms ------------------------------------------------------------------
com.metrics.HistogramTest.request.histogram
             count = 2
               min = 23833
               max = 91252
              mean = 57795.32
            stddev = 33708.55
            median = 91252.00
              75% <= 91252.00
              95% <= 91252.00
              98% <= 91252.00
              99% <= 91252.00
            99.9% <= 91252.00


18-1-8 15:33:23 ================================================================

-- Histograms ------------------------------------------------------------------
com.metrics.HistogramTest.request.histogram
             count = 3
               min = 23833
               max = 91252
              mean = 54671.26
            stddev = 27765.11
            median = 48562.00
              75% <= 91252.00
              95% <= 91252.00
              98% <= 91252.00
              99% <= 91252.00
            99.9% <= 91252.00

5、Timers

package com.metrics;

import com.codahale.metrics.ConsoleReporter;
import com.codahale.metrics.MetricRegistry;
import com.codahale.metrics.Timer;
import java.util.Random;
import java.util.concurrent.TimeUnit;
public class TimerTest {
    public static Random random = new Random();
    public static void main(String[] args) throws InterruptedException {
        MetricRegistry registry = new MetricRegistry();
        ConsoleReporter reporter = ConsoleReporter.forRegistry(registry).build();
        reporter.start(1, TimeUnit.SECONDS);
        Timer timer = registry.timer(MetricRegistry.name(TimerTest.class,"get-latency"));
        Timer.Context ctx;
        while(true){
            ctx = timer.time();//计时开始
            Thread.sleep(random.nextInt(1000));//模拟操作耗时
            //计时结束,meter:统计总数、每秒、分等调用的次数,histogram:记录每次操作耗时的分布情况
            ctx.stop();
        }
    }
}

输出结果:

18-1-8 15:56:55 ================================================================

-- Timers ----------------------------------------------------------------------
com.metrics.TimerTest.get-latency
             count = 2
         mean rate = 1.99 calls/second
     1-minute rate = 0.00 calls/second
     5-minute rate = 0.00 calls/second
    15-minute rate = 0.00 calls/second
               min = 88.08 milliseconds
               max = 693.13 milliseconds
              mean = 390.60 milliseconds
            stddev = 302.52 milliseconds
            median = 693.13 milliseconds
              75% <= 693.13 milliseconds
              95% <= 693.13 milliseconds
              98% <= 693.13 milliseconds
              99% <= 693.13 milliseconds
            99.9% <= 693.13 milliseconds


18-1-8 15:56:56 ================================================================

-- Timers ----------------------------------------------------------------------
com.metrics.TimerTest.get-latency
             count = 3
         mean rate = 1.50 calls/second
     1-minute rate = 0.00 calls/second
     5-minute rate = 0.00 calls/second
    15-minute rate = 0.00 calls/second
               min = 88.08 milliseconds
               max = 848.11 milliseconds
              mean = 544.63 milliseconds
            stddev = 327.80 milliseconds
            median = 693.13 milliseconds
              75% <= 848.11 milliseconds
              95% <= 848.11 milliseconds
              98% <= 848.11 milliseconds
              99% <= 848.11 milliseconds
            99.9% <= 848.11 milliseconds


18-1-8 15:56:57 ================================================================

-- Timers ----------------------------------------------------------------------
com.metrics.TimerTest.get-latency
             count = 5
         mean rate = 1.67 calls/second
     1-minute rate = 0.00 calls/second
     5-minute rate = 0.00 calls/second
    15-minute rate = 0.00 calls/second
               min = 88.08 milliseconds
               max = 848.11 milliseconds
              mean = 519.35 milliseconds
            stddev = 323.80 milliseconds
            median = 693.13 milliseconds
              75% <= 796.57 milliseconds
              95% <= 848.11 milliseconds
              98% <= 848.11 milliseconds
              99% <= 848.11 milliseconds
            99.9% <= 848.11 milliseconds















评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值