在看dubbo源码时，发现了HashedWheelTimer这个东西，用于实现失败重试,心跳等任务。仔细研究了一下。才知道这是一种基于时间轮算法的任务调度器。在Netty中也有自己的实现。于是查了下资料，并阅读源码，在此记录下一点心得。
dubbo官方给的描述是：
HashedWheelTimer is based on George Varghese and Tony Lauck’s paper, Hashed and Hierarchical Timing Wheels: data structures to efficiently implement a timer facility’. More comprehensive slides are located here
HashedWheelTimer理论结构图：
| 组件 | 描述 |
|---|---|
| Wheel | 时间轮里面的轮子 |
| Tick Duration | 时间轮转动时，每一次滴答的时间。也就是每一格中停留的时间 |
| Ticks per Wheel (Wheel Size) | 时间轮走一圈, 滴答完的总数。时间轮子的格子数量。 |
| Bucket | 时间轮中的格子。包含带有终止期限的任务。 |
先看一下Dubbo中实现的结构图：
| 组件 | 描述 |
|---|---|
| HashedWheelTimer | 时间轮超时任务调度器。包含了一组Bucket,默认的大小是512 |
| HashedWheelBucket | HashedWheelTimeouts的容器，对应时间轮上一个格子。是一个链式结构 |
| HashedWheelTimeout | 包含了一个TimerTask，是HashedWheelBucket中的一个node |
| TimerTask | 指定延时后要执行的任务 |
| workerThread | 执行任务的线程 |
| Worker | 实现Runnable，包含了一组HashedWheelTimeout |
再来看一下dubbo中的实际应用，如下是注册失败重试FailbackRegistry中的实现逻辑：
①类初始化时在构造函数中初始化时间轮调度器
public abstract class FailbackRegistry extends AbstractRegistry {
/**
* The time in milliseconds the retryExecutor will wait
*/
private final int retryPeriod;
// Timer for failure retry, regular check if there is a request for failure, and if there is, an unlimited retry
private final HashedWheelTimer retryTimer;
//构造方法初始化时间轮任务调度器
public FailbackRegistry(URL url) {
super(url);
this.retryPeriod = url.getParameter(REGISTRY_RETRY_PERIOD_KEY, DEFAULT_REGISTRY_RETRY_PERIOD);
// since the retry task will not be very much. 128 ticks is enough.
retryTimer = new HashedWheelTimer(new NamedThreadFactory("DubboRegistryRetryTimer", true), retryPeriod, TimeUnit.MILLISECONDS, 128);
}
HashedWheelTimer初始化需要5个参数：
| 参数名 | 描述 |
|---|---|
| ThreadFactory | 创建一个后台的Thread来执行TimerTask |
| tickDuration | 两次滴答之间的时间 |
| unit | tickDuration参数的时间单位 |
| ticksPerWheel | 时间轮中格子的个数 |
| maxPendingTimeouts | 最大的待执行的HashedWheelTimeout实例个数，默认为-1 。如果设置之后，在使用newTimeout添加新的任务时，会校验这个值，如果待执行的任务数大于设置的值，就会抛出RejectedExecutionException异常。 |
初始化的逻辑如下：
public class HashedWheelTimer implements Timer {
public HashedWheelTimer(ThreadFactory threadFactory,
long tickDuration,
TimeUnit unit,
int ticksPerWheel,
long maxPendingTimeouts) {
//1
if (threadFactory == null) {
throw new NullPointerException("threadFactory");
}
if (unit == null) {
throw new NullPointerException("unit");
}
if (tickDuration <= 0) {
throw new IllegalArgumentException("tickDuration must be greater than 0: " + tickDuration);
}
if (ticksPerWheel <= 0) {
throw new IllegalArgumentException("ticksPerWheel must be greater than 0: " + ticksPerWheel);
}
//2
// Normalize ticksPerWheel to power of two and initialize the wheel.
wheel = createWheel(ticksPerWheel);
mask = wheel.length - 1;
//3
// Convert tickDuration to nanos.
this.tickDuration = unit.toNanos(tickDuration);
// Prevent overflow.
if (this.tickDuration >= Long.MAX_VALUE / wheel.length) {
throw new IllegalArgumentException(String.format(
"tickDuration: %d (expected: 0 < tickDuration in nanos < %d",
tickDuration, Long.MAX_VALUE / wheel.length));
}
//4
workerThread = threadFactory.newThread(worker);
//5
this.maxPendingTimeouts = maxPendingTimeouts;
if (INSTANCE_COUNTER.incrementAndGet() > INSTANCE_COUNT_LIMIT &&
WARNED_TOO_MANY_INSTANCES.compareAndSet(false, true)) {
reportTooManyInstances();
}
}
//创建时间轮子，是一个HashedWheelBucket数组
private static HashedWheelBucket[] createWheel(int ticksPerWheel) {
if (ticksPerWheel <= 0) {
throw new IllegalArgumentException(
"ticksPerWheel must be greater than 0: " + ticksPerWheel);
}
if (ticksPerWheel > 1073741824) {
throw new IllegalArgumentException(
"ticksPerWheel may not be greater than 2^30: " + ticksPerWheel);
}
ticksPerWheel = normalizeTicksPerWheel(ticksPerWheel);
HashedWheelBucket[] wheel = new HashedWheelBucket[ticksPerWheel];
for (int i = 0; i < wheel.length; i++) {
wheel[i] = new HashedWheelBucket();
}
return wheel;
}
//格式话传入参数ticksPerWheel，格式化ticksPerWheel的值，称为最接近它的值的为2的倍数。
// 如输入1 2 3 4 5 6 7 8 -> 1 2 4 4 8 8 8 8
private static int normalizeTicksPerWheel(int ticksPerWheel) {
int normalizedTicksPerWheel = ticksPerWheel - 1;
normalizedTicksPerWheel |= normalizedTicksPerWheel >>> 1;
normalizedTicksPerWheel |= normalizedTicksPerWheel >>> 2;
normalizedTicksPerWheel |= normalizedTicksPerWheel >>> 4;
normalizedTicksPerWheel |= normalizedTicksPerWheel >>> 8;
normalizedTicksPerWheel |= normalizedTicksPerWheel >>> 16;
return normalizedTicksPerWheel + 1;
}
}
初始化步骤如下：
- 校验参数threadFactory，unit 不能为空，tickDuration ticksPerWheel 参数要大于0
- 根据ticksPerWheel来创建一个固定大小的时间轮子。
- 校验时间轮中走一格所用的时间长度不能过大
- 新建一个执行时间轮中任务的线程workerThread
- 校验时间轮中待执行的任务的总数量是否超过设定值。
② 注册时，如果发生异常，就向时间轮调度器中添加一个重新注册的任务，启动时间轮。
/* retry task map */
//注册任务失败缓存
private final ConcurrentMap<URL, FailedRegisteredTask> failedRegistered = new ConcurrentHashMap<URL, FailedRegisteredTask>();
//取消注册失败任务缓存
private final ConcurrentMap<URL, FailedUnregisteredTask> failedUnregistered = new ConcurrentHashMap<URL, FailedUnregisteredTask>();
//订阅服务任务执行失败缓存
private final ConcurrentMap<Holder, FailedSubscribedTask> failedSubscribed = new ConcurrentHashMap<Holder, FailedSubscribedTask>();
//取消订阅服务任务执行失败缓存
private final ConcurrentMap<Holder, FailedUnsubscribedTask> failedUnsubscribed = new ConcurrentHashMap<Holder, FailedUnsubscribedTask>();
//通知失败任务缓存
private final ConcurrentMap<Holder, FailedNotifiedTask> failedNotified = new ConcurrentHashMap<Holder, FailedNotifiedTask>();
@Override
public void register(URL url) {
//校验URL中的协议是否在dubbo.registry.accepts配置的范围中，如果没有配置这个参数，就默认通过校验
if (!acceptable(url)) {
logger.info("URL " + url + " will not be registered to Registry. Registry " + url + " does not accept service of this protocol type.");
return;
}
//调用父类注册方法，实现注册信息缓存功能。
super.register(url);
//从缓存中移除当前注册任务
removeFailedRegistered(url);
removeFailedUnregistered(url);
try {
// Sending a registration request to the server side
//这是一个模版方法，给不同注册中心的实现类来自己实现。
doRegister(url);
} catch (Exception e) {
Throwable t = e;
// If the startup detection is opened, the Exception is thrown directly.
boolean check = getUrl().getParameter(Constants.CHECK_KEY, true)
&& url.getParameter(Constants.CHECK_KEY, true)
&& !CONSUMER_PROTOCOL.equals(url.getProtocol());
boolean skipFailback = t instanceof SkipFailbackWrapperException;
if (check || skipFailback) {
if (skipFailback) {
t = t.getCause();
}
throw new IllegalStateException("Failed to register " + url + " to registry " + getUrl().getAddress() + ", cause: " + t.getMessage(), t);
} else {
logger.error("Failed to register " + url + ", waiting for retry, cause: " + t.getMessage(), t);
}
// Record a failed registration request to a failed list, retry regularly
//添加失败重试任务到时间轮任务调度器
addFailedRegistered(url);
}
}
private void addFailedRegistered(URL url) {
FailedRegisteredTask oldOne = failedRegistered.get(url);
if (oldOne != null) {
return;
}
//新建一个注册失败重试的任务
FailedRegisteredTask newTask = new FailedRegisteredTask(url, this);
oldOne = failedRegistered.putIfAbsent(url, newTask);
if (oldOne == null) {
// never has a retry task. then start a new task for retry.
//添加到时间轮中执行
retryTimer.newTimeout(newTask, retryPeriod, TimeUnit.MILLISECONDS);
}
}
}
注册失败重试任务结构如下：
以上是实现时间轮的调用。时间轮的调用只有有一个入口，就是：
retryTimer.newTimeout(newTask, retryPeriod, TimeUnit.MILLISECONDS);
下面就来分析这个流程：
private final Queue<HashedWheelTimeout> timeouts = new LinkedBlockingQueue<>();
private final Queue<HashedWheelTimeout> cancelledTimeouts = new LinkedBlockingQueue<>();
private final AtomicLong pendingTimeouts = new AtomicLong(0);
private final long maxPendingTimeouts;
private volatile long startTime;
@Override
public Timeout newTimeout(TimerTask task, long delay, TimeUnit unit) {
//1
if (task == null) {
throw new NullPointerException("task");
}
if (unit == null) {
throw new NullPointerException("unit");
}
//2
long pendingTimeoutsCount = pendingTimeouts.incrementAndGet();
if (maxPendingTimeouts > 0 && pendingTimeoutsCount > maxPendingTimeouts) {
pendingTimeouts.decrementAndGet();
throw new RejectedExecutionException("Number of pending timeouts ("
+ pendingTimeoutsCount + ") is greater than or equal to maximum allowed pending "
+ "timeouts (" + maxPendingTimeouts + ")");
}
//3
start();
//4
// Add the timeout to the timeout queue which will be processed on the next tick.
// During processing all the queued HashedWheelTimeouts will be added to the correct HashedWheelBucket.
long deadline = System.nanoTime() + unit.toNanos(delay) - startTime;
// Guard against overflow.
if (delay > 0 && deadline < 0) {
deadline = Long.MAX_VALUE;
}
//5
HashedWheelTimeout timeout = new HashedWheelTimeout(this, task, deadline);
timeouts.add(timeout);
return timeout;
}
步骤如下:
- 校验任务task和延时单位unit非空
- 校验待运行的任务数量是否超过最大限制，
- 启动HashWheelTimer
- 根据启动时间和延迟时间计算任务的终止时间，时间单位是纳秒
- 使用任务task和结束时间deadline来构造一个HashedWheelTimeout实例。并添加到timeouts队列。
这里最重的步骤是启动HashWheelTimer。 来看一下它的启动逻辑。
介绍启动逻辑之前需要普及一下AtomicIntegerFieldUpdater这个原子工具类。
官方解释是：
A reflection-based utility that enables atomic updates to designated {@code volatile int} fields of designated classes. This
class is designed for use in atomic data structures in which several fields of the same node are independently subject to atomic updates.
一个基于反射的工具类。用于原子性的更新一个class类中指定的volatile int类型成员属性。也就是可以线程安全的更新一个类的一些成员属性。
private static final AtomicIntegerFieldUpdater<HashedWheelTimer> WORKER_STATE_UPDATER =
AtomicIntegerFieldUpdater.newUpdater(HashedWheelTimer.class, "workerState");
private final Thread workerThread;
/**
* 0 - init, 1 - started, 2 - shut down
*/
@SuppressWarnings({"unused", "FieldMayBeFinal"})
private volatile int workerState;
/**
* Starts the background thread explicitly. The background thread will
* start automatically on demand even if you did not call this method.
*
* @throws IllegalStateException if this timer has been
* {@linkplain #stop() stopped} already
*/
public void start() {
//判断workerState的值
switch (WORKER_STATE_UPDATER.get(this)) {
//如果等于0 ，便是已经初始化，可以直接启动。实现逻辑是启动workerThread线程
case WORKER_STATE_INIT:
//如果时间轮调度器当前状态是0，就将这个值设置1（started） 。
if (WORKER_STATE_UPDATER.compareAndSet(this, WORKER_STATE_INIT, WORKER_STATE_STARTED)) {
workerThread.start();
}
break;
//如果已经启动，就不执行任何操作
case WORKER_STATE_STARTED:
break;
//如果时间轮调度器已经停止运行，就抛出异常
case WORKER_STATE_SHUTDOWN:
throw new IllegalStateException("cannot be started once stopped");
default:
throw new Error("Invalid WorkerState");
}
// Wait until the startTime is initialized by the worker.
//startTime== 0表示时间轮还没有初始化。如果还没初始化，当前线程就开始等待其初始化。
//当有一个任务添加到其中时，就会将startTime置为1，并startTimeInitialized.countDown();提醒当前线程继续执行。
while (startTime == 0) {
try {
startTimeInitialized.await();
} catch (InterruptedException ignore) {
// Ignore - it will be ready very soon.
}
}
}
可以看到。HashedWheelTimer的启动就是启动workerThread线程。也就是说最核心的调度逻辑，都在线程执行的任务Worker中。
Worker有两个参数属性：
- unprocessedTimeouts 没有执行的HashedWheelTimeout实例
- tick 当前滴答的数量。起始为0
private final Queue<HashedWheelTimeout> timeouts = new LinkedBlockingQueue<>();
private final Queue<HashedWheelTimeout> cancelledTimeouts = new LinkedBlockingQueue<>();
private final class Worker implements Runnable {
private final Set<Timeout> unprocessedTimeouts = new HashSet<Timeout>();
private long tick;
@Override
public void run() {
// Initialize the startTime.
//初始化时间轮任务调度器的启动时间，0表示没有初始化。非0表示已经被初始化。可以启动了。
startTime = System.nanoTime();
if (startTime == 0) {
// We use 0 as an indicator for the uninitialized value here, so make sure it's not 0 when initialized.
startTime = 1;
}
// Notify the other threads waiting for the initialization at start().
//由于这个倒计数的锁的大小初始值是1.因此此处执行countDown()一次时，这个锁就失效了。并提示启动主线程继续执行。
startTimeInitialized.countDown();
//遍历时间轮。
do {
//等待下一次滴答。并获取时间轮中一个格子执行任务的终止时间
final long deadline = waitForNextTick();
if (deadline > 0) {
//计算时间轮执行到第几个。
//举个例子。 假如时间轮的初始化大小ticksPerWheel=8 ，则mask= ticksPerWheel-1=7
//tickPerWheel是2的n次方。因此mask的二进制表现形式为11111...111 ,所以idx = tick
int idx = (int) (tick & mask);
//清空cancelledTimeouts
processCancelledTasks();
//根据index从时间轮中获取一个HashedWheelBucket（组成时间轮的基本单位）
HashedWheelBucket bucket = wheel[idx];
//遍历timeouts中的HashedWheelTimeout，将timeout实例移动到未被执行的HashedWheelBucket中
transferTimeoutsToBuckets();
//执行当前HashedWheelBucket中的所有HashedWheelTimeout
bucket.expireTimeouts(deadline);
//滴答数加1
tick++;
}
} while (WORKER_STATE_UPDATER.get(HashedWheelTimer.this) == WORKER_STATE_STARTED);
// Fill the unprocessedTimeouts so we can return them from stop() method.
//当HashedWheelTimer.stop();调用之后。遍历时间轮wheel中所有的HashWheelBucket。
//将未执行的Timeout实例添加到unprocessedTimeouts中存起来
for (HashedWheelBucket bucket : wheel) {
bucket.clearTimeouts(unprocessedTimeouts);
}
//从timeouts中取出没有取消的Timeout实例。添加到unprocessedTimeouts
for (; ; ) {
HashedWheelTimeout timeout = timeouts.poll();
if (timeout == null) {
break;
}
if (!timeout.isCancelled()) {
unprocessedTimeouts.add(timeout);
}
}
processCancelledTasks();
}
private void transferTimeoutsToBuckets() {
// transfer only max. 100000 timeouts per tick to prevent a thread to stale the workerThread when it just
// adds new timeouts in a loop.
for (int i = 0; i < 100000; i++) {
HashedWheelTimeout timeout = timeouts.poll();
if (timeout == null) {
// all processed
break;
}
if (timeout.state() == HashedWheelTimeout.ST_CANCELLED) {
// Was cancelled in the meantime.
continue;
}
long calculated = timeout.deadline / tickDuration;
timeout.remainingRounds = (calculated - tick) / wheel.length;
// Ensure we don't schedule for past.
final long ticks = Math.max(calculated, tick);
int stopIndex = (int) (ticks & mask);
HashedWheelBucket bucket = wheel[stopIndex];
bucket.addTimeout(timeout);
}
}
private void processCancelledTasks() {
for (; ; ) {
HashedWheelTimeout timeout = cancelledTimeouts.poll();
if (timeout == null) {
// all processed
break;
}
try {
timeout.remove();
} catch (Throwable t) {
if (logger.isWarnEnabled()) {
logger.warn("An exception was thrown while process a cancellation task", t);
}
}
}
}
/**
* calculate goal nanoTime from startTime and current tick number,
* then wait until that goal has been reached.
*
* @return Long.MIN_VALUE if received a shutdown request,
* current time otherwise (with Long.MIN_VALUE changed by +1)
*/
private long waitForNextTick() {
long deadline = tickDuration * (tick + 1);
for (; ; ) {
final long currentTime = System.nanoTime() - startTime;
long sleepTimeMs = (deadline - currentTime + 999999) / 1000000;
if (sleepTimeMs <= 0) {
if (currentTime == Long.MIN_VALUE) {
return -Long.MAX_VALUE;
} else {
return currentTime;
}
}
if (isWindows()) {
sleepTimeMs = sleepTimeMs / 10 * 10;
}
try {
Thread.sleep(sleepTimeMs);
} catch (InterruptedException ignored) {
if (WORKER_STATE_UPDATER.get(HashedWheelTimer.this) == WORKER_STATE_SHUTDOWN) {
return Long.MIN_VALUE;
}
}
}
}
Set<Timeout> unprocessedTimeouts() {
return Collections.unmodifiableSet(unprocessedTimeouts);
}
}
以上就将执行Timeout实例的流程说清楚了。现在就来看一下执行Timeout实例中task的逻辑。
HashedWheelBucket与HashedWheelTimeout关系
**HashedWheelTimeout是一个包含TimerTask的双向链表数据结构。**每一个HashedWheelTimeout都有自己的状态。它执行task任务的方法是expire()。源码如下：
public void expire() {
//校验HashedWheelTimeout的状态必须为init
if (!compareAndSetState(ST_INIT, ST_EXPIRED)) {
return;
}
try {
//调用task的run()方法。执行具体的任务逻辑
task.run(this);
} catch (Throwable t) {
if (logger.isWarnEnabled()) {
logger.warn("An exception was thrown by " + TimerTask.class.getSimpleName() + '.', t);
}
}
}
315
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
