在使用async的 eachLimit(arr,limit,iterator,callback)方法时,当数组arr比较大时会报错:Maximum call stack size exceeded
导致这个问题是因为iterator是同步方法,请参考:https://github.com/caolan/async/issues/700
那么为什么同步迭代器会导致这种错误呢?
翻看了一段async的源代码:
var _eachLimit = function (limit) {
return function (arr, iterator, callback) {
callback = callback || function () {};
if (!arr.length || limit <= 0) {
return callback();
}
var completed = 0;
var started = 0;
var running = 0;
(function replenish () {
if (completed >= arr.length) {
return callback();
}
while (running < limit && started < arr.length) {
started += 1;
running += 1;
iterator(arr[started - 1], function (err) {
if (err) {
callback(err);
callback = function () {};
}
else {
completed += 1;
running -= 1;
if (completed >= arr.length) {
callback();
}
else {
replenish();
}
}
});
}
})();
};
};
从上面这段代码可以看到,当迭代器变成同步代码的时候,整个调用就相当于递归调用,当数组变大的时候,递归就会变深,以致堆栈溢出。可以用下面代码测试:
function foo(deepth) {
if (deep) foo(--deepth);
}
在我本地调用 foo(17698) 时控制台就报相同错误:RangeError: Maximum call stack size exceeded
在使用async库时,出现这种错误时通常是应为我们写了类似下面的代码:
if (condition) {
//异步代码
} else {
//同步代码
}
这种情况下,应该尽量先进行条件过滤,再对异步条件使用async的方法。
对于不能先进行过滤的情况,就需要对同步代码进行包装。
那么应该如何包装同步代码呢?通常有下面三种考虑:
function foo(<span style="font-family: Arial, Helvetica, sans-serif;">deepth</span>) {
if (deepth) process.nextTick(function (){foo(--deepth);});
}
function foo(deepth) {
if (deepth) setImmediate(function (){foo(--deepth);});
}
function foo(deepth) {
if (deep) setTimeout(function (){foo(--deepth);}, 0);
}
对于第一种情况,我在本地(node v0.10.33)运行时,当deepth大于1000时,会报出:
(node) warning: Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral.
RangeError: Maximum call stack size exceeded
这是因为:
process.maxTickDepth
- Number Default = 1000
Callbacks passed to process.nextTick
will usually be called at the end of the current flow of execution, and are thus approximately as fast as calling a function synchronously. Left unchecked, this would starve the event loop, preventing any I/O from occurring.
但是v0.12版本的node已经移除了process.maxTickDepth:
process.maxTickDepth
has been removed, allowing process.nextTick
to starve I/O indefinitely.
下面对这三种方式作个比较:
1.process.nextTick:回调结果保存到数组,事件优先级最高,最快,但会阻塞IO,慎用;
2.setImmediate:回调结果保存到链表,不会阻塞IO,推荐使用;
3.setTimeout:需要动用红黑树,性能较低,服务器端不推荐。
这里关于这三者的解释比较粗浅,深入了解请参考《深入浅出nodejs》3.4节。