[Erlang危机](3.3)丢弃数据下篇





 原创文章，转载请注明出处：服务器非业余研究http://blog.csdn.net/erlib 作者Sunface

Stack Buffers

堆栈缓冲区

Stack buffers are ideal when you want the amount of control offered by queue buffers, but you have an important requirement for low latency.
To use a stack as a buffer, you’ll need two processes, just like you would with queue buffers, but a list 20 will be used instead of a queue data structure.

  堆栈缓冲区适用场景：当你想拥有队列缓冲区对消息的控制权(见丢弃数据上篇)，又有系统低延迟需求时。

  使用堆栈来作缓冲，你也需要2个进程，就和队列缓冲一样的，但是需要用List(列表)21 结构来取代队列缓冲的queue结构(队列).

The reason the stack buffer is particularly good for low latency is related to issues similar to buffer bloat ²¹. If you get behind on a few messages being buffered in a queue, all the messages in the queue get to be slowed down and acquire milliseconds of wait time.
Eventually, they all get to be too old and the entire buffer needs to be discarded.

 堆栈缓冲对于低延迟系统特别好的一个重要原因是：不容易引起与缓冲膨胀相关的问题²¹。如果你想随时跟进队列缓冲中的某几个消息，所有的在队列中的消息处理都会变慢并且需要毫秒级的等待时间。最终，他们会因为延时太久都变成旧数据，整个缓冲区都要被丢弃掉。

On the other hand, a stack will make it so only a restricted number of elements are kept waiting while the newer ones keep making it to the server to be processed in a timely manner.
Whenever you see the stack grow beyond a certain size or notice that an element in it is too old for your QoS requirements you can just drop the rest of the stack and keep going from there. PO Box also offers such a buffer implementation.

   而另一方面，堆栈缓冲区会对保持等待的消息数量有一个严格的限制，并且将新来的消息交给服务器进行及时处理。

   当你看到堆栈缓冲区已增长到一定大小或者发现堆栈中某个元素对于你的QoS需求来说已经太旧而过时了，你就可以把剩余的堆栈部分都丢弃掉，然后再继续工作，POBox就实现了这样的缓冲buffer。

A major downside of stack buffers is that messages are not necessarily going to be processed in the order they were submitted — they’re nicer for independent tasks, but will ruin your day if you expect a sequence of events to be respected.

 堆栈缓冲区主要的缺点就是消息并不是按他们的进入顺序依次处理的，他们对相对独立的任务支持非常好，但你千万不要期望它按先来先处理的原则执行。

[20] Erlang lists are stacks. For all we care, they provide push and pop operations that take O(1) complexity and are very fast.
[21] http://queue.acm.org/detail.cfm?id=2071893

[注20]：Erlang List就是一个堆栈，我们都关心地是他提供压栈和弹出操作且只有O(1)的复杂度。
[注21]：http://queue.acm.org/detail.cfm?id=2071893

Time-Sensitive Buffers

对时间敏感的缓冲区

If you need to react to old events before they are too old, then things become more complex, as you can’t know about it without looking deep in the stack each time, and dropping from the bottom of the stack in a constant manner gets to be inefficient.

 如果你需要消息在过时之前被处理掉，那问题就会变得更加复杂了，如果是堆栈缓冲区，你只能每次都深入里面才能知道每条消息是否过时，如果过时就丢弃，这样的做法效率是很低下的。

An interesting approach could be done with buckets, where multiple stacks are used, with each of them containing a given time slice. When requests get too old for the QoS constraints, drop an entire bucket, but not the entire buffer.

 下面介绍一个有意思实现方法：使用多重堆栈，每个堆栈都包含一个给定的时间片(time slice).当消息过时并且不符合QoS的约束时，就把相应的堆栈丢弃掉，注意不是整个缓冲区。

It may sound counter-intuitive to make some requests a lot worse to benefit the majority — you’ll have great medians but poor 99 percentiles — but this happens in a state where you would drop messages anyway, and is preferable in cases where you do need low latency.

  听起来有悖常理：让一些请求消息失败来造福其余的大多数请求，但这只发生在你无论如何都要丢弃消息的情况下，特别是要保证低延迟的场合。

Dealing With Constant Overload

处理持续性的过载

Being under constant overload may require a new solution. Whereas both queues and buffers will be great for cases where overload happens from time to time (even if it’s a rather prolonged period of time), they both work more reliably when you expect the input rate to eventually drop, letting you catch up.

 针对持续性的过载，需要一个新的解决方案。鉴于队列和缓冲可以很好的应付突发性的过载(即使这是一个相当长的时间)，只要消息传入量最终会下降，这两个缓冲区就是可以信赖的，并且让系统逐渐回复正常。

You’ll mostly get problems when trying to send so many messages they can’t make it all to one process without overloading it.

 当你试图发非常多的消息到同一个进程时一定会遇到麻烦。

Two approaches are generally good for this case:
 • Have many processes that act as buffers and load-balance through them (scale horizontally)
 • use ETS tables as locks and counters (reduce the input)

  以下有2个方法非常适用于这种场景：  

       • 使用更多的进程来作为缓冲区，通过这些缓冲区实现负载均衡 (水平扩展)
    • 使用ETS表来作锁(locks)和计数器(counters) (减少输入)

ETS tables are generally able to handle a ton more requests per second than a process, but the operations they support are a lot more basic. A single read, or adding or removing from a counter atomically is as fancy as you should expect things to get for the general case.
ETS tables will be required for both approaches. Generally speaking, the first approach could work well with the regular process registry:

 ETS表通常能比进程处理更多的请求，但是ETS表的操作往往较为简单基础。一个简单的增减查删操作就已经是你能想象到的高级操作了。

 上述两种方法都需要使用ETS表。一般来说，第一种方法使用注册进程就能达成。

you take N processes to divide up the load, give them all a known name, and pick one of them to send the message to. Given you’re pretty much going to assume you’ll be overloaded, randomly picking a process with an even distribution tends to be reliable: no state communication is required, work will be shared in a roughly equal manner, and it’s rather insensitive to failure.

 你可以用N个进程来分担负载，给每个进程都注册一个名字，这样发消息时就能指定具体的某个进程。假定你的系统必定会过载，那随机挑选一个进程来分配任务还是很靠谱的：不需要状态通信，任务就能以相同的方式共享，并且对失败的容忍度也是较高的。

In practice, though, we want to avoid atoms generated dynamically, so I tend to prefer to register workers in an ETS table with read_concurrency set to true.
It’s a bit more work, but it gives more flexibility when it comes to updating the number of workers later on.

 在实践中，我们都会尽量禁止动态生成原子，所以我更喜欢在ETS表中注册工作进程，并把read_concurrency设置为true.这虽然需要多做一些工作，但是在更新工作进程的数目时更灵活，伸缩性更好。

An approach similar to this one is used in the lhttpc ²² library mentioned earlier, to split load balancers on a per-domain basis.

还有个与上面这个类似的方法：此方法在前面提到的lhttpc²²有使用，那就是将负载均衡放在每一个基本域上(per-domain basis).

For the second approach, using counters and locks, the same basic structure still remains (pick one of many options, send it a message), but before actually sending a message, you must atomically update an ETS counter ²³. There is a known limit shared across all clients (either through their supervisor, or any other config or ETS value) and each request that can be made to a process needs to clear this limit first.

第二个方法就是：使用计数器和锁机制，仍然是原来的基本结构(选择其中一个发送消、时息)，但是在真正发消息之前，你必须要自动更新一下ETS的计数器(counter)²³.同时系统中会有一个所有客户端共享的限制值(该值可以存储在监督者，config,ETS表中)，然后每一个请求处理前必须要验证通过这个限制。

This approach has been used in dispcount ²⁴ to avoid message queues, and to guarantee low-latency responses to any message that won’t be handled so that you do not need to wait to know your request was denied. It is then up to the user of the library whether to give up as soon as possible, or to keep retrying with different workers.

 这个方法被dispcount²⁴所使用：用来避免消息队列过长，同时也保证了可以很快地回应任何不能处理的消息，这样你就不需要等待很久才知道请求的结果，然后让用户库决定是放弃请求还是继续尝试其他的工作进程。

[22] The lhttpc_lb module in this library implements it.
[23] By using ets:update_counter/3.
[24] https://github.com/ferd/dispcount

[注22] ：可以查看lhttpc_l模块实现了。
[注23] :通过使用ets:update_counter/3。
[注24] ：https://github.com/ferd/dispcount

How Do You Drop

丢弃数据的原则

Most of the solutions outlined here work based on message quantity, but it’s also possible to try and do it based on message size, or expected complexity, if you can predict it. When using a queue or stack buffer, instead of counting entries, all you may need to do is count their size or assign them a given load as a limit.

 大部分这里列出的方法都是基于消息数量的，但也可以试下基于消息大小或某个更复杂的指标。当你使用队列或多缓冲区(stack buffer)，而不是计算消息总数量，你可能需要做的是计算好消息大小或者给予一个限定大小的负载。

I’ve found that in practice, dropping without regard to the specifics of the message works rather well, but each application has its share of unique compromises that can be acceptable or not ²⁵.

 我在实践中发现：丢弃消息时不去关心消息的用途是一个很好的实现，但每个application都要有自己的机制来决定这样做是否可以²⁵。

There are also cases where the data is sent to you in a "fire and forget" manner — the entire system is part of an asynchronous pipeline — and it proves difficult to provide feedback to the end-user about why some requests were dropped or are missing.

 还有一些情况是是数据发给系统后，就置之不理了 ---整个系统都是异步管道的一部分----这个就很难给终端用户反馈为什么有些请求会被忽略或丢弃掉。

If you can reserve a special type of message that accumulates dropped responses and tells the user " N messages were dropped for reason X", that can, on its own, make the compromise far more acceptable to the user.

 如果你可以创建一种特定的消息类型用来统计丢弃的信息数目和原因，那就告诉用户"N条消息被因X原因被丢弃掉了",那么这就更能让用户接受点(相对什么都不找反馈来讲)。

This is the choice that was made with Heroku’s logplex log routing system, which can spit out L10 errors, alerting the user that a part of the system can’t deal with all the volume right now.

 HeroKu的logplex 就是这样为系统处理日志的，它可以把日志分成10个等级，并会警告用户哪个等级的日志现在不能处理。

In the end, what is acceptable or not to deal with overload tends to depend on the humans that use the system. It is often easier to bend the requirements a bit than develop new technology, but sometimes it is just not avoidable.

 最后，是否处理过载往往取决于使用这个系统的用户，一般来说改一下需求可能会比开发一个新技术更加容易，但有时开发新技术又是不可避免地。

[25] Old papers such as Hints for Computer System Designs by Butler W. Lampson recommend dropping messages: "Shed load to control demand, rather than allowing the system to become overloaded." The paper also mentions that "A system cannot be expected to function well if the demand for any resource exceeds two-thirds of the capacity, unless the load can be characterized extremely well." adding that "The only systems in which cleverness has worked are those with very well-known loads."

[注25] ： Butler W.Lampson写的 Hints for Computer System Designs 论文中建议:"宁愿控制负载，也不能让系统超负荷",论文还指出"一个系统如果被要求工作大于2/3的负载之上，就不可以被认为是正常工作的"，"聪明的系统都会工作在适当的负载下"。