storm调优

storm调优

@(STORM)[storm]

本文从2个方面讨论storm的调优，第一个是集群的调优，第二个是运行在集群中的拓扑的调优，这部分还包括了使用storm-kafka从kafka中读取消息的调优。

官方的一些建议请见：http://storm.apache.org/documentation/FAQ.html
中文版：http://ifeve.com/storm-faq/

一、集群调优

1、netty的调优

netty的配置项主要包括以下几个：

storm.messaging.netty.server_worker_threads: 1
storm.messaging.netty.client_worker_threads: 1
storm.messaging.netty.buffer_size: 5242880 #5MB buffer
# Since nimbus.task.launch.secs and supervisor.worker.start.timeout.secs are 120, other workers should also wait at least that long before giving up on connecting to the other worker. The reconnection period need also be bigger than storm.zookeeper.session.timeout(default is 20s), so that we can abort the reconnection when the target worker is dead.
storm.messaging.netty.max_retries: 300
storm.messaging.netty.max_wait_ms: 1000
storm.messaging.netty.min_wait_ms: 100

# If the Netty messaging layer is busy(netty internal buffer not writable), the Netty client will try to batch message as more as possible up to the size of storm.messaging.netty.transfer.batch.size bytes, otherwise it will try to flush message as soon as possible to reduce latency.
storm.messaging.netty.transfer.batch.size: 262144
# Sets the backlog value to specify when the channel binds to a local address
storm.messaging.netty.socket.backlog: 500

# By default, the Netty SASL authentication is set to false.  Users can override and set it true for a specific topology.
sto