Memcached Timed out(1000) waiting for operation

同事在使用memcached 遇到Timed out(1000) waiting for operation问题。

排除问题的步骤：

先telnet  127.0.0.1 11211     然后stats 一下,观看服务器查看

Here are the stats as far as I understand them:
pid = process id
uptime = number of seconds since the process was started
time = current time
version = memcached version
rusage_user = seconds the cpu has devoted to the process as the user
rusage_system = seconds the cpu has devoted to the process as the system
curr_items = total number of items currently in memcache
total_items = total number of items that have passed through the cache
bytes = total number of bytes currently in use by curr_items
curr_connections = total number of open connections to memcached
connection_structures = ???
cmd_get = total GET commands issued to the server
cmd_set = total SET commands issued to the server
get_hits = total number of times a GET command was able to retrieve and
return data
get_misses = total number of times a GET command was unable to retrieve and
return data
bytes_read = total number of bytes input into the server
bytes_written = total number of bytes written by the server
limit_maxbytes = total storage bytes available to the server.

看一下使用情况是不是很多

top 一下观察一下服务器cup,内存,磁盘IO使用情况。

观察到swap值使用率很高。通过在网上查找资料：http://www.cnblogs.com/cmt/archive/2013/04/25/3041555.html#!comments

swap使用很高的时候会影响IO流的响应时间。可以通过修改swappiness值来修改。前提示你的内存够用。

http://en.wikipedia.org/wiki/Swappiness

Value	Strategy
vm.swappiness = 0	The kernel will swap only to avoid an out of memory condition.
vm.swappiness = 60	The default value.
vm.swappiness = 100	The kernel will swap aggressively which may affect over all performance.

linux 会使用硬盘的一部分做为SWAP分区，用来进行进程调度--进程是正在运行的程序--把当前不用的进程调成‘等待（standby）‘，甚至‘睡眠（sleep）’，一旦要用，再调成‘活动（active）’，睡眠的进程就躺到SWAP分区睡大觉，把内存空出来让给‘活动’的进程。

　　如果内存够大，应当告诉 linux 不必太多的使用 SWAP 分区， 可以通过修改 swappiness 的数值。swappiness=0的时候表示最大限度使用物理内存，然后才是 swap空间，swappiness＝100的时候表示积极的使用swap分区，并且把内存上的数据及时的搬运到swap空间里面。
　　在ubuntu 里面，默认设置swappiness这个值等于60。
　　　　!!!! 如果内存较小，而进程调度频繁，硬盘的响动就会大了 !!!!
　　现在一般1个G的内存可修改为10， 2个G的可改为5， 甚至是0。具体这样做：
　　1.查看你的系统里面的swappiness
　　$ cat /proc/sys/vm/swappiness
　　不出意外的话，你应该看到是 60
　　2.修改swappiness值为10
　　$ sudo sysctl vm.swappiness=10
　　但是这只是临时性的修改，在你重启系统后会恢复默认的60，为长治久安，还要更进一步：
　　$ sudo gedit /etc/sysctl.conf
　　在这个文档的最后加上这样一行:
　　vm.swappiness=10
　　然后保存，重启。ok，你的设置就生效了。