Memcached is a great tool for speeding up your database access. A “stats” command returns usage statistics but few documentation is available on what’s returned.
The official Memcached Wiki shows a bounce of stats commands but doesn’t explain their meaning. I collected some information from various Internet sites.
Perl’s Cache::Memcached module has a ->stats call for getting the data,Cache::Memcached::Fast lacks this call (but I don’t understand why). Here is a sample from a real life multi user development environment:
perl -MCache::Memcached -MData::Dumper=Dumper -le 'print Dumper(Cache::Memcached->new( servers => ["memcache-01:11211", "memcache-02:11211"])->stats);' $VAR1 = { 'hosts' => { 'memcache-02:11211' => { 'misc' => { 'cas_hits' => '0', 'conn_yields' => '1', 'cmd_set' => '584479', 'bytes_written' => '1029238847', 'curr_items' => '28088', 'pid' => '31690', 'uptime' => '538844', 'rusage_system' => '54.690000', 'reclaimed' => '3738', 'get_hits' => '153312', 'incr_misses' => '0', 'total_connections' => '12780', 'delete_misses' => '19082', 'decr_misses' => '0', 'get_misses' => '537174', 'bytes' => '6479905', 'curr_connections' => '32', 'time' => '1323007700', 'pointer_size' => '64', 'connection_structures' => '34', 'total_items' => '84110', 'decr_hits' => '0', 'evictions' => '0', 'auth_cmds' => '0', 'accepting_conns' => '1', 'limit_maxbytes' => '1073741824', 'rusage_user' => '24.910000', 'delete_hits' => '37031', 'cmd_get' => '690486', 'incr_hits' => '0', 'version' => '1.4.5', 'cas_badval' => '0', 'bytes_read' => '329112270', 'threads' => '4', 'cas_misses' => '0', 'auth_errors' => '0', 'cmd_flush' => '0', 'listen_disabled_num' => '0' }, 'sizes' => { '8448' => '1', '384' => '4', '2016' => '1', '9984' => '1', '1376' => '1', [...] } }, 'memcache-01:11211' => { 'misc' => { 'cas_hits' => '0', 'conn_yields' => '1', 'cmd_set' => '2279779', 'bytes_written' => '3607288198', 'curr_items' => '30345', 'pid' => '24040', 'uptime' => '1145391', 'rusage_system' => '310.020000', 'reclaimed' => '14740', 'get_hits' => '391278', 'incr_misses' => '0', 'total_connections' => '27373', 'delete_misses' => '107095', 'decr_misses' => '0', 'get_misses' => '1235540', 'bytes' => '6775829', 'curr_connections' => '34', 'time' => '1323007699', 'pointer_size' => '64', 'connection_structures' => '42', 'total_items' => '323610', 'decr_hits' => '0', 'evictions' => '0', 'auth_cmds' => '0', 'accepting_conns' => '1', 'limit_maxbytes' => '1073741824', 'rusage_user' => '103.220000', 'delete_hits' => '138702', 'cmd_get' => '1626818', 'incr_hits' => '0', 'version' => '1.4.5', 'cas_badval' => '0', 'bytes_read' => '880394652', 'threads' => '4', 'cas_misses' => '0', 'auth_errors' => '0', 'cmd_flush' => '0', 'listen_disabled_num' => '0' }, 'sizes' => { '8448' => '1', '384' => '4', '2016' => '1', '9984' => '1', '1376' => '1', [...] } } }, 'self' => {}, 'total' => { 'cmd_get' => 2317304, 'bytes' => 13255734, 'get_hits' => 544590, 'connection_structures' => 76, 'total_items' => 407720, 'bytes_read' => 1209506922, 'total_connections' => 40153, 'cmd_set' => 2864258, 'bytes_written' => 4636527045, 'curr_items' => 58433, 'get_misses' => 1772714 } };
Huge amount of data, isn’t it? The same data is available via telnet:
telnet memcache-01 11211 Trying 192.168.3.5... Connected to memcache-01. Escape character is '^]'. stats STAT pid 24040 STAT uptime 1145873 STAT time 1323008181 STAT version 1.4.5 STAT pointer_size 64 STAT rusage_user 103.230000 STAT rusage_system 310.030000 STAT curr_connections 34 STAT total_connections 27384 STAT connection_structures 42 STAT cmd_get 1626823 STAT cmd_set 2279784 STAT cmd_flush 0 STAT get_hits 391283 STAT get_misses 1235540 STAT delete_misses 107095 STAT delete_hits 138707 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 880545081 STAT bytes_written 3607442137 STAT limit_maxbytes 1073741824 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 1 STAT bytes 6775829 STAT curr_items 30345 STAT total_items 323615 STAT evictions 0 STAT reclaimed 14740 END quit Connection closed by foreign host.
(Other types of stats may be requested using the commands from the Memcached wiki.)
What do all these values mean?
Field | Sample value | Description |
accepting_conns | 1 | The Memcached server is currently accepting new connections. |
auth_cmds | 0 | Number of authentication commands processed by the server – if you use authentication within your installation. The default is IP (routing) level security which speeds up the actual Memcached usage by removing the authentication requirement. |
auth_errors | 0 | Number of failed authentication tries of clients. |
bytes | 6775829 | Number of bytes currently used for caching items, this server currently uses ~6 MB of it’s maximum allowed (limit_maxbytes) 1 GB cache size. |
bytes_read | 880545081 | Total number of bytes received from the network by this server. |
bytes_written | 3607442137 | Total number of bytes send to the network by this server. |
cas_badval | 0 | The “cas” command is some kind of Memcached’s way to avoid locking. “cas” calls with bad identifier are counted in this stats key. |
cas_hits | 0 | Number of successful “cas” commands. |
cas_misses | 0 | “cas” calls fail if the value has been changed since it was requested from the cache. We’re currently not using “cas” at all, so all three cas values are zero. |
cmd_flush | 0 | The “flush_all” command clears the whole cache and shouldn’t be used during normal operation. |
cmd_get | 1626823 | Number of “get” commands received since server startup not counting if they were successful or not. |
cmd_set | 2279784 | Number of “set” commands serviced since startup. |
connection_structures | 42 | Number of internal connection handles currently held by the server. May be used as some kind of “maximum parallel connection count” but the server may destroy connection structures (don’t know if he ever does) or prepare some without having actual connections for them (also don’t know if he does). 42 maximum connections and 34 current connections (curr_connections) sounds reasonable, the live servers also have about 10% more connection_structures than curr_connections. |
conn_yields | 1 | Memcached has a configurable maximum number of requests per event (-R command line argument), this counter shows the number of times any client hit this limit. |
curr_connections | 34 | Number of open connections to this Memcached server, should be the same value on all servers during normal operation. This is something like the count of mySQL’s “SHOW PROCESSLIST” result rows. |
curr_items | 30345 | Number of items currently in this server’s cache. The production system of this development environment holds more than 8 million items. |
decr_hits | 0 | The “decr” command decreases a stored (integer) value by 1. A “hit” is a “decr” call to an existing key. |
decr_misses | 0 | “decr” command calls to undefined keys. |
delete_hits | 138707 | Stored keys may be deleted using the “delete” command, this system doesn’t delete cached data itself, but it’s using the Memcached to avoid recaching-races and the race keys are deleted once the race is over and fresh content has been cached. |
delete_misses | 107095 | Number of “delete” commands for keys not existing within the cache. These 107k failed deletes are deletions of non existent race keys (see above). |
evictions | 0 | Number of objects removed from the cache to free up memory for new items because Memcached reached it’s maximum memory setting (limit_maxbytes). |
get_hits | 391283 | Number of successful “get” commands (cache hits) since startup, divide them by the “cmd_get” value to get the cache hitrate: This server was able to serve 24% of it’s get requests from the cache, the live servers of this installation usually have more than 98% hits. |
get_misses | 1235540 | Number of failed “get” requests because nothing was cached for this key or the cached value was too old. |
incr_hits | 0 | Number of successful “incr” commands processed. “incr” is a replace adding 1 to the stored value and failing if no value is stored. This specific installation (currently) doesn’t use incr/decr commands, so all their values are zero. |
incr_misses | 0 | Number of failed “incr” commands (see incr_hits). |
limit_maxbytes | 1073741824 | Maximum configured cache size (set on the command line while starting the memcached server), look at the “bytes” value for the actual usage. |
listen_disabled_num | 0 | Number of denied connection attempts because memcached reached it’s configured connection limit (“-c” command line argument). |
pid | 24040 | Current process ID of the Memcached task. |
pointer_size | 64 | Number of bits of the hostsystem, may show “32″ instead of “64″ if the running Memcached binary was compiled for 32 bit environments and is running on a 64 bit system. |
reclaimed | 14740 | Numer of times a write command to the cached used memory from another expired key. These are not storage operations deleting old items due to a full cache. |
rusage_system | 310.030000 | Number of system time seconds for this server process. |
rusage_user | 103.230000 | Numer of user time seconds for this server process. |
threads | 4 | Number of threads used by the current Memcached server process. |
time | 1323008181 | Current unix timestamp of the Memcached’s server. |
total_connections | 27384 | Numer of successful connect attempts to this server since it has been started. Roughly $number_of_connections_per_task * $number_of_webserver_tasks * $number_of_webserver_restarts. |
total_items | 323615 | Numer of items stored ever stored on this server. This is no “maximum item count” value but a counted increased by every new item stored in the cache. |
uptime | 1145873 | Numer of seconds the Memcached server has been running since last restart.1145873 / (60 * 60 * 24) = ~13 days since this server has been restarted |
version | 1.4.5 | Version number of the server |
The ->stat method of Cache::Memcached returns all data (“misc” key) for all currently connected servers plus the number of items per item size (“sizes” key). Another “total” key is calculated for some of the most important statistics keys.
Descriptions for most but not all keys are shown in the Memcached repository, let’s see if this list is being updated in the future.