Setting MySQL INNODB Compression KEY_BLOCK_SIZE

http://serverfault.com/questions/358444/setting-mysql-innodb-compression-key-block-size

Setting MySQL INNODB Compression KEY_BLOCK_SIZE

up vote 3 down vote favorite

I recently started using the Barracuda InnoDB/MySQL table format which allows compression. I compressed one of my tables by running: alter table pricing row_format=compressed, key_block_size=8; After I ran this I viewed the compression statistics (I had cleared them right before the ALTER TABLE): mysql> select * from INFORMATION_SCHEMA.INNODB_CMP; +-----------+--------------+-----------------+---------------+----------------+-----------------+ | page_size | compress_ops | compress_ops_ok | compress_time | uncompress_ops | uncompress_time | +-----------+--------------+-----------------+---------------+----------------+-----------------+ | 1024 | 0 | 0 | 0 | 0 | 0 | | 2048 | 0 | 0 | 0 | 0 | 0 | | 4096 | 0 | 0 | 0 | 0 | 0 | | 8192 | 7029231 | 6352315 | 1437 | 339708 | 41 | | 16384 | 0 | 0 | 0 | 0 | 0 | +-----------+--------------+-----------------+---------------+----------------+-----------------+ 5 rows in set (0.00 sec) mysql> select * from INFORMATION_SCHEMA.INNODB_CMPMEM; +-----------+------------+------------+----------------+-----------------+ | page_size | pages_used | pages_free | relocation_ops | relocation_time | +-----------+------------+------------+----------------+-----------------+ | 128 | 11214 | 0 | 8434571 | 2 | | 256 | 0 | 37 | 0 | 0 | | 512 | 0 | 34 | 0 | 0 | | 1024 | 0 | 2 | 0 | 0 | | 2048 | 0 | 141 | 0 | 0 | | 4096 | 0 | 298 | 96657 | 0 | | 8192 | 15133 | 0 | 4121178 | 5 | | 16384 | 0 | 0 | 0 | 0 | +-----------+------------+------------+----------------+-----------------+ 8 rows in set (0.00 sec) If I divide compress_ops_ok by compress_ops, that's 6352315/7029231 = 90.4%. My understanding is that basically 90.4% of the pages compressed from 16 KB to 8KB, and the remainder were not able to compress down by 2x. I have read that these pages that fail compression hurt performance, but the over 90% that compressed successfully should improve performance quite a bit (by lowering I/O ops). Is there a rule of thumb of what percentage of pages should compress for this to be considered OK? My other option would probably be to just disable compression. My net goal is to reduce the number of I/O operations, and I don't want to enable compression if this is going to be counterproductive. mysql innodb compression share | improve this question asked Feb 9 '12 at 16:08 Nick 70 1 10

add a comment  |

2 Answers

active oldest votes

up vote 2 down vote

Even after running the compression, you still may not get the performance you are looking for. Why ? InnoDB has the Buffer Pool to load data pages and index pages read to fulfill queries. When reading a table and its indexes for the first time, the compressed page must be uncompressed. In fact, you may have twice as much data in the buffer pool as a result of this. Note how this is the case from the MySQL Documentation Compression and the InnoDB Buffer Pool In a compressed InnoDB table, every compressed page (whether 1K, 2K, 4K or 8K) corresponds to an uncompressed page of 16K bytes. To access the data in a page, InnoDB reads the compressed page from disk if it is not already in the buffer pool, then uncompresses the page to its original 16K byte form. This section describes how InnoDB manages the buffer pool with respect to pages of compressed tables. To minimize I/O and to reduce the need to uncompress a page, at times the buffer pool contains both the compressed and uncompressed form of a database page. To make room for other required database pages, InnoDB may “evict” from the buffer pool an uncompressed page, while leaving the compressed page in memory. Or, if a page has not been accessed in a while, the compressed form of the page may be written to disk, to free space for other data. Thus, at any given time, the buffer pool may contain both the compressed and uncompressed forms of the page, or only the compressed form of the page, or neither. InnoDB keeps track of which pages to keep in memory and which to evict using a least-recently-used (LRU) list, so that “hot” or frequently accessed data tends to stay in memory. When compressed tables are accessed, InnoDB uses an adaptive LRU algorithm to achieve an appropriate balance of compressed and uncompressed pages in memory. This adaptive algorithm is sensitive to whether the system is running in an I/O-bound or CPU-bound manner. The goal is to avoid spending too much processing time uncompressing pages when the CPU is busy, and to avoid doing excess I/O when the CPU has spare cycles that can be used for uncompressing compressed pages (that may already be in memory). When the system is I/O-bound, the algorithm prefers to evict the uncompressed copy of a page rather than both copies, to make more room for other disk pages to become memory resident. When the system is CPU-bound, InnoDB prefers to evict both the compressed and uncompressed page, so that more memory can be used for “hot” pages and reducing the need to uncompress data in memory only in compressed form. If this duplication of data content is going on in the Buffer Pool, you need to increase innodb_buffer_pool_size by a small linear factor of the new compression rate. Here is how: SCENARIO You have a DB Server with a 8G Buffer Pool You ran compression with key_block_size=8 8 is 50.00% of 16 50.00% of 8G is 4G raise innodb_buffer_pool_size to 12G (8G + 4G) You ran compression with key_block_size=4 4 is 25.00% of 16 25.00% of 8G is 2G raise innodb_buffer_pool_size to 10G (8G + 2G) You ran compression with key_block_size=2 2 is 12.50% of 16 12.50% of 8G is 1G raise innodb_buffer_pool_size to 9G (8G + 1G) You ran compression with key_block_size=1 1 is 06.25% of 16 06.25% of 8G is 0.5G (512M) raise innodb_buffer_pool_size to 8704M (8G (8192M) + 512M) MORAL OF THE STORY : The InnoDB Buffer Pool just needs additional breathing room when handling compressed data and index pages.