几种开源压缩算法比较

Name            Ratio   C.speed D.speed

LZ4 (r41)       2.08    319     1070

LZO 2.05 1x_1   2.07    318      466
QuickLZ 1.5 -1  2.22    274      295
Snappy 1.0.4    2.02    242      683

LZF             2.08    204      482
FastLZ          2.03    196      424

zlib 1.2.5 -1   2.74     46      179

zlib 1.2.5 -6   3.10     17      196



ZLIB: [b 1M] bytes 152089 -> 54404 35.8% comp 12.4 MB/s uncomp 189.3 MB/s

LZ4: [b 1M] bytes 152089 -> 90735 59.7% comp 188.3 MB/s uncomp 1055.6 MB/s

SNAPPY: [b 4M] bytes 152089 -> 90965 59.8% comp 133.0 MB/s uncomp 594.3 MB/s

testdata/asyoulik.txt :

ZLIB: [b 1M] bytes 125179 -> 48897 39.1% comp 11.3 MB/s uncomp 177.4 MB/s

LZ4: [b 1M] bytes 125179 -> 79026 63.1% comp 180.7 MB/s uncomp 1062.6 MB/s

SNAPPY: [b 4M] bytes 125179 -> 80207 64.1% comp 130.0 MB/s uncomp 568.6 MB/s

testdata/baddata1.snappy :

ZLIB: [b 1M] bytes 27512 -> 22941 83.4% comp 24.0 MB/s uncomp 113.8 MB/s

LZ4: [b 1M] bytes 27512 -> 26486 96.3% comp 500.4 MB/s uncomp 3963.1 MB/s

SNAPPY: [b 4M] bytes 27512 -> 26675 97.0% comp 387.4 MB/s uncomp 3445.4 MB/s

testdata/baddata2.snappy :

ZLIB: [b 1M] bytes 27483 -> 23023 83.8% comp 24.1 MB/s uncomp 113.0 MB/s

LZ4: [b 1M] bytes 27483 -> 26633 96.9% comp 549.6 MB/s uncomp 4310.0 MB/s

SNAPPY: [b 4M] bytes 27483 -> 26724 97.2% comp 423.3 MB/s uncomp 3647.4 MB/s

testdata/baddata3.snappy :

ZLIB: [b 1M] bytes 28384 -> 23718 83.6% comp 23.9 MB/s uncomp 112.6 MB/s

LZ4: [b 1M] bytes 28384 -> 27529 97.0% comp 542.0 MB/s uncomp 4256.5 MB/s

SNAPPY: [b 4M] bytes 28384 -> 27476 96.8% comp 342.9 MB/s uncomp 3005.0 MB/s

testdata/cp.html :

ZLIB: [b 1M] bytes 24603 -> 7961 32.4% comp 27.8 MB/s uncomp 198.8 MB/s

LZ4: [b 1M] bytes 24603 -> 12039 48.9% comp 327.8 MB/s uncomp 1561.8 MB/s

SNAPPY: [b 4M] bytes 24603 -> 11838 48.1% comp 195.6 MB/s uncomp 1002.1 MB/s

testdata/fields.c :

ZLIB: [b 1M] bytes 11150 -> 3122 28.0% comp 29.0 MB/s uncomp 237.6 MB/s

LZ4: [b 1M] bytes 11150 -> 5285 47.4% comp 453.1 MB/s uncomp 1356.9 MB/s

SNAPPY: [b 4M] bytes 11150 -> 4728 42.4% comp 224.1 MB/s uncomp 990.1 MB/s

testdata/geo.protodata :

ZLIB: [b 1M] bytes 118588 -> 15131 12.8% comp 56.0 MB/s uncomp 396.9 MB/s

LZ4: [b 1M] bytes 118588 -> 19862 16.7% comp 627.9 MB/s uncomp 2526.5 MB/s

SNAPPY: [b 4M] bytes 118588 -> 27459 23.2% comp 422.5 MB/s uncomp 1640.4 MB/s

testdata/grammar.lsp :

ZLIB: [b 1M] bytes 3721 -> 1222 32.8% comp 30.9 MB/s uncomp 208.9 MB/s

LZ4: [b 1M] bytes 3721 -> 1947 52.3% comp 461.7 MB/s uncomp 1556.7 MB/s

SNAPPY: [b 4M] bytes 3721 -> 1800 48.4% comp 281.0 MB/s uncomp 1118.8 MB/s

testdata/house.jpg :

ZLIB: [b 1M] bytes 126958 -> 126513 99.6% comp 24.7 MB/s uncomp 224.6 MB/s

LZ4: [b 1M] bytes 126958 -> 127272 100.2% comp 2339.8 MB/s uncomp 7389.2 MB/s

SNAPPY: [b 4M] bytes 126958 -> 126803 99.9% comp 2866.3 MB/s uncomp 14627.9 MB/s

testdata/html :

ZLIB: [b 1M] bytes 102400 -> 13699 13.4% comp 44.6 MB/s uncomp 356.4 MB/s

LZ4: [b 1M] bytes 102400 -> 21835 21.3% comp 501.6 MB/s uncomp 1990.6 MB/s

SNAPPY: [b 4M] bytes 102400 -> 24140 23.6% comp 356.6 MB/s uncomp 1381.3 MB/s

testdata/html_x_4 :

ZLIB: [b 1M] bytes 409600 -> 53367 13.0% comp 41.4 MB/s uncomp 356.1 MB/s

LZ4: [b 1M] bytes 409600 -> 85864 21.0% comp 507.6 MB/s uncomp 1962.6 MB/s

SNAPPY: [b 4M] bytes 409600 -> 96472 23.6% comp 350.9 MB/s uncomp 1346.3 MB/s

testdata/kennedy.xls :

ZLIB: [b 1M] bytes 1029744 -> 203992 19.8% comp 18.8 MB/s uncomp 305.2 MB/s

LZ4: [b 1M] bytes 1029744 -> 375713 36.5% comp 416.4 MB/s uncomp 1665.8 MB/s

SNAPPY: [b 4M] bytes 1029744 -> 425735 41.3% comp 304.5 MB/s uncomp 879.3 MB/s

testdata/kppkn.gtb :

ZLIB: [b 1M] bytes 184320 -> 38751 21.0% comp 8.1 MB/s uncomp 248.7 MB/s

LZ4: [b 1M] bytes 184320 -> 84897 46.1% comp 325.8 MB/s uncomp 933.0 MB/s

SNAPPY: [b 4M] bytes 184320 -> 70535 38.3% comp 190.9 MB/s uncomp 690.2 MB/s

testdata/lcet10.txt :

ZLIB: [b 1M] bytes 426754 -> 144904 34.0% comp 12.8 MB/s uncomp 196.0 MB/s

LZ4: [b 1M] bytes 426754 -> 237316 55.6% comp 191.6 MB/s uncomp 1050.0 MB/s

SNAPPY: [b 4M] bytes 426754 -> 243710 57.1% comp 138.7 MB/s uncomp 621.7 MB/s

testdata/mapreduce-osdi-1.pdf :

ZLIB: [b 1M] bytes 94330 -> 74928 79.4% comp 28.8 MB/s uncomp 181.6 MB/s

LZ4: [b 1M] bytes 94330 -> 77760 82.4% comp 794.6 MB/s uncomp 4056.2 MB/s

SNAPPY: [b 4M] bytes 94330 -> 77477 82.1% comp 761.5 MB/s uncomp 3864.8 MB/s

testdata/plrabn12.txt :

ZLIB: [b 1M] bytes 481861 -> 195261 40.5% comp 9.6 MB/s uncomp 175.4 MB/s

LZ4: [b 1M] bytes 481861 -> 323373 67.1% comp 172.4 MB/s uncomp 997.3 MB/s

SNAPPY: [b 4M] bytes 481861 -> 329339 68.3% comp 122.3 MB/s uncomp 518.8 MB/s

testdata/ptt5 :

ZLIB: [b 1M] bytes 513216 -> 56465 11.0% comp 31.9 MB/s uncomp 331.6 MB/s

LZ4: [b 1M] bytes 513216 -> 86327 16.8% comp 659.1 MB/s uncomp 1793.4 MB/s

SNAPPY: [b 4M] bytes 513216 -> 93455 18.2% comp 488.2 MB/s uncomp 1342.0 MB/s

testdata/sum :

ZLIB: [b 1M] bytes 38240 -> 12990 34.0% comp 14.8 MB/s uncomp 186.6 MB/s

LZ4: [b 1M] bytes 38240 -> 18916 49.5% comp 317.7 MB/s uncomp 1582.3 MB/s

SNAPPY: [b 4M] bytes 38240 -> 19837 51.9% comp 195.5 MB/s uncomp 853.2 MB/s

testdata/urls.10K :

ZLIB: [b 1M] bytes 702087 -> 222613 31.7% comp 23.9 MB/s uncomp 207.0 MB/s

LZ4: [b 1M] bytes 702087 -> 329450 46.9% comp 306.6 MB/s uncomp 1448.9 MB/s

SNAPPY: [b 4M] bytes 702087 -> 357267 50.9% comp 209.3 MB/s uncomp 948.3 MB/s

testdata/xargs.1 :

ZLIB: [b 1M] bytes 4227 -> 1736 41.1% comp 31.0 MB/s uncomp 197.2 MB/s

LZ4: [b 1M] bytes 4227 -> 2671 63.2% comp 401.6 MB/s uncomp 1459.2 MB/s

SNAPPY: [b 4M] bytes 4227 -> 2509 59.4% comp 237.4 MB/s uncomp 950.0 MB/s