几种压缩算法的压缩和速度比较 LZ4太快了

本文详细比较了Gzip、Bzip2、LZMA、XZ、LZ4和LZO在压缩文件大小、速度、内存消耗及解压性能上的表现,适用于Linux系统和ARM64架构。通过实例展示了不同算法在不同设置下的优劣,为选择最合适的压缩工具提供参考。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

 

Quick Benchmark: Gzip vs Bzip2 vs LZMA vs XZ vs LZ4 vs LZO

Contents

  [hide

Selected archives

I have selected:

  • Source of the kernel to test source compression
  • Stream protocol with flush

Test conditions

Tests were run on a desktop:

  • Intel Core i5 CPU 750 at 2.67GHz
  • 8GB of DDR3 memory
  • tmpfs as ram disk is used
  • Linux kernel 3.3.2, gentoo amd64
  • CFLAGS: -pipe -O2 -g -floop-block -floop-interchange -fgraphite
  • bzip2-1.0.6-r3, xz-utils-5.0.3, gzip-1.4

Only normal mode will be tested firstly.

The file test results

Note: The first column with numbers 1..9 indicates the compression setting passed to gzip, bzip2 and lzmash (e.g. "gzip -9").

Tarball mode from linux-3.3, original size: 466083840 (445M)

Compressed file size in bytes

 gzipbzip2lzmalzma -exzxz -elz4lzop
 gzipbzip2lzmalzma -exzxz -elz4lzop
1124875819 (120M)93997047 (90M)85618192 (82M)72042179 (69M)85630688 (82M)72069084 (69M)165844264 (159M)168012430 (161M)
2119040249 (114M)87741348 (84M)81480824 (78M)70264395 (68M)81492504 (78M)70282944 (68M)165844264 (159M)166987891 (160M)
3114931686 (110M)84816957 (81MB)79575087 (76MB)69015118 (66MB)79586568 (76MB)69029204 (66MB)165844264 (159M)166987891 (160M)
5102328357 (98M)81837328 (79M)69557610 (67M)67879362 (65M)69583428 (67M)67875988 (65M)-166987891 (160M)
7100128597 (96M)80197758 (77M)67276420 (65M)66868212 (64M)67294092 (65M)66852780 (64M)-116205578 (111M)
999740486 (96M)78963640 (76M)65841213 (63M)65362226 (63M)65859432 (63M)65372696 (63M)-114824102 (110M)

Compression ratio

 gzipbzip2lzmalzma -exzxz -elz4lzop
 gzipbzip2lzmalzma -exzxz -elz4lzop
126.8%20.2%18.4%15.5%18.4%15.5%35.6%36.0%
225.5%18.8%17.5%15.1%17.5%15.1%35.6%35.8%
324.7%18.2%17.1%14.8%17.1%14.8%35.6%35.8%
522.0%17.6%14.9%14.6%14.9%14.6%-35.8%
721.5%17.2%14.4%14.3%14.4%14.3%-24.9%
921.4%16.9%14.1%14.0%14.1%14.0%-24.6%

Compression time

 gzipbzip2lzmalzma -exzxz -elz4lzop
 gzipbzip2lzmalzma -exzxz -elz4lzop
18.1s58.3s31.7s4m37s32.2s4m40s1.3s1.6s
28.5s58.4s40.7s4m49s41.9s4m53s1.4s1.6s
39.6s59.1s1m2s4m36s1m1s4m39s1.3s1.5s
514s1m1s3m5s5m3m6s4m53s-1.5s
721s1m2s4m14s5m52s4m13s5m57s-35s
933s1m3s4m48s6m40s4m51s6m40s-1m5s
  • gzip -1 vs lz4 -1 on x86: lz4 6.2x more fast
  • gzip -1 vs lz4 -1 on ARM: lz4 3.6x more fast

Decompression time

 gzipbzip2lzmalzma -exzxz -elz4lzop
 gzipbzip2lzmalzma -exzxz -elz4lzop
13.5s3.4s6.7s5.9s7.2s6.5s0.4s1.5s
23s15.76.3s5.6s6.8s6.3s0.3s1.4s
33.2s15.9s6s5.6s6.7s6.2s0.4s1.4s
53.2s16s5.5s5.4s6.2s6s-1.5s
73s15s5.3s5.3s5.9s5.8s-1.3s
93s15s5s5.1s5.6s5.6s-1.2s

Memory requirements on compression

 gzipbzip2lzmalzma -exzxz -elz4lzop
 gzipbzip2lzmalzma -exzxz -elz4lzop
10.4MB1.1MB8.3MB12.6MB8.3MB12.5MB12MB0.7MB
20.4MB1.9MB15.8MB24MB15.8MB24MB12MB0.7MB
30.4MB2.7MB30.7MB46.9MB30.8MB47M13.2MB0.7MB
50.4MB4.2MB93MB93MB93MB93MB-0.7MB
70.4MB5.7MB185MB185MB185MB185MB-0.9MB
90.4MB7.2MB672MB673MB673MB673MB-0.9MB
  • Note: lz4 it's the program using this size, the code for internal lz4 use very less memory

Memory requirements on decompression

 gzipbzip2lzmalzma -exzxz -elz4lzop
 gzipbzip2lzmalzma -exzxz -elz4lzop
10.1MB0.3MB1MB1MB1MB1MB13MB0.2MB
20.1MB0.9MB2.1MB2.1MB2.1MB2.1MB12MB0.2MB
30.1MB1.1MB4.1MB4.1MB4.0MB4.0MB13MB0.2MB
50.1MB1.9MB8.1MB8.1MB8.1MB8.1MB-0.2MB
70.1MB2.6MB16MB16MB16MB16MB-0.2MB
90.1MB3.4MB64MB64MB64MB64MB-0.2MB
  • Note: lz4 it's the program using this size, the code for internal lz4 use very less memory

Cli used for -1 compression:

  • time gzip -1 linux-3.3.tar;du -ksh linux-3.3.tar*;ls -l linux-3.3.tar*;time gunzip linux-3.3.tar*
  • time bzip2 -1 linux-3.3.tar;du -ksh linux-3.3.tar*;ls -l linux-3.3.tar*;time bunzip2 linux-3.3.tar.bz2
  • time lzma -1 linux-3.3.tar;du -ksh linux-3.3.tar*;ls -l linux-3.3.tar*;read PAUSe;time unlzma linux-3.3.tar*
  • time lzma -e -1 linux-3.3.tar;du -ksh linux-3.3.tar*;ls -l linux-3.3.tar*;read PAUSe;time unlzma linux-3.3.tar*
  • time xz -1 linux-3.3.tar;du -ksh linux-3.3.tar*;ls -l linux-3.3.tar*;read PAUSe;time unxz linux-3.3.tar*
  • time xz -e -1 linux-3.3.tar;du -ksh linux-3.3.tar*;ls -l linux-3.3.tar*;read PAUSe;time unxz linux-3.3.tar*
  • time /home/user/lz4-read-only/lz4demo64 -c0 linux-3.3.tar linux-3.3.tar.lz4;du -ksh linux-3.3.tar*;ls -l linux-3.3.tar*;read PAUSe;time /home/user/lz4-read-only/lz4demo64 -d linux-3.3.tar.lz4 linux-3.3.tar;rm linux-3.3.tar.*
  • time lzop -1 linux-3.3.tar;rm linux-3.3.tar;du -ksh linux-3.3.tar*;ls -l linux-3.3.tar*;read PAUSe;time lzop -d linux-3.3.tar*;rm linux-3.3.tar.*

Stream test

Time to compress

 gzipzliblz4lz4hcnone
 gzipzliblz4lz4hcnone
137636813031110
2371391   
3378362   
5390392   
7389387   
9393384   

Time to decompress

 gzipzliblz4lz4hcnone
 gzipzliblz4lz4hcnone
1179717788097969
218411831   
317711780   
518061960   
718611851   
918351806   

Messured with:

timespec tstart,tstop;
clock_gettime(CLOCK_REALTIME, &tstart);
//compression/decompression
clock_gettime(CLOCK_REALTIME, &tstop);
compressionTime+=((quint64)tstop.tv_sec * 1000000LL + (quint64)tstop.tv_nsec / 1000LL)-((quint64)tstart.tv_sec * 1000000LL + (quint64)tstart.tv_nsec / 1000LL);

Memory used

gzipzliblz4lz4hcnone
gzipzliblz4lz4hcnone
0.5MB0.5MB0MB0MB0MB

The socket is always used as full duplex, then it's compression + decompression memory.

Compression ratio

It's transferred size/raw size (lower is better)

With the benchmark application

Uploaded at: https://github.com/alphaonex86/debug-devel/tree/master/QTcpCompressionBenchmark

gzipzliblz4lz4hcnone
gzipzliblz4lz4hcnone
0.990.971.591.591

The most over-head of small packet (3Bytes) is drop by high compression with zlib/gzip for the big packet.

Graphics

Compression-ratio.png Compression-time.pngCompression-time.png

Quick benchmark on ARM64

A quick benchmark on ARM64 (odroid, Cortex A53), on kernel Image (12MB), use default compression level (-6) because no way to configure the compression level of btrfs

The speed is on compressed stream, mean the hdd.

 xzgziplz4zstd
Size4.2M5.5M7.8M5.8M
Compression speed0.2 MB/s1.8 MB/s21.4 MB/s2.8 MB/s
Decompression speed4.8 MB/s13.6 MB/s48.4 MB/s19.1 MB/s
### 关于 UniApp 框架推荐资源与教程 #### 1. **Uniapp 官方文档** 官方文档是最权威的学习资料之一,涵盖了从基础概念到高级特性的全方位讲解。对于初学者来说,这是了解 UniApp 架构技术细节的最佳起点[^3]。 #### 2. **《Uniapp 从入门到精通:案例分析与最佳实践》** 该文章提供了系统的知识体系,帮助开发者掌握 Uniapp 的基础知识、实际应用以及开发过程中的最佳实践方法。它不仅适合新手快速上手,也能够为有经验的开发者提供深入的技术指导[^1]。 #### 3. **ThorUI-uniapp 开源项目教程** 这是一个专注于 UI 组件库设计实现的教学材料,基于 ThorUI 提供了一系列实用的功能模块。通过学习此开源项目的具体实现方式,可以更好地理解如何高效构建美观且一致的应用界面[^2]。 #### 4. **跨平台开发利器:UniApp 全面解析与实践指南** 这篇文章按照章节形式详细阐述了 UniApp 的各个方面,包括但不限于其工作原理、技术栈介绍、开发环境配置等内容,并附带丰富的实例演示来辅助说明理论知识点。 以下是几个重要的主题摘选: - **核心特性解析**:解释了跨端运行机制、底层架构组成及其主要功能特点。 - **开发实践指南**:给出了具体的页面编写样例代码,展示了不同设备间 API 调用的方法论。 - **性能优化建议**:针对启动时间缩短、图形绘制效率提升等方面提出了可行策略。 ```javascript // 示例代码片段展示条件编译语法 export default { methods: { showPlatform() { console.log(process.env.UNI_PLATFORM); // 输出当前平台名称 #ifdef APP-PLUS console.log('Running on App'); #endif #ifdef H5 console.log('Running on Web'); #endif } } } ``` #### 5. **其他补充资源** 除了上述提到的内容外,还有许多在线课程视频可供选择,比如 Bilibili 上的一些免费系列讲座;另外 GitHub GitCode 平台上也有不少优质的社区贡献作品值得借鉴研究。 --- ###
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值