HDFS-NFSgatway speed test on AWS VM
-
Backgroud
VM type: t2.micro, 1vCPU, 1GB RAM, 8 GBEBS
OS: Linux, REHL 6
-
Test Method
dd commard
File size
1MBbyte, 5MByte
FS type:
Raw, nfs
-
Test result
(MB/s) | Raw FS | NFS-HDFS |
1M | 81.7 | 62.0 |
5M | 6.0 | 12.2 |
分析:
a) 的确变慢了不少。需要优化,缓冲等等技术
b) AWS的存储的性能,也是随时间变化的。好像在前台和后台之间切换,一般在晚6点左右比较高。
-
test sample
a) write 1M-size files to HDFS throughnfs gateway
b) write 5M-size files to HDFS throughnfs gateway