Hadoop2.7.0学习——基准测试

最新推荐文章于 2024-09-24 09:03:30 发布
最新推荐文章于 2024-09-24 09:03:30 发布
阅读量5.2k 收藏 2
点赞数
分类专栏： hadoop 文章标签： hadoop 基准测试
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/flygoa/article/details/52127382
版权

Hadoop2.7.0学习——基准测试

Hadoop 带有一些基准测试程序，可以最少的准备成本轻松运行。基准测试被打包在测试程序JAR文件中，通过无参调用JAR文件可以得到其列表
——《Hadoop权威指南》

查看信息

bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.ja

执行结果

[root@hadoop-master hadoop-2.7.0]# bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar 
An example program must be given as the first argument.
Valid program names are:
  DFSCIOTest: Distributed i/o benchmark of libhdfs.
  DistributedFSCheck: Distributed checkup of the file system consistency.
  JHLogAnalyzer: Job History Log analyzer.
  MRReliabilityTest: A program that tests the reliability of the MR framework by injecting faults/failures
  NNdataGenerator: Generate the data to be used by NNloadGenerator
  NNloadGenerator: Generate load on Namenode using NN loadgenerator run WITHOUT MR
  NNloadGeneratorMR: Generate load on Namenode using NN loadgenerator run as MR job
  NNstructureGenerator: Generate the structure to be used by NNdataGenerator
  SliveTest: HDFS Stress Test and Live Data Verification.
  TestDFSIO: Distributed i/o benchmark.
  fail: a job that always fails
  filebench: Benchmark SequenceFile(Input|Output)Format (block,record compressed and uncompressed), Text(Input|Output)Format (compressed and uncompressed)
  largesorter: Large-Sort tester
  loadgen: Generic map/reduce load generator
  mapredtest: A map/reduce test check.
  minicluster: Single process HDFS and MR cluster.
  mrbench: A map/reduce benchmark that can create many small jobs
  nnbench: A benchmark that stresses the namenode.
  sleep: A job that sleeps at each map and reduce task.
  testbigmapoutput: A map/reduce program that works on a very big non-splittable file and does identity map/reduce
  testfilesystem: A test for FileSystem read/write.
  testmapredsort: A map/reduce program that validates the map-reduce framework's sort.
  testsequencefile: A test for flat files of binary key value pairs.
  testsequencefileinputformat: A test for sequence file input format.
  testtextinputformat: A test for text input format.
  threadedmapbench: A map/reduce benchmark that compares the performance of maps with multiple spills over maps with 1 spill

写入速度测试

测试写入10个10M文件

bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 10MB

[root@hadoop-master hadoop-2.7.0]# bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 10MB
16/08/05 10:11:42 INFO fs.TestDFSIO: TestDFSIO.1.8
16/08/05 10:11:42 INFO fs.TestDFSIO: nrFiles = 10
16/08/05 10:11:42 INFO fs.TestDFSIO: nrBytes (MB) = 10.0
16/08/05 10:11:42 INFO fs.TestDFSIO: bufferSize = 1000000
16/08/05 10:11:42 INFO fs.TestDFSIO: baseDir = /benchmarks/TestDFSIO
16/08/05 10:11:43 INFO fs.TestDFSIO: creating control file: 10485760 bytes, 10 files
16/08/05 10:11:44 INFO fs.TestDFSIO: created control files for: 10 files
16/08/05 10:11:45 INFO client.RMProxy: Connecting to ResourceManager at hadoop-master/192.168.20.141:8032
16/08/05 10:11:45 INFO client.RMProxy: Connecting to ResourceManager at hadoop-master/192.168.20.141:8032
16/08/05 10:11:46 INFO mapred.FileInputFormat: Total input paths to process : 10
16/08/05 10:11:46 INFO mapreduce.JobSubmitter: number of splits:10
16/08/05 10:11:46 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1470403413050_0001
16/08/05 10:11:46 INFO impl.YarnClientImpl: Submitted application application_1470403413050_0001
16/08/05 10:11:47 INFO mapreduce.Job: The url to track the job: http://hadoop-master:8088/proxy/application_1470403413050_0001/
16/08/05 10:11:47 INFO mapreduce.Job: Running job: job_1470403413050_0001
16/08/05 10:11:57 INFO mapreduce.Job: Job job_1470403413050_0001 running in uber mode : false
16/08/05 10:11:57 INFO mapreduce.Job:  map 0% reduce 0%
16/08/05 10:12:27 INFO mapreduce.Job:  map 40% reduce 0%
16/08/05 10:12:53 INFO mapreduce.Job:  map 47% reduce 13%
16/08/05 10:12:54 INFO mapreduce.Job:  map 53% reduce 13%
16/08/05 10:13:03 INFO mapreduce.Job:  map 60% reduce 13%
16/08/05 10:13:04 INFO mapreduce.Job:  map 67% reduce 13%
16/08/05 10:13:06 INFO mapreduce.Job:  map 80% reduce 13%
16/08/05 10:13:08 INFO mapreduce.Job:  map 83% reduce 13%
16/08/05 10:13:10 INFO mapreduce.Job:  map 90% reduce 13%
16/08/05 10:13:11 INFO mapreduce.Job:  map 100% reduce 13%
16/08/05 10:13:12 INFO mapreduce.Job:  map 100% reduce 17%
16/08/05 10:13:18 INFO mapreduce.Job:  map 100% reduce 20%
16/08/05 10:13:30 INFO mapreduce.Job:  map 100% reduce 67%
16/08/05 10:13:36 INFO mapreduce.Job:  map 100% reduce 100%
16/08/05 10:13:37 INFO mapreduce.Job: Job job_1470403413050_0001 completed successfully
16/08/05 10:13:37 INFO mapreduce.Job: Counters: 51
    File System Counters
        FILE: Number of bytes read=840
        FILE: Number of bytes written=1269539
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=2390
        HDFS: Number of bytes written=104857676
        HDFS: Number of read operations=43
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=12
    Job Counters 
        Killed map tasks=2
        Launched map tasks=13
        Launched reduce tasks=1
        Data-local map tasks=11
        Rack-local map tasks=2
        Total time spent by all maps in occupied slots (ms)=646035
        Total time spent by all reduces in occupied slots (ms)=66302
        Total time spent by all map tasks (ms)=646035
        Total time spent by all reduce tasks (ms)=66302
        Total vcore-seconds taken by all map tasks=646035
        Total vcore-seconds taken by all reduce tasks=66302
        Total megabyte-seconds taken by all map tasks=661539840
        Total megabyte-seconds taken by all reduce tasks=67893248
    Map-Reduce Framework
        Map input records=10
        Map output records=50
        Map output bytes=734
        Map output materialized bytes=894
        Input split bytes=1270
        Combine input records=0
        Combine output records=0
        Reduce input groups=5
        Reduce shuffle bytes=894
        Reduce input records=50
        Reduce output records=5
        Spilled Records=100
        Shuffled Maps =10
        Failed Shuffles=0
        Merged Map outputs=10
        GC time elapsed (ms)=8218
        CPU time spent (ms)=15840
        Physical memory (bytes) snapshot=2184921088
        Virtual memory (bytes) snapshot=9295441920
        Total committed heap usage (bytes)=1374396416
    Shuffle Errors
        BAD_ID=0
        CONNECTION=0
        IO_ERROR=0
        WRONG_LENGTH=0
        WRONG_MAP=0
        WRONG_REDUCE=0
    File Input Format Counters 
        Bytes Read=1120
    File Output Format Counters 
        Bytes Written=76
16/08/05 10:13:37 WARN hdfs.DFSClient: DFSInputStream has been closed already
16/08/05 10:13:37 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write
16/08/05 10:13:37 INFO fs.TestDFSIO:            Date & time: Fri Aug 05 10:13:37 EDT 2016
16/08/05 10:13:37 INFO fs.TestDFSIO:        Number of files: 10
16/08/05 10:13:37 INFO fs.TestDFSIO: Total MBytes processed: 100.0
16/08/05 10:13:37 INFO fs.TestDFSIO:      Throughput mb/sec: 1.4243796826482067
16/08/05 10:13:37 INFO fs.TestDFSIO: Average IO rate mb/sec: 6.660604000091553
16/08/05 10:13:37 INFO fs.TestDFSIO:  IO rate std deviation: 8.936692949846902
16/08/05 10:13:37 INFO fs.TestDFSIO:     Test exec time sec: 112.884
16/08/05 10:13:37 INFO fs.TestDFSIO:

测试读文件速度

bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar TestDFSIO -read -nrFiles 10 -fileSize 10MB

执行结果

[root@hadoop-master hadoop-2.7.0]# bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar TestDFSIO -read -nrFiles 10 -fileSize 10MB
16/08/05 10:25:55 INFO fs.TestDFSIO: TestDFSIO.1.8
16/08/05 10:25:55 INFO fs.TestDFSIO: nrFiles = 10
16/08/05 10:25:55 INFO fs.TestDFSIO: nrBytes (MB) = 10.0
16/08/05 10:25:55 INFO fs.TestDFSIO: bufferSize = 1000000
16/08/05 10:25:55 INFO fs.TestDFSIO: baseDir = /benchmarks/TestDFSIO
16/08/05 10:25:56 INFO fs.TestDFSIO: creating control file: 10485760 bytes, 10 files
16/08/05 10:25:57 INFO fs.TestDFSIO: created control files for: 10 files
16/08/05 10:25:57 INFO client.RMProxy: Connecting to ResourceManager at hadoop-master/192.168.20.141:8032
16/08/05 10:25:57 INFO client.RMProxy: Connecting to ResourceManager at hadoop-master/192.168.20.141:8032
16/08/05 10:25:58 INFO mapred.FileInputFormat: Total input paths to process : 10
16/08/05 10:25:58 INFO mapreduce.JobSubmitter: number of splits:10
16/08/05 10:25:59 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1470403413050_0002
16/08/05 10:25:59 INFO impl.YarnClientImpl: Submitted application application_1470403413050_0002
16/08/05 10:25:59 INFO mapreduce.Job: The url to track the job: http://hadoop-master:8088/proxy/application_1470403413050_0002/
16/08/05 10:25:59 INFO mapreduce.Job: Running job: job_1470403413050_0002
16/08/05 10:26:07 INFO mapreduce.Job: Job job_1470403413050_0002 running in uber mode : false
16/08/05 10:26:07 INFO mapreduce.Job:  map 0% reduce 0%
16/08/05 10:26:36 INFO mapreduce.Job:  map 20% reduce 0%
16/08/05 10:26:37 INFO mapreduce.Job:  map 30% reduce 0%
16/08/05 10:26:38 INFO mapreduce.Job:  map 50% reduce 0%
16/08/05 10:26:39 INFO mapreduce.Job:  map 100% reduce 0%
16/08/05 10:26:44 INFO mapreduce.Job:  map 100% reduce 100%
16/08/05 10:26:46 INFO mapreduce.Job: Job job_1470403413050_0002 completed successfully
16/08/05 10:26:46 INFO mapreduce.Job: Counters: 49
    File System Counters
        FILE: Number of bytes read=832
        FILE: Number of bytes written=1269501
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=104859990
        HDFS: Number of bytes written=77
        HDFS: Number of read operations=53
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=2
    Job Counters 
        Launched map tasks=10
        Launched reduce tasks=1
        Data-local map tasks=10
        Total time spent by all maps in occupied slots (ms)=291173
        Total time spent by all reduces in occupied slots (ms)=5582
        Total time spent by all map tasks (ms)=291173
        Total time spent by all reduce tasks (ms)=5582
        Total vcore-seconds taken by all map tasks=291173
        Total vcore-seconds taken by all reduce tasks=5582
        Total megabyte-seconds taken by all map tasks=298161152
        Total megabyte-seconds taken by all reduce tasks=5715968
    Map-Reduce Framework
        Map input records=10
        Map output records=50
        Map output bytes=726
        Map output materialized bytes=886
        Input split bytes=1270
        Combine input records=0
        Combine output records=0
        Reduce input groups=5
        Reduce shuffle bytes=886
        Reduce input records=50
        Reduce output records=5
        Spilled Records=100
        Shuffled Maps =10
        Failed Shuffles=0
        Merged Map outputs=10
        GC time elapsed (ms)=3177
        CPU time spent (ms)=7570
        Physical memory (bytes) snapshot=2147774464
        Virtual memory (bytes) snapshot=9255862272
        Total committed heap usage (bytes)=1374396416
    Shuffle Errors
        BAD_ID=0
        CONNECTION=0
        IO_ERROR=0
        WRONG_LENGTH=0
        WRONG_MAP=0
        WRONG_REDUCE=0
    File Input Format Counters 
        Bytes Read=1120
    File Output Format Counters 
        Bytes Written=77
16/08/05 10:26:46 WARN hdfs.DFSClient: DFSInputStream has been closed already
16/08/05 10:26:46 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
16/08/05 10:26:46 INFO fs.TestDFSIO:            Date & time: Fri Aug 05 10:26:46 EDT 2016
16/08/05 10:26:46 INFO fs.TestDFSIO:        Number of files: 10
16/08/05 10:26:46 INFO fs.TestDFSIO: Total MBytes processed: 100.0
16/08/05 10:26:46 INFO fs.TestDFSIO:      Throughput mb/sec: 29.197080291970803
16/08/05 10:26:46 INFO fs.TestDFSIO: Average IO rate mb/sec: 47.35454177856445
16/08/05 10:26:46 INFO fs.TestDFSIO:  IO rate std deviation: 29.781282953365924
16/08/05 10:26:46 INFO fs.TestDFSIO:     Test exec time sec: 48.942
16/08/05 10:26:46 INFO fs.TestDFSIO:

命令查看

安装目录执行

cat TestDFSIO_results.log

执行结果

[root@hadoop-master hadoop-2.7.0]# cat TestDFSIO_results.log 
----- TestDFSIO ----- : write
           Date & time: Fri Aug 05 10:13:37 EDT 2016
       Number of files: 10
Total MBytes processed: 100.0
     Throughput mb/sec: 1.4243796826482067
Average IO rate mb/sec: 6.660604000091553
 IO rate std deviation: 8.936692949846902
    Test exec time sec: 112.884

----- TestDFSIO ----- : read
           Date & time: Fri Aug 05 10:26:46 EDT 2016
       Number of files: 10
Total MBytes processed: 100.0
     Throughput mb/sec: 29.197080291970803
Average IO rate mb/sec: 47.35454177856445
 IO rate std deviation: 29.781282953365924
    Test exec time sec: 48.942

web查看

http://hadoop-master:50070/

清除测试数据

将测试数据删除

bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar TestDFSIO -clean

单词统计测试

新建一个words.txt

vim words.txt

内容

hello hadoop hbase mytest
hadoop-node1
hadoop-master
hadoop-node2
this is my test

上传文件

bin/hadoop fs -put words.txt /tmp/

使用mapreduce统计单词个数
统计指定文件单词个数，并将结果输入到指定文件

bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar wordcount /tmp/words.txt /tmp/words_result.txt

结果 

hadoop  1
hadoop-master   1
hadoop-node1     1
hadoop-node2     1
hbase   1
hello   1
is  1
my  1
mytest  1
test    1
this    1

代码查看
查看根目录

bin/hadoop fs -ls /

查看指定文件

bin/hadoop fs -cat hdfs:///tmp/words_result.txt/part-r-00000

执行结果

[root@hadoop-master hadoop-2.7.0]# bin/hadoop fs -cat hdfs:///tmp/words_result.txt/part-r-00000
hadoop  1
hadoop-master   1
hadoop-node1    1
hadoop-node2    1
hbase   1
hello   1
is  1
my  1
mytest  1
test    1
this    1

更多Hadoop2.7.0常用操作HDFS的Shell命令

http://www.cuiweiyou.com/1405.html

参考

http://blog.itpub.net/25854343/viewspace-1425183/

-贫寒豌豆
关注
专栏目录
Hadoop——测试代码
Devin
04-23 958
1、本地测试代码class WCMap extends Mapper<LongWritable, Text, Text, IntWritable>{ /** * key: 偏移量 * value:每行文本内容 * context：输出内容上下文 */ @Override protected void map(LongWritable key, Text value,Contex...
hadoop基准测试
Hadoop工作室
10-12 1489
一、测试条件 集群完全安装配置后，应立即开始基准测试。基准测试时集群里不应该运行其他一切任务。 二、测试目标 1. 硬盘故障：新系统最常见的故障。可以通过运行高强度的IO基准测试程序集中测试。例如TestDFSIO 2. MapReduce的性能 三、测试方法 1、TestDFSIO基准测试HDFS 测试顺序应该是先写测试后读测试 写测试： 使用10个map任务写10个文件
hadoop测试代码
11-21
centos7安装hadoop2.6.0后，使用eclipse安装mapreduce插件，并测试是否正常使用
架构师-测试精确度排名(真实的程序、核心程序、小型基准程序、合成基准程序)
技术博客
09-24 312
按基准程序的构造特点可以把它划分成四类，即：核心程序、小基准程序、合成基准程序以及基准测试程序组.真实程序、核心程序、小型基准程序和合成基准程序，其评测准确程度依次递减。解析：核心程序测试的精确度高于小型基准程序，但低于真实的程序。以下哪种程序测试的精确度高于小型基准程序，但低于真实的程序？解析：真实的程序测试最能反映真实的使用场景，因而最真实。核心程序测试与真实的程序测试相比，哪种更具精确性？解析：真实的程序测试精确度高于核心程序测试。哪种程序测试的精确度次于真实的程序？以下哪种测试类型的精确度最高？
Hadoop基准测试
weixin_35994859的博客
12-07 158
基准测试 为了更加全面的了解系统，我们需要通过测试，发现瓶颈所在。通过测试，可以用来分析系统的性能。基准测试被打包在测试程序JAR文件中，通过无参调用JAR文件可以得到其列表 IO写操作 命令： hadoop jar /opt/module/hadoop-2.7.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tes...
【大数据Hadoop】Hadoop源代码调试（持续更新...）
如切如磋，如琢如磨，臻于至善。
01-29 270
hadoop 源代码 各个组件的调试，分析源代码，跟踪源代码，解析源代码，理解源代码，熟读源代码
MapReduce简介
intmiss的博客
10-29 987
MapReduce MapReduce是一种编程模型，用于大规模数据集（大于1TB）的并行运算。概念"Map（映射）“和"Reduce（归约）”，是它们的主要思想，都是从函数式编程语言里借来的，还有从矢量编程语言里借来的特性。它极大地方便了编程人员在不会分布式并行编程的情况下，将自己的程序运行在分布式系统上。 当前的软件实现是指定一个Map（映射）函数，用来把一组键值对映射成一组新的键值对，指定并...
Hadoop2.7.0学习——Windows下hadoop-eclipse-plugin-2.7.0插件安装-附件资源
03-02
Hadoop2.7.0学习——Windows下hadoop-eclipse-plugin-2.7.0插件安装-附件资源
hadoop2.7.0 windows
09-21
Hadoop 2.7.0 是一个开源框架，主要用于分布式存储和计算，是大数据处理领域的重要工具。在Windows环境下安装和使用Hadoop可能与在Linux上有所不同，但同样可以实现强大的功能。本教程将详细解释如何在Windows系统上...
hadoop-2.7.0.tar
最新发布
09-25
hadoop-2.7.0.tar
hadoop安装与配置（本地模式）
lihuan1124的博客
06-22 1220
hadoop版本:hadoop-2.7.2虚拟机安装前期配置：IP地址、IP与主机映射、关闭防火墙一、安装java jdk   1、下载java jdk1.8版本，放在/home/software目录下，   2、解压：tar -zxvf java-jdk*****.jar   3、vim ~/.bash_profile12345#添加export JAVA_HOME=root/app/jdk1....
linux运行hadoop实例wordcount详解
DaiDaiHan1的专栏
02-24 1736
在服务器上配置了hadoop准备自己从wordcount.java开始到跑起wordcount程序，但是就这简单的几步遇到了很多问题，现在总结一下我碰到的问题： 整个过程步骤是： 1.首先创建文件目录： mkdir  Test 2.编译wordcount.java文件生成.class文件，存放在第一步创建的文件夹Test中： javac  -classpath /hadoop*core*
MapReduce 计数器简介
阿龙_的专栏
06-11 1041
1、计数器简介 在许多情况下，一个用户需要了解待分析的数据，尽管这并非所要执行的分析任务 的核心内容。以统计数据集中无效记录数目的任务为例，如果发现无效记录的比例 相当高，那么就需要认真思考为何存在如此多无效记录。是所采用的检测程序存在 缺陷，还是数据集质量确实很低，包含大量无效记录？如果确定是数据集的质量问 题，则可能需要扩大数据集的规模，以增大有效记录的比例，从而进行有意义的 分析。 
执行 bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount /user/input出错
热门推荐
simply
05-26 1万+
[root@node1 hadoop-2.7.3]# bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount /user/input /user/output 19/05/26 09:44:33 INFO client.RMProxy: Connecting to ResourceMan...
hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.5.jar grep input output
yuantuiwan3221的博客
06-04 1254
问题：hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.5.jar grep input output文件系统不相通 解决方式：1、Linux本地文件系统和hadoop的hdfs文件系统是通过hadoop fs -put/get等联系。 2、可以通过file:/// 和hdfs://localhost:9000/user/ 直接说明地址区分不同的文件系统 hadoop jar share/hadoop/mapreduce/h
hadoop基准性能测试
dgqg1223
07-17 297
HDFS写性能 测试内容：向HDFS集群写10个128m的文件 hadoop jar /opt/hadoop-2.7.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 128MB 20/07/17 08:09:34 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write 20/07/17
eclipse开发hadoop环境搭建
天高任鸟飞，海阔凭鱼跃
02-13 275
[b]Hadoop2.6.0集群搭建完毕后，下面介绍一下eclipse是如何开发hadoop程序（即MapReduce程序）的。[/b] 1.jdk安装hadoop集群的搭建，不再详述，参考[url]http://kevin12.iteye.com/blog/2273556[/url]； 下面运行下hadoop自带的wordcount例子： 2.先将hadoop-2.6.0目录下面的READ...
hadoop--基压测试
没有合适的昵称
02-09 1601
在安装完集群后,我们都需要先对集群做一些测试,下面讲解测试读写的性能 写性能 包名: Apache: hadoop-mapreduce-client-jobclient-2.7.5-tests.jar CDH: hadoop-mapreduce-client-jobclient-3.0.0-cdh6.2.0-tests.jar 包路径: /home/hadoop-jrq/bigdata/hadoo...
Hadoop2.7.0双节点集群搭建教程与通信配置
- Hadoop2.7.0采用主备NameNode架构，一个作为active节点提供服务，另一个standby节点用于备份并保持同步。 - 使用QJM（Quorum Journal Manager）方式实现HDFS元数据一致性，确保数据在多数JournalNode中写入成功...
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

-贫寒豌豆

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值