Big Data 2-- 单机配置hadoop

最新推荐文章于 2024-09-25 15:58:18 发布
最新推荐文章于 2024-09-25 15:58:18 发布
阅读量609 收藏
点赞数
分类专栏： big data 文章标签： hadoop
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/robin622/article/details/25943249
版权

1, 找到一个想要安装的hadoop版本, 例如http://hadoop.apache.org/docs/r1.2.1/。

2，下载并解压： tar -zxvf XXXXX.

3, 安装jdk, 和ssh软件（如果之前没安装的话）。

4， 编辑conf/hadoop-env.sh, 指定java_home。

5，Setup passphraseless ssh

Now check that you can ssh to the localhost without a passphrase:
$ ssh localhost

If you cannot ssh to localhost without a passphrase, execute the following commands:
$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

6, $ mkdir input
$ cp conf/*.xml input
$ bin/hadoop jar hadoop-examples-*.jar grep input output 'dfs[a-z.]+'
$ cat output/*


7, The result is:

14/05/16 07:21:23 INFO util.NativeCodeLoader: Loaded the native-hadoop library
14/05/16 07:21:23 WARN snappy.LoadSnappy: Snappy native library not loaded
14/05/16 07:21:23 INFO mapred.FileInputFormat: Total input paths to process : 7
14/05/16 07:21:24 INFO mapred.JobClient: Running job: job_local1725642642_0001
14/05/16 07:21:24 INFO mapred.LocalJobRunner: Waiting for map tasks
14/05/16 07:21:24 INFO mapred.LocalJobRunner: Starting task: attempt_local1725642642_0001_m_000000_0
14/05/16 07:21:24 INFO util.ProcessTree: setsid exited with exit code 0
14/05/16 07:21:24 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2ea45536
14/05/16 07:21:24 INFO mapred.MapTask: Processing split: file:/home/wezhao/Downloads/hadoop-1.2.1/input/capacity-scheduler.xml:0+7457
14/05/16 07:21:24 INFO mapred.MapTask: numReduceTasks: 1
14/05/16 07:21:24 INFO mapred.MapTask: io.sort.mb = 100
14/05/16 07:21:24 INFO mapred.MapTask: data buffer = 79691776/99614720
14/05/16 07:21:24 INFO mapred.MapTask: record buffer = 262144/327680
14/05/16 07:21:24 INFO mapred.MapTask: Starting flush of map output
14/05/16 07:21:24 INFO mapred.Task: Task:attempt_local1725642642_0001_m_000000_0 is done. And is in the process of commiting
14/05/16 07:21:24 INFO mapred.LocalJobRunner: file:/home/wezhao/Downloads/hadoop-1.2.1/input/capacity-scheduler.xml:0+7457
14/05/16 07:21:24 INFO mapred.Task: Task 'attempt_local1725642642_0001_m_000000_0' done.
14/05/16 07:21:24 INFO mapred.LocalJobRunner: Finishing task: attempt_local1725642642_0001_m_000000_0
14/05/16 07:21:24 INFO mapred.LocalJobRunner: Starting task: attempt_local1725642642_0001_m_000001_0
14/05/16 07:21:25 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@76933bcb
14/05/16 07:21:25 INFO mapred.MapTask: Processing split: file:/home/wezhao/Downloads/hadoop-1.2.1/input/hadoop-policy.xml:0+4644
14/05/16 07:21:25 INFO mapred.MapTask: numReduceTasks: 1
14/05/16 07:21:25 INFO mapred.MapTask: io.sort.mb = 100
14/05/16 07:21:25 INFO mapred.MapTask: data buffer = 79691776/99614720
14/05/16 07:21:25 INFO mapred.MapTask: record buffer = 262144/327680
14/05/16 07:21:25 INFO mapred.MapTask: Starting flush of map output
14/05/16 07:21:25 INFO mapred.MapTask: Finished spill 0
14/05/16 07:21:25 INFO mapred.Task: Task:attempt_local1725642642_0001_m_000001_0 is done. And is in the process of commiting
14/05/16 07:21:25 INFO mapred.LocalJobRunner: file:/home/wezhao/Downloads/hadoop-1.2.1/input/hadoop-policy.xml:0+4644
14/05/16 07:21:25 INFO mapred.Task: Task 'attempt_local1725642642_0001_m_000001_0' done.
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Finishing task: attempt_local1725642642_0001_m_000001_0
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Starting task: attempt_local1725642642_0001_m_000002_0
14/05/16 07:21:25 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7ed75415
14/05/16 07:21:25 INFO mapred.MapTask: Processing split: file:/home/wezhao/Downloads/hadoop-1.2.1/input/mapred-queue-acls.xml:0+2033
14/05/16 07:21:25 INFO mapred.MapTask: numReduceTasks: 1
14/05/16 07:21:25 INFO mapred.MapTask: io.sort.mb = 100
14/05/16 07:21:25 INFO mapred.MapTask: data buffer = 79691776/99614720
14/05/16 07:21:25 INFO mapred.MapTask: record buffer = 262144/327680
14/05/16 07:21:25 INFO mapred.MapTask: Starting flush of map output
14/05/16 07:21:25 INFO mapred.Task: Task:attempt_local1725642642_0001_m_000002_0 is done. And is in the process of commiting
14/05/16 07:21:25 INFO mapred.LocalJobRunner: file:/home/wezhao/Downloads/hadoop-1.2.1/input/mapred-queue-acls.xml:0+2033
14/05/16 07:21:25 INFO mapred.Task: Task 'attempt_local1725642642_0001_m_000002_0' done.
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Finishing task: attempt_local1725642642_0001_m_000002_0
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Starting task: attempt_local1725642642_0001_m_000003_0
14/05/16 07:21:25 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@561777b1
14/05/16 07:21:25 INFO mapred.MapTask: Processing split: file:/home/wezhao/Downloads/hadoop-1.2.1/input/fair-scheduler.xml:0+327
14/05/16 07:21:25 INFO mapred.MapTask: numReduceTasks: 1
14/05/16 07:21:25 INFO mapred.MapTask: io.sort.mb = 100
14/05/16 07:21:25 INFO mapred.MapTask: data buffer = 79691776/99614720
14/05/16 07:21:25 INFO mapred.MapTask: record buffer = 262144/327680
14/05/16 07:21:25 INFO mapred.MapTask: Starting flush of map output
14/05/16 07:21:25 INFO mapred.JobClient:  map 42% reduce 0%
14/05/16 07:21:25 INFO mapred.Task: Task:attempt_local1725642642_0001_m_000003_0 is done. And is in the process of commiting
14/05/16 07:21:25 INFO mapred.LocalJobRunner: file:/home/wezhao/Downloads/hadoop-1.2.1/input/fair-scheduler.xml:0+327
14/05/16 07:21:25 INFO mapred.Task: Task 'attempt_local1725642642_0001_m_000003_0' done.
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Finishing task: attempt_local1725642642_0001_m_000003_0
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Starting task: attempt_local1725642642_0001_m_000004_0
14/05/16 07:21:25 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7c9e67a
14/05/16 07:21:25 INFO mapred.MapTask: Processing split: file:/home/wezhao/Downloads/hadoop-1.2.1/input/core-site.xml:0+296
14/05/16 07:21:25 INFO mapred.MapTask: numReduceTasks: 1
14/05/16 07:21:25 INFO mapred.MapTask: io.sort.mb = 100
14/05/16 07:21:25 INFO mapred.MapTask: data buffer = 79691776/99614720
14/05/16 07:21:25 INFO mapred.MapTask: record buffer = 262144/327680
14/05/16 07:21:25 INFO mapred.MapTask: Starting flush of map output
14/05/16 07:21:25 INFO mapred.Task: Task:attempt_local1725642642_0001_m_000004_0 is done. And is in the process of commiting
14/05/16 07:21:25 INFO mapred.LocalJobRunner: file:/home/wezhao/Downloads/hadoop-1.2.1/input/core-site.xml:0+296
14/05/16 07:21:25 INFO mapred.Task: Task 'attempt_local1725642642_0001_m_000004_0' done.
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Finishing task: attempt_local1725642642_0001_m_000004_0
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Starting task: attempt_local1725642642_0001_m_000005_0
14/05/16 07:21:25 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7fd88db7
14/05/16 07:21:25 INFO mapred.MapTask: Processing split: file:/home/wezhao/Downloads/hadoop-1.2.1/input/mapred-site.xml:0+292
14/05/16 07:21:25 INFO mapred.MapTask: numReduceTasks: 1
14/05/16 07:21:25 INFO mapred.MapTask: io.sort.mb = 100
14/05/16 07:21:25 INFO mapred.MapTask: data buffer = 79691776/99614720
14/05/16 07:21:25 INFO mapred.MapTask: record buffer = 262144/327680
14/05/16 07:21:25 INFO mapred.MapTask: Starting flush of map output
14/05/16 07:21:25 INFO mapred.Task: Task:attempt_local1725642642_0001_m_000005_0 is done. And is in the process of commiting
14/05/16 07:21:25 INFO mapred.LocalJobRunner: file:/home/wezhao/Downloads/hadoop-1.2.1/input/mapred-site.xml:0+292
14/05/16 07:21:25 INFO mapred.Task: Task 'attempt_local1725642642_0001_m_000005_0' done.
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Finishing task: attempt_local1725642642_0001_m_000005_0
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Starting task: attempt_local1725642642_0001_m_000006_0
14/05/16 07:21:25 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6d7e845a
14/05/16 07:21:25 INFO mapred.MapTask: Processing split: file:/home/wezhao/Downloads/hadoop-1.2.1/input/hdfs-site.xml:0+274
14/05/16 07:21:25 INFO mapred.MapTask: numReduceTasks: 1
14/05/16 07:21:25 INFO mapred.MapTask: io.sort.mb = 100
14/05/16 07:21:25 INFO mapred.MapTask: data buffer = 79691776/99614720
14/05/16 07:21:25 INFO mapred.MapTask: record buffer = 262144/327680
14/05/16 07:21:25 INFO mapred.MapTask: Starting flush of map output
14/05/16 07:21:25 INFO mapred.MapTask: Finished spill 0
14/05/16 07:21:25 INFO mapred.Task: Task:attempt_local1725642642_0001_m_000006_0 is done. And is in the process of commiting
14/05/16 07:21:25 INFO mapred.LocalJobRunner: file:/home/wezhao/Downloads/hadoop-1.2.1/input/hdfs-site.xml:0+274
14/05/16 07:21:25 INFO mapred.Task: Task 'attempt_local1725642642_0001_m_000006_0' done.
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Finishing task: attempt_local1725642642_0001_m_000006_0
14/05/16 07:21:25 INFO mapred.LocalJobRunner: Map task executor complete.
14/05/16 07:21:25 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@18987a33
14/05/16 07:21:25 INFO mapred.LocalJobRunner:
14/05/16 07:21:25 INFO mapred.Merger: Merging 7 sorted segments
14/05/16 07:21:25 INFO mapred.Merger: Down to the last merge-pass, with 2 segments left of total size: 49 bytes
14/05/16 07:21:25 INFO mapred.LocalJobRunner:
14/05/16 07:21:25 INFO mapred.Task: Task:attempt_local1725642642_0001_r_000000_0 is done. And is in the process of commiting
14/05/16 07:21:25 INFO mapred.LocalJobRunner:
14/05/16 07:21:25 INFO mapred.Task: Task attempt_local1725642642_0001_r_000000_0 is allowed to commit now
14/05/16 07:21:25 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local1725642642_0001_r_000000_0' to file:/home/wezhao/Downloads/hadoop-1.2.1/grep-temp-1256449998
14/05/16 07:21:25 INFO mapred.LocalJobRunner: reduce > reduce
14/05/16 07:21:25 INFO mapred.Task: Task 'attempt_local1725642642_0001_r_000000_0' done.
14/05/16 07:21:26 INFO mapred.JobClient:  map 100% reduce 100%
14/05/16 07:21:26 INFO mapred.JobClient: Job complete: job_local1725642642_0001
14/05/16 07:21:26 INFO mapred.JobClient: Counters: 21
14/05/16 07:21:26 INFO mapred.JobClient:   File Input Format Counters
14/05/16 07:21:26 INFO mapred.JobClient:     Bytes Read=15323
14/05/16 07:21:26 INFO mapred.JobClient:   File Output Format Counters
14/05/16 07:21:26 INFO mapred.JobClient:     Bytes Written=155
14/05/16 07:21:26 INFO mapred.JobClient:   FileSystemCounters
14/05/16 07:21:26 INFO mapred.JobClient:     FILE_BYTES_READ=1278506
14/05/16 07:21:26 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=1572630
14/05/16 07:21:26 INFO mapred.JobClient:   Map-Reduce Framework
14/05/16 07:21:26 INFO mapred.JobClient:     Map output materialized bytes=87
14/05/16 07:21:26 INFO mapred.JobClient:     Map input records=378
14/05/16 07:21:26 INFO mapred.JobClient:     Reduce shuffle bytes=0
14/05/16 07:21:26 INFO mapred.JobClient:     Spilled Records=4
14/05/16 07:21:26 INFO mapred.JobClient:     Map output bytes=41
14/05/16 07:21:26 INFO mapred.JobClient:     Total committed heap usage (bytes)=1204920320
14/05/16 07:21:26 INFO mapred.JobClient:     CPU time spent (ms)=0
14/05/16 07:21:26 INFO mapred.JobClient:     Map input bytes=15323
14/05/16 07:21:26 INFO mapred.JobClient:     SPLIT_RAW_BYTES=819
14/05/16 07:21:26 INFO mapred.JobClient:     Combine input records=2
14/05/16 07:21:26 INFO mapred.JobClient:     Reduce input records=2
14/05/16 07:21:26 INFO mapred.JobClient:     Reduce input groups=2
14/05/16 07:21:26 INFO mapred.JobClient:     Combine output records=2
14/05/16 07:21:26 INFO mapred.JobClient:     Physical memory (bytes) snapshot=0
14/05/16 07:21:26 INFO mapred.JobClient:     Reduce output records=2
14/05/16 07:21:26 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=0
14/05/16 07:21:26 INFO mapred.JobClient:     Map output records=2
14/05/16 07:21:26 INFO mapred.FileInputFormat: Total input paths to process : 1
14/05/16 07:21:26 INFO mapred.JobClient: Running job: job_local162864386_0002
14/05/16 07:21:26 INFO mapred.LocalJobRunner: Waiting for map tasks
14/05/16 07:21:26 INFO mapred.LocalJobRunner: Starting task: attempt_local162864386_0002_m_000000_0
14/05/16 07:21:26 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@10f102d3
14/05/16 07:21:26 INFO mapred.MapTask: Processing split: file:/home/wezhao/Downloads/hadoop-1.2.1/grep-temp-1256449998/part-00000:0+143
14/05/16 07:21:26 INFO mapred.MapTask: numReduceTasks: 1
14/05/16 07:21:26 INFO mapred.MapTask: io.sort.mb = 100
14/05/16 07:21:26 INFO mapred.MapTask: data buffer = 79691776/99614720
14/05/16 07:21:26 INFO mapred.MapTask: record buffer = 262144/327680
14/05/16 07:21:26 INFO mapred.MapTask: Starting flush of map output
14/05/16 07:21:26 INFO mapred.MapTask: Finished spill 0
14/05/16 07:21:26 INFO mapred.Task: Task:attempt_local162864386_0002_m_000000_0 is done. And is in the process of commiting
14/05/16 07:21:26 INFO mapred.LocalJobRunner: file:/home/wezhao/Downloads/hadoop-1.2.1/grep-temp-1256449998/part-00000:0+143
14/05/16 07:21:26 INFO mapred.Task: Task 'attempt_local162864386_0002_m_000000_0' done.
14/05/16 07:21:26 INFO mapred.LocalJobRunner: Finishing task: attempt_local162864386_0002_m_000000_0
14/05/16 07:21:26 INFO mapred.LocalJobRunner: Map task executor complete.
14/05/16 07:21:26 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@198e261d
14/05/16 07:21:26 INFO mapred.LocalJobRunner:
14/05/16 07:21:26 INFO mapred.Merger: Merging 1 sorted segments
14/05/16 07:21:26 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 47 bytes
14/05/16 07:21:26 INFO mapred.LocalJobRunner:
14/05/16 07:21:26 INFO mapred.Task: Task:attempt_local162864386_0002_r_000000_0 is done. And is in the process of commiting
14/05/16 07:21:26 INFO mapred.LocalJobRunner:
14/05/16 07:21:26 INFO mapred.Task: Task attempt_local162864386_0002_r_000000_0 is allowed to commit now
14/05/16 07:21:26 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local162864386_0002_r_000000_0' to file:/home/wezhao/Downloads/hadoop-1.2.1/output
14/05/16 07:21:26 INFO mapred.LocalJobRunner: reduce > reduce
14/05/16 07:21:26 INFO mapred.Task: Task 'attempt_local162864386_0002_r_000000_0' done.
14/05/16 07:21:27 INFO mapred.JobClient:  map 100% reduce 100%
14/05/16 07:21:27 INFO mapred.JobClient: Job complete: job_local162864386_0002
14/05/16 07:21:27 INFO mapred.JobClient: Counters: 21
14/05/16 07:21:27 INFO mapred.JobClient:   File Input Format Counters
14/05/16 07:21:27 INFO mapred.JobClient:     Bytes Read=155
14/05/16 07:21:27 INFO mapred.JobClient:   File Output Format Counters
14/05/16 07:21:27 INFO mapred.JobClient:     Bytes Written=41
14/05/16 07:21:27 INFO mapred.JobClient:   FileSystemCounters
14/05/16 07:21:27 INFO mapred.JobClient:     FILE_BYTES_READ=612459
14/05/16 07:21:27 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=783045
14/05/16 07:21:27 INFO mapred.JobClient:   Map-Reduce Framework
14/05/16 07:21:27 INFO mapred.JobClient:     Map output materialized bytes=51
14/05/16 07:21:27 INFO mapred.JobClient:     Map input records=2
14/05/16 07:21:27 INFO mapred.JobClient:     Reduce shuffle bytes=0
14/05/16 07:21:27 INFO mapred.JobClient:     Spilled Records=4
14/05/16 07:21:27 INFO mapred.JobClient:     Map output bytes=41
14/05/16 07:21:27 INFO mapred.JobClient:     Total committed heap usage (bytes)=262946816
14/05/16 07:21:27 INFO mapred.JobClient:     CPU time spent (ms)=0
14/05/16 07:21:27 INFO mapred.JobClient:     Map input bytes=57
14/05/16 07:21:27 INFO mapred.JobClient:     SPLIT_RAW_BYTES=125
14/05/16 07:21:27 INFO mapred.JobClient:     Combine input records=0
14/05/16 07:21:27 INFO mapred.JobClient:     Reduce input records=2
14/05/16 07:21:27 INFO mapred.JobClient:     Reduce input groups=1
14/05/16 07:21:27 INFO mapred.JobClient:     Combine output records=0
14/05/16 07:21:27 INFO mapred.JobClient:     Physical memory (bytes) snapshot=0
14/05/16 07:21:27 INFO mapred.JobClient:     Reduce output records=2
14/05/16 07:21:27 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=0
14/05/16 07:21:27 INFO mapred.JobClient:     Map output records=2


reference:

1, https://www.ibm.com/developerworks/community/blogs/theTechTrek/entry/test_driving_apache_hadoop_standalone_pseudo_distributed_mode1?lang=en

2, http://hadoop.apache.org/docs/r1.2.1/single_node_setup.html



robin622
关注
专栏目录
Mac安装hadoop--单机模式
amberwest
09-16 539
在mac上安装Hadoop3.3.1，可以使用HomeBrew快捷安装。 安装HomeBrew /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" 如果已经安装了HomeBrew，但更新失败可以参考其他做法： Mac-Homebrew-常见问题 | TimeMachine Notes 先删除core源，再重新brewtab关联，就可以更新bre..
spark-3.1.3-bin-without-hadoop.tgz
05-20
2. 配置环境变量：在`~/.bashrc`或`~/.bash_profile`中设置SPARK_HOME，并将Spark的bin目录添加到PATH。 3. 如果在Hadoop环境下，需要配置HADOOP_CONF_DIR指向Hadoop配置目录。 4. 启动Spark：可以启动单机模式，...
hive3.1.2安装 hadoop-3.2.3单机版安装
flyfelu的博客
05-16 1766
前置安装：jdk1.8和hadoop3.x.x 一、hadoop-3.2.3单机版安装 1.下载安装包 wget https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-3.2.3/hadoop-3.2.3.tar.gz 2.解压缩 tar -zxvf hadoop-3.2.3.tar.gz 3.进入hadoop-3.2.3/目录下，创建文件夹input,并将配置文件复制到该文件夹下 mkdir input cp.
在linux安装单机版hadoop-3.3.6
11-09 1244
进入etc/hadoop目录下，修改hadoop-env.sh文件。进入sbin目录下输入./start-all.sh。3.5、修改mapred-site.xml。3.4、修改 hdfs-site.xml。3.2、修改hadoop-env.sh。3.3、修改yarn-site.xml。3.6、修改core-site.xml。3.1、在hadoop目录下创建目录。三、修改hadoop对应的配置文件。2、配置hadoop环境变量。五、启动hadoop单机版。1、配置java环境变量。一、下载hadoop。
hadoop 单机式配置 + 伪分布式配置
Bernard's
08-13 1744
安装 非分布式配置（单机模式） summary # 把hadoop的所有配置xml文件作为输入，复制到新建的input文件夹里 mkdir ./input && cp /opt/software/hadoop-3.3.1/etc/hadoop/*.xml ./input # 运行grep例子，将input文件中的所有文件作为输入，筛选符合表达式dfs[a-z.]+的单词并统计出现的次数，输出到output文件夹 hadoop jar /opt/software/hadoop-3.3.1/sh
Hadoop【部署 02】hadoop-3.1.3 单机版YARN（配置、启动停止shell脚本修改及服务验证）
Java与大数据相关实践内容分享！
06-22 734
Hadoop【部署 02】hadoop-3.1.3 单机版YARN（配置、启动停止shell脚本修改及服务验证）
Hadoop-3.3.5 全分布式集群部署
好记性不如烂笔头
10-11 1112
一、前期准备 1、hadoop-3.2.0安装包 http://archive.apache.org/dist/hadoop/common/hadoop-3.2.0/hadoop-3.2.0.tar.gz 2、配置jdk1.8 vim /etc/profile #添加如下信息 export JAVA_HOME=jdk安装目录 export CLASSPATH=$JAVA_HOME/lib/ export PATH=$PATH:JAVA_HOME/bin 再保存执行　　 #source /etc/profil
1、hadoop3.1.4简单介绍及部署、简单验证
热门推荐
alanchanchn的专栏
04-23 5万+
Hadoop是Apache旗下的一个用java语言实现开源软件框架，是一个开发和运行处理大规模数据的软件平台。允许使用简单的编程模型在大量计算机集群上对大型数据集进行分布式处理。HDFS（分布式文件系统）：解决海量数据存储YARN（作业调度和集群资源管理的框架）：解决资源任务调度MAPREDUCE（分布式运算编程框架）：解决海量数据计算Hadoop通常是指一个更广泛的概念——Hadoop生态圈。
在Windows 10上安装单机版的hadoop-3.3.5
11-08 881
1、Hadoop是一个由Apache基金会所开发的分布式系统基础架构。用户可以不需要了解分布式底层细节的情况下，开发分布式程序。5.7、进入hadoop/sbin目录执行start-all.cmd。5.6、输入hdfs namenode -format格式化。2、下载Hadoop，我们在清华大学的镜像站下载。5.1、修改start-all.cmd中的配置。5.2、修改yarn-site.xml。单机版的hadoop就启动成功。4.2、配置hadoop环境。5、修改hadoop配置。5.9、启动成功后界面。
1、apache-kylin-3.1.3-bin-hadoop3介绍及部署、验证详解
alanchanchn的专栏
06-22 4万+
Apache Kylin™是一个开源的、分布式的分析型数据仓库，提供 Hadoop 之上的 SQL 查询接口及多维分析（OLAP）能力以支持超大规模数据，最初由eBay Inc.开发并贡献至开源社区。Apache Kylin™ 令使用者仅需三步，即可实现超大数据集上的亚秒级查询。定义数据集上的一个星形或雪花形模型在定义的数据表上构建cube使用标准 SQL 通过 ODBC、JDBC 或 RESTFUL API 进行查询，仅需亚秒级响应时间即可获得查询结果。
spark-3.1.3-bin-hadoop3.2.tgz
05-19
在解压并安装"spark-3.1.3-bin-hadoop3.2.tgz"后，你需要配置环境变量，如SPARK_HOME，然后可以通过启动Master和Worker节点来建立Spark集群。对于单机测试，可以使用本地模式。使用Spark时，你可以编写Python、Scala...
Linux运维-运维课程MP4频-06-大数据之Hadoop部署-17hadoop单机部署.mp4
06-06
Linux运维-运维课程MP4频-06-大数据之Hadoop部署-17hadoop单机部署.mp4
Linux运维-运维课程MP4频-06-大数据之Hadoop部署-18hadoop单机部署应用测试.mp4
06-06
Linux运维-运维课程MP4频-06-大数据之Hadoop部署-18hadoop单机部署应用测试.mp4
Linux运维-运维课程MP4频-06-大数据之Hadoop部署-16hadoop单机部署介绍及软件包获取.mp4
06-06
Linux运维-运维课程MP4频-06-大数据之Hadoop部署-16hadoop单机部署介绍及软件包获
Hadoop各组件工作流程
2301_77836489的博客
09-25 273
Hadoop各组件工作流程
WPF入门教学七 DockPanel与StackPanel的应用
Life is not divided
09-20 780
在WPF（Windows Presentation Foundation）中，DockPanel和StackPanel是两种常用的布局控件，它们各自具有独特的用途和特点。
WPF一个控件根据另一个控件的某种状态的改变从而改变自身某种状态
最新发布
m0_51005282的博客
09-25 126
让 Image 控件和 Label 控件在 xaml 中互换，这样当 Image 有图片时就会将 Label 控件给覆盖，从而达到让 Label 控件不可见的效果。使用 DataTrigger，根据值的变换时，来触发对应的操作，从而达到对应的效果。但是如果图片是透明的，用这种方法就不行。Image 控件有图时。Image 控件无图时。
大数据毕业设计选题推荐-超市进货推荐系统-Hive-Hadoop-Spark
IT研究室的博客
09-25 787
随着我国零售业的持续发展，超市成为了日常消费的主要场所之一。根据中国连锁经营协会发布的《2022年中国连锁百强报告》，2022年我国超市行业市场规模已超过4.5万亿元，市场竞争日趋激烈。面对日益复杂的市场环境和日益多样化的消费者需求，超市的运营与管理正变得更加依赖数据驱动的决策。如何通过精准进货、优化库存管理、合理安排促销策略，成为了超市保持竞争力的关键。在信息化快速发展的今天，传统的采购和库存管理模式已难以满足精细化管理的需求。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值