Hadoop基础之《（6）—Hadoop单机伪集群安装》

csj50

已于 2023-02-06 14:46:31 修改

阅读量550

点赞数

分类专栏：大数据文章标签： hadoop

于 2023-01-30 13:35:25 首次发布

本文链接：https://blog.csdn.net/csj50/article/details/128800667

版权

大数据专栏收录该内容

11 篇文章 4 订阅

订阅专栏

一、安装JDK

yum install java-1.8*

二、关闭防火墙

systemctl status firewalld
systemctl stop firewalld
systemctl disable firewalld

三、配置ip地址和主机名映射

vi /etc/hosts
加入：
192.168.1.1  hadoop001

四、配置免密登录

1、生成公私钥

ssh-keygen -t rsa

2、写入公钥到授权文件

cd /root/.ssh/
cat id_rsa.pub >> authorized_keys
chmod 600 authorized_keys

五、建立安装目录

mkdir -p /appserver/hadoop

六、下载安装包

cd /root
wget https://archive.apache.org/dist/hadoop/common/hadoop-3.3.4/hadoop-3.3.4.tar.gz

七、解压安装包

tar -zxvf hadoop-3.3.4.tar.gz -C /appserver/hadoop/

解压到目录为：/appserver/hadoop/hadoop-3.3.4

Hadoop目录结构
bin etc include lib libexec LICENSE-binary licenses-binary LICENSE.txt logs NOTICE-binary NOTICE.txt README.txt sbin share

1、bin目录
会用到hdfs、mapred、yarn命令

2、etc目录
提供大量的配置文件
core-site.xml，核心配置
hdfs-site.xml，和hdfs相关的
mapred-site.xml，和计算相关的
yarn-site.xml，和yarn相关的
workers，后期会用到

3、sbin目录
启动脚本
start-dfs.sh，启动HDFS
start-yarn.sh，启动yarn
mr-jobhistory-daemon.sh，启动历史服务器

4、share目录
学习资料和官方案例

八、hadoop的配置

0、配置HADOOP_HOME和JAVA_HOME

vi /etc/profile

添加：
export HADOOP_HOME=/appserver/hadoop/hadoop-3.3.4
export PATH=$PATH:$HADOOP_HOME/bin
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.362.b08-1.el7_9.x86_64
export PATH=$PATH:$JAVA_HOME/bin

source /etc/profile

1、修改hadoop-env.sh
配置JAVA_HOME，yum安装openjdk的路径在/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.362.b08-1.el7_9.x86_64

cd /appserver/hadoop/hadoop-3.3.4/etc/hadoop/
vi hadoop-env.sh
加入：
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.362.b08-1.el7_9.x86_64

2、测试hadoop环境

bin/hadoop version

Hadoop 3.3.4
Source code repository https://github.com/apache/hadoop.git -r a585a73c3e02ac62350c136643a5e7f6095a3dbb
Compiled by stevel on 2022-07-29T12:32Z
Compiled with protoc 3.7.1
From source with checksum fb9dd8918a7b8a5b430d61af858f6ec
This command was run using /appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-common-3.3.4.jar

3、修改core-site.xml
（1）指定namenode的hdfs协议文件系统的通信地址
（2）指定hadoop存储临时文件的目录

vi core-site.xml

<configuration>
    <property>
        <!--指定 namenode 的 hdfs 协议文件系统的通信地址-->
        <name>fs.defaultFS</name>
        <value>hdfs://hadoop001:8020</value>
    </property>
    <property>
        <!--指定 hadoop 存储临时文件的目录-->
        <name>hadoop.tmp.dir</name>
        <value>/appserver/hadoop/tmp</value>
    </property>
</configuration>

4、修改hdfs-site.xml
（1）设置namenode节点数据保存目录
（2）设置datanode节点数据保存目录
（3）指定副本数量，在分布式文件系统中，数据通常会被冗余的存储多份，以保证可靠性和安全性，但是这里用的是伪分布式模式，节点只有一个，也有就只有一个副本

vi hdfs-site.xml

<configuration>
    <property>
        <!--设置namenode节点数据保存目录-->
        <name>dfs.namenode.name.dir</name>
        <value>/appserver/hadoop/namenode/data</value>
    </property>
    <property>
        <!--datanode节点数据保存目录-->
        <name>dfs.datanode.data.dir</name>
        <value>/appserver/hadoop/datanode/data</value>
    </property>
    <property>
        <!--指定副本数量-->
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

5、修改mapred-site.xml
（1）指定mapreduce作业运行在yarn上
（2）配置历史服务器地址

vi mapred-site.xml

<configuration>
    <property>
        <!--指定 mapreduce 作业运行在 yarn 上-->
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <!-- 配置历史服务器的内部服务端地址 -->
    <property>
        <name>mapreduce.jobhistory.address</name>
        <value>hadoop001:10020</value>
    </property>
    <!-- 配置历史服务器暴露的web地址 -->
    <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>hadoop001:19888</value>
    </property>
</configuration>

6、修改yarn-site.xml
（1）配置NodeManager上运行的附属服务。需要配置成mapreduce_shuffle后才可以在Yarn上运行MapReduce程序
（2）配置resourcemanager在哪个节点上
（3）配置yarn的classpath
（4）配置日志聚集功能

vi yarn-site.xml

<configuration>
    <property>
        <!--配置 NodeManager 上运行的附属服务。需要配置成 mapreduce_shuffle 后才可以在 Yarn 上运行 MapReduce 程序。-->
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <!--resourcemanager 的主机名-->
        <name>yarn.resourcemanager.hostname</name>
        <value>hadoop001</value>
    </property>
    <!-- 配置classpath -->
    <property>
        <name>yarn.application.classpath</name>
        <value>/appserver/hadoop/hadoop-3.3.4/etc/hadoop:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/*</value>
    </property>
    <!-- 配置日志聚集功能，yarn-default.xml中该项默认为false -->
    <property>
        <name>yarn.log-aggregation-enable</name>
        <value>true</value>
    </property>
    <!-- 设置日志聚集服务器地址，设置聚集到历史日志服务器上，方便查看 -->
    <property>
        <name>yarn.log.server.url</name>
        <value>http://hadoop001:19888/jobhistory/logs</value>
    </property>
    <!-- 设置日志保留时间：7天 -->
    <property>
        <name>yarn.log-aggregation.retain-seconds</name>
        <value>604800</value>
    </property>
</configuration>

7、slaves
（1）在/appserver/hadoop/hadoop-3.3.4/etc/hadoop/workers文件，配置节点的主机名或IP地址
（2）单机版本，指定本机即可
（3）真集群环境下，所有从属节点上的DataNode服务和NodeManager服务都会被启动

vi workers

hadoop001

九、首次启动格式化HDFS

cd /appserver/hadoop/hadoop-3.3.4/bin
./hdfs namenode -format

WARNING: /appserver/hadoop/hadoop-3.3.4/logs does not exist. Creating.
2023-01-30 00:24:08,201 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = myhost/192.168.1.1
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 3.3.4
STARTUP_MSG:   classpath = /appserver/hadoop/hadoop-3.3.4/etc/hadoop:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/hadoop-auth-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jersey-core-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jsp-api-2.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jersey-json-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/checker-qual-2.5.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-util-ajax-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/re2j-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-util-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jsch-0.1.55.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-security-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/paranamer-2.3.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerby-util-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-webapp-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/avro-1.7.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-util-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/hadoop-shaded-guava-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-databind-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-net-3.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-server-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-core-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/netty-3.10.6.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-logging-1.1.3.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-compress-1.21.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/metrics-core-3.2.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-codec-1.15.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-text-1.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/guava-27.0-jre.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/reload4j-1.2.22.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-client-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/failureaccess-1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/zookeeper-jute-3.5.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-common-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/httpclient-4.5.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-collections-3.2.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/snappy-java-1.1.8.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/gson-2.8.9.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerby-config-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/woodstox-core-5.3.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-http-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-io-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jsr305-3.0.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/curator-recipes-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/dnsjava-2.1.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/zookeeper-3.5.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/asm-5.0.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jersey-server-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/curator-client-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/stax2-api-4.2.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/curator-framework-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/token-provider-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/accessors-smart-2.4.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-core-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-io-2.8.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jersey-servlet-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jettison-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-servlet-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-math3-3.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/json-smart-2.4.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-xml-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/httpcore-4.4.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/nimbus-jose-jwt-9.8.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-cli-1.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/hadoop-annotations-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-server-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-nfs-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-registry-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-common-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-common-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-kms-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/hadoop-auth-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/okio-2.8.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/re2j-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-redis-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.77.Final-osx-x86_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-security-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/paranamer-2.3.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.77.Final-osx-aarch_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-webapp-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/avro-1.7.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-util-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-databind-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-net-3.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-3.10.6.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-compress-1.21.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kotlin-stdlib-1.4.10.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-buffer-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-text-1.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/zookeeper-jute-3.5.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-http2-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-udt-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/okhttp-4.9.3.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-xml-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kotlin-stdlib-common-1.4.10.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.77.Final-linux-aarch_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-handler-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/snappy-java-1.1.8.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-dns-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/gson-2.8.9.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.77.Final-linux-x86_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/woodstox-core-5.3.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-http-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-io-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-all-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-socks-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-common-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/curator-recipes-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/zookeeper-3.5.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.77.Final-osx-aarch_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/asm-5.0.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/curator-client-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/curator-framework-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/accessors-smart-2.4.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-io-2.8.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-resolver-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jettison-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-http-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-servlet-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.77.Final-osx-x86_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/json-smart-2.4.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-xml-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.8.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/hadoop-annotations-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-server-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-nfs-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-client-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-client-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/asm-tree-9.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jetty-plus-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/fst-2.50.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/javax.inject-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/guice-4.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/snakeyaml-1.26.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/bcprov-jdk15on-1.60.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/aopalliance-1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jetty-annotations-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/asm-analysis-9.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jna-5.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/asm-commons-9.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/websocket-servlet-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jline-3.9.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/websocket-api-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/objenesis-2.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/json-io-2.5.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/websocket-server-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/websocket-common-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jetty-client-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/bcpkix-jdk15on-1.60.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jersey-client-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/websocket-client-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/java-util-1.9.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jetty-jndi-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-api-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-client-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-registry-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-common-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-tests-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-services-core-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-router-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-common-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-services-api-3.3.4.jar
STARTUP_MSG:   build = https://github.com/apache/hadoop.git -r a585a73c3e02ac62350c136643a5e7f6095a3dbb; compiled by 'stevel' on 2022-07-29T12:32Z
STARTUP_MSG:   java = 1.8.0_362
************************************************************/
2023-01-30 00:24:08,217 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2023-01-30 00:24:08,364 INFO namenode.NameNode: createNameNode [-format]
2023-01-30 00:24:09,371 INFO common.Util: Assuming 'file' scheme for path /appserver/hadoop/namenode/data in configuration.
2023-01-30 00:24:09,373 INFO common.Util: Assuming 'file' scheme for path /appserver/hadoop/namenode/data in configuration.
2023-01-30 00:24:09,383 INFO namenode.NameNode: Formatting using clusterid: CID-4adad145-e4cb-4b11-bb30-526830848a22
2023-01-30 00:24:09,441 INFO namenode.FSEditLog: Edit logging is async:true
2023-01-30 00:24:09,489 INFO namenode.FSNamesystem: KeyProvider: null
2023-01-30 00:24:09,492 INFO namenode.FSNamesystem: fsLock is fair: true
2023-01-30 00:24:09,494 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
2023-01-30 00:24:09,502 INFO namenode.FSNamesystem: fsOwner                = root (auth:SIMPLE)
2023-01-30 00:24:09,502 INFO namenode.FSNamesystem: supergroup             = supergroup
2023-01-30 00:24:09,526 INFO namenode.FSNamesystem: isPermissionEnabled    = true
2023-01-30 00:24:09,526 INFO namenode.FSNamesystem: isStoragePolicyEnabled = true
2023-01-30 00:24:09,527 INFO namenode.FSNamesystem: HA Enabled: false
2023-01-30 00:24:09,674 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2023-01-30 00:24:09,701 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
2023-01-30 00:24:09,701 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
2023-01-30 00:24:09,711 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2023-01-30 00:24:09,711 INFO blockmanagement.BlockManager: The block deletion will start around 2023 Jan 30 00:24:09
2023-01-30 00:24:09,715 INFO util.GSet: Computing capacity for map BlocksMap
2023-01-30 00:24:09,715 INFO util.GSet: VM type       = 64-bit
2023-01-30 00:24:09,718 INFO util.GSet: 2.0% max memory 3.4 GB = 70.6 MB
2023-01-30 00:24:09,718 INFO util.GSet: capacity      = 2^23 = 8388608 entries
2023-01-30 00:24:09,899 INFO blockmanagement.BlockManager: Storage policy satisfier is disabled
2023-01-30 00:24:09,899 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false
2023-01-30 00:24:09,917 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.999
2023-01-30 00:24:09,918 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0
2023-01-30 00:24:09,918 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
2023-01-30 00:24:09,920 INFO blockmanagement.BlockManager: defaultReplication         = 1
2023-01-30 00:24:09,920 INFO blockmanagement.BlockManager: maxReplication             = 512
2023-01-30 00:24:09,920 INFO blockmanagement.BlockManager: minReplication             = 1
2023-01-30 00:24:09,920 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
2023-01-30 00:24:09,921 INFO blockmanagement.BlockManager: redundancyRecheckInterval  = 3000ms
2023-01-30 00:24:09,921 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
2023-01-30 00:24:09,921 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
2023-01-30 00:24:10,010 INFO namenode.FSDirectory: GLOBAL serial map: bits=29 maxEntries=536870911
2023-01-30 00:24:10,011 INFO namenode.FSDirectory: USER serial map: bits=24 maxEntries=16777215
2023-01-30 00:24:10,011 INFO namenode.FSDirectory: GROUP serial map: bits=24 maxEntries=16777215
2023-01-30 00:24:10,011 INFO namenode.FSDirectory: XATTR serial map: bits=24 maxEntries=16777215
2023-01-30 00:24:10,059 INFO util.GSet: Computing capacity for map INodeMap
2023-01-30 00:24:10,060 INFO util.GSet: VM type       = 64-bit
2023-01-30 00:24:10,060 INFO util.GSet: 1.0% max memory 3.4 GB = 35.3 MB
2023-01-30 00:24:10,060 INFO util.GSet: capacity      = 2^22 = 4194304 entries
2023-01-30 00:24:10,110 INFO namenode.FSDirectory: ACLs enabled? true
2023-01-30 00:24:10,110 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true
2023-01-30 00:24:10,110 INFO namenode.FSDirectory: XAttrs enabled? true
2023-01-30 00:24:10,111 INFO namenode.NameNode: Caching file names occurring more than 10 times
2023-01-30 00:24:10,125 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536
2023-01-30 00:24:10,129 INFO snapshot.SnapshotManager: SkipList is disabled
2023-01-30 00:24:10,138 INFO util.GSet: Computing capacity for map cachedBlocks
2023-01-30 00:24:10,138 INFO util.GSet: VM type       = 64-bit
2023-01-30 00:24:10,139 INFO util.GSet: 0.25% max memory 3.4 GB = 8.8 MB
2023-01-30 00:24:10,139 INFO util.GSet: capacity      = 2^20 = 1048576 entries
2023-01-30 00:24:10,156 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
2023-01-30 00:24:10,157 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
2023-01-30 00:24:10,157 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
2023-01-30 00:24:10,165 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
2023-01-30 00:24:10,165 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2023-01-30 00:24:10,168 INFO util.GSet: Computing capacity for map NameNodeRetryCache
2023-01-30 00:24:10,168 INFO util.GSet: VM type       = 64-bit
2023-01-30 00:24:10,169 INFO util.GSet: 0.029999999329447746% max memory 3.4 GB = 1.1 MB
2023-01-30 00:24:10,169 INFO util.GSet: capacity      = 2^17 = 131072 entries
2023-01-30 00:24:10,248 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1636510195-173.249.200.110-1675056250223
2023-01-30 00:24:10,284 INFO common.Storage: Storage directory /appserver/hadoop/namenode/data has been successfully formatted.
2023-01-30 00:24:10,347 INFO namenode.FSImageFormatProtobuf: Saving image file /appserver/hadoop/namenode/data/current/fsimage.ckpt_0000000000000000000 using no compression
2023-01-30 00:24:10,567 INFO namenode.FSImageFormatProtobuf: Image file /appserver/hadoop/namenode/data/current/fsimage.ckpt_0000000000000000000 of size 399 bytes saved in 0 seconds .
2023-01-30 00:24:10,844 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
2023-01-30 00:24:10,916 INFO namenode.FSNamesystem: Stopping services started for active state
2023-01-30 00:24:10,917 INFO namenode.FSNamesystem: Stopping services started for standby state
2023-01-30 00:24:10,928 INFO namenode.FSImage: FSImageSaver clean checkpoint: txid=0 when meet shutdown.
2023-01-30 00:24:10,929 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at myhost/192.168.1.1
************************************************************/

十、添加root启动

可以加在profile内，也可以加在hadoop-env.sh内

vi /etc/profile

添加：
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root

source /etc/profile

十一、启动服务

进入到sbin目录下

1、启动dfs服务

./start-dfs.sh

Starting namenodes on [hadoop001]
Last login: Mon Jan 30 01:26:56 EST 2023 on pts/1
hadoop001: Warning: Permanently added 'hadoop001,192.168.1.1' (ECDSA) to the list of known hosts.
Starting datanodes
Last login: Mon Jan 30 01:31:55 EST 2023 on pts/1
localhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Starting secondary namenodes [myhost]
Last login: Mon Jan 30 01:31:57 EST 2023 on pts/1
myhost: Warning: Permanently added 'myhost' (ECDSA) to the list of known hosts.

2、启动yarn服务

./start-yarn.sh

Starting resourcemanager
Last login: Mon Jan 30 01:32:03 EST 2023 on pts/1
Starting nodemanagers
Last login: Mon Jan 30 01:35:43 EST 2023 on pts/1

3、手动启动历史服务器
进入到bin目录下

./mapred --daemon start historyserver

十二、查看启动的服务

jps

15570 SecondaryNameNode
20229 NodeManager
20502 Jps
15179 NameNode
19901 ResourceManager
20445 JobHistoryServer
15358 DataNode

十三、管理页面

1、hadoop NameNode管理界面
http://192.168.1.1:9870/

2、yarn ResourceManager管理界面
http://192.168.1.1:8088/

3、hadoop SecondaryNameNode管理界面
http://192.168.1.1:9868

4、yarn历史服务器管理界面
http://192.168.1.1:19888

参考资料：
https://blog.csdn.net/weixin_44433834/article/details/122408455
https://blog.csdn.net/cai750415222/article/details/116530976
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html

csj50

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
Hadoop基础之《（6）—Hadoop单机伪集群安装》

配置JAVA_HOME，yum安装openjdk的路径在/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.362.b08-1.el7_9.x86_64。（3）指定副本数量，在分布式文件系统中，数据通常会被冗余的存储多份，以保证可靠性和安全性，但是这里用的是伪分布式模式，节点只有一个，也有就只有一个副本。在/appserver/hadoop/hadoop-3.3.4/etc/hadoop/下建立slaves，配置节点的主机名或IP地址。1、修改hadoop-env.sh。
复制链接

扫一扫