cloudera/quickstart

GP0000968523

已于 2023-06-26 12:59:31 修改

阅读量943

点赞数

文章标签： cloudera docker big data

于 2021-12-04 00:39:09 首次发布

本文为博主原创文章，未经博主允许不得转载。 https://fly6.applinzi.com

本文链接：https://blog.csdn.net/GP0000968523/article/details/121709876

版权

appledeMacBook-Pro:~ apple$ docker pull cloudera/quickstart
Using default tag: latest
latest: Pulling from cloudera/quickstart
Image docker.io/cloudera/quickstart:latest uses outdated schema1 manifest format. Please upgrade to a schema2 image for better future compatibility. More information at https://docs.docker.com/registry/spec/deprecated-schema-v1/
1d00652ce734: Pull complete
Digest: sha256:f91bee4cdfa2c92ea3652929a22f729d4d13fc838b00f120e630f91c941acb63
Status: Downloaded newer image for cloudera/quickstart:latest
docker.io/cloudera/quickstart:latest
appledeMacBook-Pro:~ apple$

2.启动cloudera/quickstart

docker run --privileged=true --hostname=quickstart.cloudera -p 8020:8020 -p 7180:7180 -p 21050:21050 -p 50070:50070 -p 50075:50075 -p 50010:50010 -p 50020:50020 -p 8888:8888 -t -i 4239cd2958c6 /usr/bin/docker-quickstart

docker run --privileged=true --hostname=quickstart.cloudera -p 8020:8020 -p 7180:7180 -p 21050:21050 -p 50070:50070 -p 50075:50075 -p 50010:50010 -p 50020:50020 -p 8888:8888 -t -i 4239cd2958c6 /usr/bin/docker-quickstart

Starting mysqld: [ OK ]

if [ “ $1" == " s t a r t "]; t h e ni f ["$ {EC2}” == ‘true’ ]; then
FIRST_BOOT_FLAG=/var/lib/cloudera-quickstart/.ec2-key-installed
if [ ! -f “ ${FIRST_BOOT_FLAG}" ]; then METADATA_API=http://169.254.169.254/latest/meta-data KEY_URL=$ {METADATA_API}/public-keys/0/openssh-key
SSH_DIR=/home/cloudera/.ssh
mkdir -p ${SSH_DIR}
chown cloudera:cloudera ${SSH_DIR}
curl ${KEY_URL} >> ${SSH_DIR}/authorized_keys
touch ${FIRST_BOOT_FLAG} fi fi if [ "$ {DOCKER}” != ‘true’ ]; then
if [ -f /sys/kernel/mm/redhat_transparent_hugepage/defrag ]; then
echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
fi

    cloudera-quickstart-ip
    HOSTNAME=quickstart.cloudera
    hostname ${HOSTNAME}
    sed -i -e "s/HOSTNAME=.*/HOSTNAME=${HOSTNAME}/" /etc/sysconfig/network
fi

(
    cd /var/lib/cloudera-quickstart/tutorial;
    nohup python -m SimpleHTTPServer 80 &
)

# TODO: check for expired CM license and update config.js accordingly

‘[’ start == start ‘]’
‘[’ ‘’ == true ‘]’
‘[’ true ‘!=’ true ‘]’
cd /var/lib/cloudera-quickstart/tutorial
nohup python -m SimpleHTTPServer 80
nohup: appending output to `nohup.out’
JMX enabled by default
Using config: /etc/zookeeper/conf/zoo.cfg
Starting zookeeper … STARTED
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-quickstart.cloudera.out
Started Hadoop datanode (hadoop-hdfs-datanode): [ OK ]
starting journalnode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-journalnode-quickstart.cloudera.out
Started Hadoop journalnode: [ OK ]
starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-quickstart.cloudera.out
Started Hadoop namenode: [ OK ]
starting secondarynamenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-secondarynamenode-quickstart.cloudera.out
Started Hadoop secondarynamenode: [ OK ]

Setting HTTPFS_HOME: /usr/lib/hadoop-httpfs
Using HTTPFS_CONFIG: /etc/hadoop-httpfs/conf
Sourcing: /etc/hadoop-httpfs/conf/httpfs-env.sh
Using HTTPFS_LOG: /var/log/hadoop-httpfs/
Using HTTPFS_TEMP: /var/run/hadoop-httpfs
Setting HTTPFS_HTTP_PORT: 14000
Setting HTTPFS_ADMIN_PORT: 14001
Setting HTTPFS_HTTP_HOSTNAME: quickstart.cloudera
Setting HTTPFS_SSL_ENABLED: false
Setting HTTPFS_SSL_KEYSTORE_FILE: /var/lib/hadoop-httpfs/.keystore
Setting HTTPFS_SSL_KEYSTORE_PASS: password
Using CATALINA_BASE: /var/lib/hadoop-httpfs/tomcat-deployment
Using HTTPFS_CATALINA_HOME: /usr/lib/bigtop-tomcat
Setting CATALINA_OUT: /var/log/hadoop-httpfs//httpfs-catalina.out
Using CATALINA_PID: /var/run/hadoop-httpfs/hadoop-httpfs-httpfs.pid

Using CATALINA_OPTS:
Adding to CATALINA_OPTS: -Dhttpfs.home.dir=/usr/lib/hadoop-httpfs -Dhttpfs.config.dir=/etc/hadoop-httpfs/conf -Dhttpfs.log.dir=/var/log/hadoop-httpfs/ -Dhttpfs.temp.dir=/var/run/hadoop-httpfs -Dhttpfs.admin.port=14001 -Dhttpfs.http.port=14000 -Dhttpfs.http.hostname=quickstart.cloudera
Using CATALINA_BASE: /var/lib/hadoop-httpfs/tomcat-deployment
Using CATALINA_HOME: /usr/lib/bigtop-tomcat
Using CATALINA_TMPDIR: /var/run/hadoop-httpfs
Using JRE_HOME: /usr/java/jdk1.7.0_67-cloudera
Using CLASSPATH: /usr/lib/bigtop-tomcat/bin/bootstrap.jar
Using CATALINA_PID: /var/run/hadoop-httpfs/hadoop-httpfs-httpfs.pid
Started Hadoop httpfs (hadoop-httpfs): [ OK ]
starting historyserver, logging to /var/log/hadoop-mapreduce/mapred-mapred-historyserver-quickstart.cloudera.out
Started Hadoop historyserver: [ OK ]
starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-quickstart.cloudera.out
Started Hadoop nodemanager: [ OK ]
starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-quickstart.cloudera.out
Started Hadoop resourcemanager: [ OK ]
starting master, logging to /var/log/hbase/hbase-hbase-master-quickstart.cloudera.out
Started HBase master daemon (hbase-master): [ OK ]
starting rest, logging to /var/log/hbase/hbase-hbase-rest-quickstart.cloudera.out
Started HBase rest daemon (hbase-rest): [ OK ]
starting thrift, logging to /var/log/hbase/hbase-hbase-thrift-quickstart.cloudera.out
Started HBase thrift daemon (hbase-thrift): [ OK ]
Starting Hive Metastore (hive-metastore): [ OK ]
Started Hive Server2 (hive-server2): [ OK ]
Starting Sqoop Server: [ OK ]
Sqoop home directory: /usr/lib/sqoop2
Setting SQOOP_HTTP_PORT: 12000
Setting SQOOP_ADMIN_PORT: 12001
Using CATALINA_OPTS: -Xmx1024m
Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
Using CATALINA_BASE: /var/lib/sqoop2/tomcat-deployment
Using CATALINA_HOME: /usr/lib/bigtop-tomcat
Using CATALINA_TMPDIR: /var/tmp/sqoop2
Using JRE_HOME: /usr/java/jdk1.7.0_67-cloudera
Using CLASSPATH: /usr/lib/bigtop-tomcat/bin/bootstrap.jar
Using CATALINA_PID: /var/run/sqoop2/sqoop-server-sqoop2.pid
Starting Spark history-server (spark-history-server): [ OK ]
Starting Hadoop HBase regionserver daemon: starting regionserver, logging to /var/log/hbase/hbase-hbase-regionserver-quickstart.cloudera.out
ERROR.
Starting hue: 6
[恢复于2021年12月3日下午9:26:17]
Last login: Fri Dec 3 21:26:01 on console
Restored session: 2021年12月 3日星期五 21时22分40秒 CST

The default interactive shell is now zsh.
To update your account to use zsh, please run chsh -s /bin/zsh.
For more details, please visit https://support.apple.com/kb/HT208050.
appledeMBP:~ apple$ docker run --privileged=true --hostname=quickstart.cloudera -p 8020:8020 -p 7180:7180 -p 21050:21050 -p 50070:50070 -p 50075:50075 -p 50010:50010 -p 50020:50020 -p 8888:8888 -t -i 4239cd2958c6 /usr/bin/docker-quickstart
docker: Error response from daemon: dial unix docker.raw.sock: connect: connection refused.
See ‘docker run --help’.
appledeMBP:~ apple$ docker run --privileged=true --hostname=quickstart.cloudera -p 8020:8020 -p 7180:7180 -p 21050:21050 -p 50070:50070 -p 50075:50075 -p 50010:50010 -p 50020:50020 -p 8888:8888 -t -i 4239cd2958c6 /usr/bin/docker-quickstart
Starting mysqld: [ OK ]

    cloudera-quickstart-ip
    HOSTNAME=quickstart.cloudera
    hostname ${HOSTNAME}
    sed -i -e "s/HOSTNAME=.*/HOSTNAME=${HOSTNAME}/" /etc/sysconfig/network
fi

(
    cd /var/lib/cloudera-quickstart/tutorial;
    nohup python -m SimpleHTTPServer 80 &
)

# TODO: check for expired CM license and update config.js accordingly

‘[’ start == start ‘]’
‘[’ ‘’ == true ‘]’
‘[’ true ‘!=’ true ‘]’
cd /var/lib/cloudera-quickstart/tutorial
nohup python -m SimpleHTTPServer 80
nohup: appending output to `nohup.out’
JMX enabled by default
Using config: /etc/zookeeper/conf/zoo.cfg
Starting zookeeper … STARTED
starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-quickstart.cloudera.out
Started Hadoop datanode (hadoop-hdfs-datanode): [ OK ]
starting journalnode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-journalnode-quickstart.cloudera.out
Started Hadoop journalnode: [ OK ]
starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-quickstart.cloudera.out
Started Hadoop namenode: [ OK ]
starting secondarynamenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-secondarynamenode-quickstart.cloudera.out
Started Hadoop secondarynamenode: [ OK ]

Using CATALINA_OPTS:
Adding to CATALINA_OPTS: -Dhttpfs.home.dir=/usr/lib/hadoop-httpfs -Dhttpfs.config.dir=/etc/hadoop-httpfs/conf -Dhttpfs.log.dir=/var/log/hadoop-httpfs/ -Dhttpfs.temp.dir=/var/run/hadoop-httpfs -Dhttpfs.admin.port=14001 -Dhttpfs.http.port=14000 -Dhttpfs.http.hostname=quickstart.cloudera
Using CATALINA_BASE: /var/lib/hadoop-httpfs/tomcat-deployment
Using CATALINA_HOME: /usr/lib/bigtop-tomcat
Using CATALINA_TMPDIR: /var/run/hadoop-httpfs
Using JRE_HOME: /usr/java/jdk1.7.0_67-cloudera
Using CLASSPATH: /usr/lib/bigtop-tomcat/bin/bootstrap.jar
Using CATALINA_PID: /var/run/hadoop-httpfs/hadoop-httpfs-httpfs.pid
Started Hadoop httpfs (hadoop-httpfs): [ OK ]
starting historyserver, logging to /var/log/hadoop-mapreduce/mapred-mapred-historyserver-quickstart.cloudera.out
Started Hadoop historyserver: [ OK ]
starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-quickstart.cloudera.out
Started Hadoop nodemanager: [ OK ]
starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-quickstart.cloudera.out
Started Hadoop resourcemanager: [ OK ]
starting master, logging to /var/log/hbase/hbase-hbase-master-quickstart.cloudera.out
Started HBase master daemon (hbase-master): [ OK ]
starting rest, logging to /var/log/hbase/hbase-hbase-rest-quickstart.cloudera.out
Started HBase rest daemon (hbase-rest): [ OK ]
starting thrift, logging to /var/log/hbase/hbase-hbase-thrift-quickstart.cloudera.out
Started HBase thrift daemon (hbase-thrift): [ OK ]
Starting Hive Metastore (hive-metastore): [ OK ]
Started Hive Server2 (hive-server2): [ OK ]
Starting Sqoop Server: [ OK ]
Sqoop home directory: /usr/lib/sqoop2
Setting SQOOP_HTTP_PORT: 12000
Setting SQOOP_ADMIN_PORT: 12001
Using CATALINA_OPTS: -Xmx1024m
Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
Using CATALINA_BASE: /var/lib/sqoop2/tomcat-deployment
Using CATALINA_HOME: /usr/lib/bigtop-tomcat
Using CATALINA_TMPDIR: /var/tmp/sqoop2
Using JRE_HOME: /usr/java/jdk1.7.0_67-cloudera
Using CLASSPATH: /usr/lib/bigtop-tomcat/bin/bootstrap.jar
Using CATALINA_PID: /var/run/sqoop2/sqoop-server-sqoop2.pid
Starting Spark history-server (spark-history-server): [ OK ]
Starting Hadoop HBase regionserver daemon: starting regionserver, logging to /var/log/hbase/hbase-hbase-regionserver-quickstart.cloudera.out
hbase-regionserver.
Starting hue: [FAILED]
Started Impala State Store Server (statestored): [ OK ]

Setting OOZIE_HOME: /usr/lib/oozie
Sourcing: /usr/lib/oozie/bin/oozie-env.sh
setting JAVA_LIBRARY_PATH=“ $JAVA_LIBRARY_PATH:/usr/lib/hadoop/lib/native" setting OOZIE_DATA=/var/lib/oozie setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat setting CATALINA_TMPDIR=/var/lib/oozie setting CATALINA_PID=/var/run/oozie/oozie.pid setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment setting OOZIE_HTTPS_PORT=11443 setting OOZIE_HTTPS_KEYSTORE_PASS=password setting CATALINA_OPTS="$ CATALINA_OPTS -Doozie.https.port= ${OOZIE_HTTPS_PORT}" setting CATALINA_OPTS="$ CATALINA_OPTS -Doozie.https.keystore.pass= ${OOZIE_HTTPS_KEYSTORE_PASS}" setting CATALINA_OPTS="$ CATALINA_OPTS -Xmx1024m”
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
Using OOZIE_CONFIG: /etc/oozie/conf
Sourcing: /etc/oozie/conf/oozie-env.sh
setting JAVA_LIBRARY_PATH=“ $JAVA_LIBRARY_PATH:/usr/lib/hadoop/lib/native" setting OOZIE_DATA=/var/lib/oozie setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat setting CATALINA_TMPDIR=/var/lib/oozie setting CATALINA_PID=/var/run/oozie/oozie.pid setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment setting OOZIE_HTTPS_PORT=11443 setting OOZIE_HTTPS_KEYSTORE_PASS=password setting CATALINA_OPTS="$ CATALINA_OPTS -Doozie.https.port= ${OOZIE_HTTPS_PORT}" setting CATALINA_OPTS="$ CATALINA_OPTS -Doozie.https.keystore.pass= ${OOZIE_HTTPS_KEYSTORE_PASS}" setting CATALINA_OPTS="$ CATALINA_OPTS -Xmx1024m”
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
Setting OOZIE_CONFIG_FILE: oozie-site.xml
Using OOZIE_DATA: /var/lib/oozie
Using OOZIE_LOG: /var/log/oozie
Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
Setting OOZIE_LOG4J_RELOAD: 10
Setting OOZIE_HTTP_HOSTNAME: quickstart.cloudera
Setting OOZIE_HTTP_PORT: 11000
Setting OOZIE_ADMIN_PORT: 11001
Using OOZIE_HTTPS_PORT: 11443
Setting OOZIE_BASE_URL: http://quickstart.cloudera:11000/oozie
Using CATALINA_BASE: /var/lib/oozie/tomcat-deployment
Setting OOZIE_HTTPS_KEYSTORE_FILE: /var/lib/oozie/.keystore
Using OOZIE_HTTPS_KEYSTORE_PASS: password
Setting OOZIE_INSTANCE_ID: quickstart.cloudera
Setting CATALINA_OUT: /var/log/oozie/catalina.out
Using CATALINA_PID: /var/run/oozie/oozie.pid

Using CATALINA_OPTS: -Doozie.https.port=11443 -Doozie.https.keystore.pass=password -Xmx1024m -Doozie.https.port=11443 -Doozie.https.keystore.pass=password -Xmx1024m -Dderby.stream.error.file=/var/log/oozie/derby.log
Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/lib/oozie -Doozie.config.dir=/etc/oozie/conf -Doozie.log.dir=/var/log/oozie -Doozie.data.dir=/var/lib/oozie -Doozie.instance.id=quickstart.cloudera -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10 -Doozie.http.hostname=quickstart.cloudera -Doozie.admin.port=11001 -Doozie.http.port=11000 -Doozie.https.port=11443 -Doozie.base.url=http://quickstart.cloudera:11000/oozie -Doozie.https.keystore.file=/var/lib/oozie/.keystore -Doozie.https.keystore.pass=password -Djava.library.path=:/usr/lib/hadoop/lib/native:/usr/lib/hadoop/lib/native

Using CATALINA_BASE: /var/lib/oozie/tomcat-deployment
Using CATALINA_HOME: /usr/lib/bigtop-tomcat
Using CATALINA_TMPDIR: /var/lib/oozie
Using JRE_HOME: /usr/java/jdk1.7.0_67-cloudera
Using CLASSPATH: /usr/lib/bigtop-tomcat/bin/bootstrap.jar
Using CATALINA_PID: /var/run/oozie/oozie.pid
Starting Solr server daemon: [ OK ]
Using CATALINA_BASE: /var/lib/solr/tomcat-deployment
Using CATALINA_HOME: /usr/lib/solr/…/bigtop-tomcat
Using CATALINA_TMPDIR: /var/lib/solr/
Using JRE_HOME: /usr/java/jdk1.7.0_67-cloudera
Using CLASSPATH: /usr/lib/solr/…/bigtop-tomcat/bin/bootstrap.jar
Using CATALINA_PID: /var/run/solr/solr.pid
Started Impala Catalog Server (catalogd) : [ OK ]
Started Impala Server (impalad): [ OK ]
[root@quickstart /]# jps
^C
[root@quickstart /]# ps
PID TTY TIME CMD
1 pts/0 00:00:07 bash
48 pts/0 00:00:00 mysqld_safe
197 pts/0 00:01:07 python
3493 pts/0 00:00:00 ps
[root@quickstart /]# jps
^C
[root@quickstart /]# ^C
[root@quickstart /]# /home/cloudera/cloudera-manager --enterprise

WARNING: It is highly recommended that you run Cloudera Enterprise in a VM with
at least 10 GB of RAM.

You can override these checks by passing in the --force option,
e.g:

sudo /home/cloudera/cloudera-manager --force

[root@quickstart /]# /home/cloudera/cloudera-manager --enterprise --force
^C
[root@quickstart /]# /home/cloudera/cloudera-manager --enterprise --force sudo /home/cloudera/cloudera-manager --force
^C
[root@quickstart /]# sudo /home/cloudera/cloudera-manager --force
You must specify --express or --enterprise
[root@quickstart /]# /home/cloudera/cloudera-manager --enterprise --force
[QuickStart] Shutting down CDH services via init scripts…
kafka-server: unrecognized service
JMX enabled by default
Using config: /etc/zookeeper/conf/zoo.cfg
[QuickStart] Disabling CDH services on boot…
error reading information on service kafka-server: No such file or directory
[QuickStart] Starting Cloudera Manager server…
[QuickStart] Waiting for Cloudera Manager API…
[QuickStart] Starting Cloudera Manager agent…
[QuickStart] Activating trial license for Enterprise…
[QuickStart] Configuring deployment…
Submitted jobs: 15
[QuickStart] Deploying client configuration…
Submitted jobs: 17
[QuickStart] Starting Cloudera Management Service…
Submitted jobs: 25
[QuickStart] Enabling Cloudera Manager daemons on boot…

Success! You can now log into Cloudera Manager from the QuickStart VM’s browser:

http://quickstart.cloudera:7180

Username: cloudera
Password: cloudera

在这里插入图片描述

GP0000968523

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
复制链接

分享到 QQ

分享到新浪微博

扫一扫