Ubuntu18.04(Bionic)安装CM6.3.1+CDH6.3.2

最新推荐文章于 2024-04-27 10:50:37 发布
最新推荐文章于 2024-04-27 10:50:37 发布
阅读量5.7k 收藏 17
点赞数 6
分类专栏： CDH Linux&Shell Hadoop 文章标签： cm cdh ubuntu hadoop
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_45883933/article/details/106202292
版权
Linux&Shell 同时被 3 个专栏收录
36 篇文章 1 订阅
订阅专栏
Hadoop
23 篇文章 0 订阅
订阅专栏
CDH
2 篇文章 0 订阅
订阅专栏

博文目录

1 初始化系统

1.1 设置root用户密码

kylin@ubuntu:~$ sudo passwd root
[sudo] kylin 的密码： 
输入新的 UNIX 密码： 
重新输入新的 UNIX 密码： 
passwd：已成功更新密码
kylin@ubuntu:~$ su root
密码：

1.2 系统软件更新

root@ubuntu:/home/kylin# sudo apt update && sudo apt upgrade
。。。。。。

1.3 安装ssh服务

kylin@ubuntu:~$ sudo apt install ssh
kylin@ubuntu:~$ service ssh start
kylin@ubuntu:~$ sshd -T
#ubuntu高级版本/etc/ssh/sshd_config中下面两项默认配置会导致ssh远程登录失败，做如下修改
kylin@ubuntu:~$ sed -i 's/PasswordAuthentication no/PasswordAuthentication yes/' /etc/ssh/sshd_config
kylin@ubuntu:~$ sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
kylin@ubuntu:~$ service ssh restart

1.4 ssh本地免密登陆设置

root@kylin:/# ssh localhost
root@localhost: Permission denied (publickey).
root@kylin:/# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa
Your public key has been saved in /root/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:9VAHvCaC5qpmdHnyQ67jo2uhRwChWHKYuLUDVFIDAD8 root@kylin
The key`s randomart image is:
+---[RSA 3072]----+
|XB*+       .o..  |
|O=o .      ...   |
|++E.   .  o  .   |
|o o.  o ...oo    |
| . . +  S. o.    |
|  + + +          |
| + o B           |
|. = + +          |
| =+=o+ .         |
+----[SHA256]-----+

root@kylin:/# cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
root@kylin:/# chmod 0600 ~/.ssh/authorized_keys
root@kylin:/# ssh localhost
root@kylin:~# exit
logout
Connection to localhost closed.

集群多台主机的话需要打通ssh无密码登录，将CM管理主机的公钥拷贝到各主机的 /.ssh/authorized_keys文件中。

1.5 设置静态IP地址

kylin@ubuntu:~$ ifconfig

Command 'ifconfig' not found, but can be installed with:

sudo apt install net-tools

kylin@ubuntu:~$ sudo apt install net-tools

# 获取系统IP
kylin@ubuntu:~$ ip=`ifconfig ens33 | egrep -o "inet [^ ]*" | grep -o "[0-9.]*"`
kylin@ubuntu:~$ echo $ip
# ssh root@192.168.234.148

#设置静态IP
root@ubuntu:~# sudo apt install vim
root@ubuntu:~# sudo  vim /etc/netplan/01-network-manager-all.yaml
# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager

  ethernets:
        ens33: #配置的网卡名称
          dhcp4: no #dhcp4关闭
          dhcp6: no #dhcp6关闭
          addresses: [192.168.234.148/24] #设置IP及掩码
          gateway4: 192.168.234.2 #设置网关
          nameservers:
                  addresses: [114.114.114.114, 8.8.8.8] #设置DNS

root@ubuntu:~# sudo netplan apply
# 能ping通则说明配置成功，系统重启不会导致IP的变化
root@ubuntu:~# ping www.baidu.com
PING www.a.shifen.com (14.215.177.39) 56(84) bytes of data.
64 bytes from 14.215.177.39 (14.215.177.39): icmp_seq=1 ttl=128 time=36.1 ms
64 bytes from 14.215.177.39 (14.215.177.39): icmp_seq=2 ttl=128 time=35.3 ms
64 bytes from 14.215.177.39 (14.215.177.39): icmp_seq=3 ttl=128 time=45.3 ms
^C
--- www.a.shifen.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2007ms
rtt min/avg/max/mdev = 35.330/38.927/45.342/4.549 ms

1.6 Windows Terminal 添加SSH连接

{
                "guid": "{6e9fa4d2-a4aa-562d-b1fa-0789dc1f83d7}", # 唯一guid
                "hidden": false,
                "name": "Ubuntu-VM",
                "commandline": "ssh.exe root@192.168.234.148" # 这里需要windows环境中有git(ssh.exe)
},

#Git上传本地ssh公钥，实现无密码登录

kylin@kylin MINGW64 ~
$ pwd
/c/Users/kylin
kylin@kylin MINGW64 ~/.ssh
$ scp id_rsa.pub root@192.168.234.148:/root/
#远程机上执行，将scp上来的公钥增加到authorized_keys
root@ubuntu:~# cat id_rsa.pub >> ./.ssh/authorized_keys

1.7 更换阿里源

root@ubuntu:~# cp /etc/apt/sources.list /etc/apt/sourses.list.bak
root@ubuntu:~# vi /etc/apt/sources.list
＃删除文件中内容并添加以下内容：
deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
＃更新升级
root@ubuntu:$ sudo apt update && sudo apt upgrade

2.Cloudera Manager与CDH安装

2.1 安装前配置

安装前配置官方

2.1.1 配置网络名称

  1. Set the hostname to a unique name (not localhost)：
    sudo hostnamectl set-hostname quickstart.cloudera.com
  2. Edit /etc/hosts with the IP address and fully qualified domain name (FQDN) of each host in the cluster, You can add the unqualified name as well：
    	1.1.1.1  foo-1.example.com  foo-1
	2.2.2.2  foo-2.example.com  foo-2
	3.3.3.3  foo-3.example.com  foo-3
	4.4.4.4  foo-4.example.com  foo-4
    root@kylin:~# echo "172.28.150.27 quickstart.cloudera cloudera" >> /etc/hosts
    我这里设置了一台主机。
  3. Edit /etc/sysconfig/network with the FQDN of this host only:
    HOSTNAME=quickstart.cloudera.com
  4. Verify that each host consistently identifies to the network:
    [cloudera@quickstart ~]$ hostname
quickstart.cloudera
[cloudera@quickstart ~]$ uname -a
Linux quickstart.cloudera 2.6.32-573.el6.x86_64 #1 SMP Thu Jul 23 15:44:03 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[cloudera@quickstart ~]$ /sbin/ifconfig eth1 | egrep -o "inet [^ ]*" | grep -o "[0-9.]*"
192.168.234.143
[cloudera@quickstart ~]$ host -v -t A quickstart.cloudera
Trying "quickstart.cloudera"
Received 37 bytes from 192.168.234.2#53 in 57 ms
Trying "quickstart.cloudera.localdomain"
Trying "quickstart.cloudera.cloudera"
Host quickstart.cloudera not found: 3(NXDOMAIN)
Received 46 bytes from 192.168.234.2#53 in 223 ms
[cloudera@quickstart ~]$

2.1.2 禁用防火墙（Ubuntu）

root@kylin:~# sudo iptables-save > ~/firewall.rules
root@kylin:~# sudo service ufw stop

2.1.3 禁用 SELinux

# 查看 SELinux状态
$ getenforce
# 配置
/bin/sed -i 's/mingetty tty/mingetty --noclear tty/' /etc/inittab
/bin/sed -i 's/SELINUX=permissive/SELINUX=disabled/' /etc/selinux/config
/bin/sed -i 's/SELINUX=enforcing/SELINUX=disabled/'  /etc/selinux/config
# 关闭 （each host）
$ setenforce 0

2.1.4 Enable an NTP Service

CDH要求集群的每台主机上配置网络时间协议（NTP）进行时间同步：

  1. Install the ntp package:
    apt-get install ntp
  2. Edit the /etc/ntp.conf file to add NTP servers, as in the following example:
    server 0.pool.ntp.org
server 1.pool.ntp.org
server 2.pool.ntp.org
  3. Start the ntpd service:
    sudo service ntpd start
  4. Configure the ntpd service to run at boot:
    chkconfig ntpd on
  5. Synchronize the system clock to the NTP server:
    ntpdate -u <ntp_server>
  6. Synchronize the hardware clock to the system clock:
    hwclock --systohc

2.1.5 禁用透明大页（THP）

root@kylin:~# echo never > /sys/kernel/mm/transparent_hugepage/enabled
root@kylin:~# echo never > /sys/kernel/mm/transparent_hugepage/defrag
root@kylin:~# echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local
root@kylin:~# echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local

2.1.6 禁用交换分区（swap）

root@kylin:~# sudo sysctl vm.swappiness=1
vm.swappiness = 1
root@kylin:~# echo 'vm.swappiness=1' >> /etc/sysctl.conf

2.1.7 Ubuntu安装httpd

root@kylin:~# sudo apt-get install httpd
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package httpd is a virtual package provided by:
  lighttpd 1.4.45-1ubuntu3.18.04
  nginx-light 1.14.0-0ubuntu1.7
  nginx-full 1.14.0-0ubuntu1.7
  nginx-extras 1.14.0-0ubuntu1.7
  nginx-core 1.14.0-0ubuntu1.7
  apache2 2.4.29-1ubuntu4.13
  yaws 2.0.4+dfsg-2
  webfs 1.21+ds1-12
  tntnet 2.2.1-3build1
  mini-httpd 1.23-1.2build1
  micro-httpd 20051212-15.1
  ebhttpd 1:1.0.dfsg.1-4.3build1
  aolserver4-daemon 4.5.1-18.1
  aolserver4-core 4.5.1-18.1
You should explicitly select one to install.

E: Package 'httpd' has no installation candidate
root@kylin:~# sudo apt-get install apache2
......

root@kylin:~# service apache2 status
 * apache2 is not running
root@kylin:~# service apache2 start
 * Starting Apache httpd web server apache2  
#自启动
root@kylin:~# sudo systemctl start apache2

# 浏览器进行URL访问

2.2 安装Cloudera Manager

Cloudera Manager安装

2.2.1 为Cloudera Manager配置存储库

root@kylin:~# wget https://archive.cloudera.com/cm6/6.3.1/ubuntu1804/apt/cloudera-manager.list
root@kylin:~# mv ./cloudera-manager.list /etc/apt/sources.list.d/
root@kylin:~# cd /etc/apt/sources.list.d/
root@kylin:/etc/apt/sources.list.d# ll
total 12
drwxr-xr-x 2 root root 4096 May 14 16:06 ./
drwxr-xr-x 7 root root 4096 May 13 00:06 ../
-rw-r--r-- 1 root root  117 May 14 16:06 cloudera-manager.list
root@kylin:~# cd /root/
root@kylin:~# cat /etc/apt/sources.list.d/cloudera-manager.list
# Cloudera Manager 6.3.1
deb [arch=amd64] http://archive.cloudera.com/cm6/6.3.1/ubuntu1804/apt bionic-cm6.3.1 contrib

root@kylin:~# wget https://archive.cloudera.com/cm6/6.3.1/ubuntu1804/apt/archive.key
--2020-05-14 16:38:06--  https://archive.cloudera.com/cm6/6.3.1/ubuntu1804/apt/archive.key
Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.108.167
Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.108.167|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1718 (1.7K) [application/pgp-keys]
Saving to: ‘archive.key’

archive.key                   100%[===============================================>]   1.68K  --.-KB/s    in 0s

2020-05-14 16:38:09 (8.10 MB/s) - ‘archive.key’ saved [1718/1718]

root@kylin:~# sudo apt-key add archive.key
OK
root@kylin:~# sudo apt-get update

root@kylin:~# cd /var/cache/apt/archives/
root@kylin:/var/cache/apt/archives# wget https://archive.cloudera.com/cm6/6.3.1/ubuntu1804/apt/pool/contrib/e/enterprise/cloudera-manager-daemons_6.3.1~1466458.ubuntu1804_all.deb
root@kylin:/var/cache/apt/archives# wget https://archive.cloudera.com/cm6/6.3.1/ubuntu1804/apt/pool/contrib/e/enterprise/cloudera-manager-agent_6.3.1~1466458.ubuntu1804_amd64.deb
root@kylin:/var/cache/apt/archives# wget https://archive.cloudera.com/cm6/6.3.1/ubuntu1804/apt/pool/contrib/e/enterprise/cloudera-manager-server_6.3.1~1466458.ubuntu1804_all.deb

或者可以配置cm的离线资源库，之后进行安装，参考官方说明。

2.2.2 安装JDK

#root@kylin:~# mkdir scpfile
#scp复制jdk-8u181-linux-x64.tar.gz
#由于我这儿里有jdk tar包，所以以下是解压部署
#解压文件
root@kylin:~/scpfile# tar -zxvf jdk-8u181-linux-x64.tar.gz -C /usr/java/
root@kylin:~/scpfile# cd /usr/java/
#更改文件夹用户组和用户名
root@kylin:/usr/java# chown -R root:root jdk1.8.0_181/
root@kylin:/usr/java# ln -sf /usr/java/jdk1.8.0_181 /usr/java/latest
root@kylin:/usr/java# ln -sf /usr/java/latest /usr/java/default
root@kylin:/usr/java# ll
total 12
drwxr-xr-x  3 root root 4096 May 13 17:29 ./
drwxr-xr-x 15 root root 4096 May 13 17:12 ../
lrwxrwxrwx  1 root root   16 May 13 17:29 default -> /usr/java/latest/
drwxr-xr-x  7 root root 4096 Jul  7  2018 jdk1.8.0_181/
lrwxrwxrwx  1 root root   22 May 13 17:29 latest -> /usr/java/jdk1.8.0_181/
root@kylin:/usr/java# sudo vim /etc/profile
#---
#添加 Java 相关的环境变量到/etc/profile
export JAVA_HOME=/usr/java/latest
export PATH=$JAVA_HOME/bin:$PATH
#---
root@kylin:/usr/java# source /etc/profile
root@kylin:/usr/java# echo $JAVA_HOME
/usr/java/latest
root@kylin:/usr/java# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
root@kylin:/usr/java#

2.2.3 安装Cloudera Manager服务器

root@kylin:~# cd /var/cache/apt/archives/
root@ubuntu:/var/cache/apt/archives# sudo apt-get install supervisor
#在线安装，会先检测/var/cache/apt/archives/ 目录下的deb文件，上面下载就是提升安装速度，等同于离线速度
root@kylin:~# sudo apt-get install -yq cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
#或者直接安装离线deb
root@ubuntu:/var/cache/apt/archives# sudo dpkg -i \
cloudera-manager-daemons_6.3.1~1466458.ubuntu1804_all.deb \
cloudera-manager-agent_6.3.1~1466458.ubuntu1804_amd64.deb \
cloudera-manager-server_6.3.1~1466458.ubuntu1804_all.deb

2.2.4 安装和配置数据库(mysql)

root@kylin:~# sudo apt-get install mysql-server mysql-client libmysql-java

root@kylin:~# service mysql status
 * MySQL is stopped.
root@kylin:~# service mysql start
 * Starting MySQL database server mysqld   

root@kylin:~# netstat -an | grep 3306
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN
root@kylin:~# service mysql stop
 * Stopping MySQL database server mysqld                                                                                       [ OK ]
root@kylin:~# cat /etc/mysql/mysql.conf.d/mysqld.cnf | grep "bind-address"
bind-address            = 127.0.0.1
root@kylin:~# sed -i 's/bind-address/#bind-address/' /etc/mysql/mysql.conf.d/mysqld.cnf
root@kylin:~# cat /etc/mysql/mysql.conf.d/mysqld.cnf | grep "bind-address"
#bind-address           = 127.0.0.1
root@kylin:~# mv /var/lib/mysql
mysql/         mysql-files/   mysql-keyring/ mysql-upgrade/
root@kylin:~# mv /var/lib/mysql/ib_logfile* /home/ubuntu1804/mysql_logfile/
root@kylin:~# cd /etc/mysql/
root@kylin:/etc/mysql# ll
total 32
drwxr-xr-x  4 root root 4096 May 15 07:09 ./
drwxr-xr-x 99 root root 4096 May 15 07:09 ../
drwxr-xr-x  2 root root 4096 May 15 07:03 conf.d/
-rwxr-xr-x  1 root root  120 Apr 30 00:40 debian-start*
-rw-------  1 root root  317 May 15 07:09 debian.cnf
lrwxrwxrwx  1 root root   24 May 15 07:03 my.cnf -> /etc/alternatives/my.cnf
-rw-r--r--  1 root root  839 Aug  3  2016 my.cnf.fallback
-rw-r--r--  1 root root  682 Jan 12  2018 mysql.cnf
drwxr-xr-x  2 root root 4096 May 15 07:14 mysql.conf.d/
root@kylin:/etc/mysql# cat my.cnf
#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

#
# * IMPORTANT: Additional settings that can override those from this file!
#   The files must end with '.cnf', otherwise they'll be ignored.
#

!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/

# 配置
root@ubuntu:~# vim /etc/mysql/conf.d/mysqldump.cnf
root@ubuntu:~# vim /etc/mysql/mysql.conf.d/mysqld.cnf
root@ubuntu:~# root@ubuntu:~# cp /etc/mysql/mysql.conf.d/mysqld.cnf /etc/mysql/mysql.conf.d/mysqld.cnf.bak
root@ubuntu:~# vim /etc/mysql/mysql.conf.d/mysqld.cnf

root@kylin:/etc/mysql# cat conf.d/mysqldump.cnf
[mysqldump]
quick
quote-names
max_allowed_packet      = 16M
root@kylin:/etc/mysql# vim conf.d/mysqldump.cnf
root@kylin:/etc/mysql# vim mysql.conf.d/mysqld.cnf
root@kylin:/etc/mysql# cat mysql.conf.d/mysqld.cnf
#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

[mysqld_safe]
socket          = /var/run/mysqld/mysqld.sock
nice            = 0

#log-error=/var/log/mysqld.log
#pid-file=/var/run/mysqld/mysqld.pid

sql_mode=STRICT_ALL_TABLES

[mysqld]
#
# * Basic Settings
#
user            = mysql
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
port            = 3306
basedir         = /usr
datadir         = /var/lib/mysql
tmpdir          = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking

# Here is an option file with Cloudera recommended settings:
#datadir=/var/lib/mysql
#socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED

symbolic-links = 0

key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550

log_bin=/var/lib/mysql/mysql_binary_log

server_id=1

binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address           = 127.0.0.1
#
# * Fine Tuning
#
#key_buffer_size                = 16M
#max_allowed_packet     = 16M
#thread_stack           = 192K
#thread_cache_size       = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover-options  = BACKUP
#max_connections        = 100
#table_open_cache       = 64
#thread_concurrency     = 10
#
# * Query Cache Configuration
#
#query_cache_limit      = 1M
#query_cache_size        = 16M
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Here you can see queries with especially long duration
#slow_query_log         = 1
#slow_query_log_file    = /var/log/mysql/mysql-slow.log
#long_query_time = 2
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#       other settings you may need to change.
#server-id              = 1
#log_bin                        = /var/log/mysql/mysql-bin.log
expire_logs_days        = 10
max_binlog_size   = 100M
#binlog_do_db           = include_database_name
#binlog_ignore_db       = include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem

root@kylin:~# service mysql start
 * Starting MySQL database server mysqld                                                                                       [ OK ]
root@kylin:~# sudo chkconfig mysql on

root@kylin:~# mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 6
Server version: 5.7.30-0ubuntu0.18.04.1-log (Ubuntu)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select host,user,authentication_string,plugin from user;
+-----------+------------------+-------------------------------------------+-----------------------+
| host      | user             | authentication_string                     | plugin                |
+-----------+------------------+-------------------------------------------+-----------------------+
| localhost | root             |                                           | auth_socket           |
| localhost | mysql.session    | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password |
| localhost | mysql.sys        | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password |
| localhost | debian-sys-maint | *A34B9AEB851FB45CDAB91FD4AFE2DE0F455D9672 | mysql_native_password |
+-----------+------------------+-------------------------------------------+-----------------------+
4 rows in set (0.00 sec)

mysql> update mysql.user set authentication_string=password('123456'),plugin="mysql_native_password" where user='root' and Host ='localhost';    #修改密码123456是密码
Query OK, 1 row affected, 1 warning (0.02 sec)
Rows matched: 1  Changed: 1  Warnings: 1

mysql> select host,user,authentication_string,plugin from user;
+-----------+------------------+-------------------------------------------+-----------------------+
| host      | user             | authentication_string                     | plugin                |
+-----------+------------------+-------------------------------------------+-----------------------+
| localhost | root             | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | mysql_native_password |
| localhost | mysql.session    | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password |
| localhost | mysql.sys        | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password |
| localhost | debian-sys-maint | *A34B9AEB851FB45CDAB91FD4AFE2DE0F455D9672 | mysql_native_password |
+-----------+------------------+-------------------------------------------+-----------------------+
4 rows in set (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.01 sec)

mysql> exit;
Bye

mysql -u root --password='123456' -e "create database scm default character set utf8 DEFAULT COLLATE utf8_general_ci"
mysql -u root --password='123456' -e "create database navms default character set utf8 DEFAULT COLLATE utf8_general_ci" 
mysql -u root --password='123456' -e "create database nav default character set utf8 DEFAULT COLLATE utf8_general_ci" 
mysql -u root --password='123456' -e "create database amon default character set utf8 DEFAULT COLLATE utf8_general_ci" 
mysql -u root --password='123456' -e "create database rman default character set utf8 DEFAULT COLLATE utf8_general_ci" 
mysql -u root --password='123456' -e "create database hive default character set utf8 DEFAULT COLLATE utf8_general_ci" 
mysql -u root --password='123456' -e "create database oozie default character set utf8 DEFAULT COLLATE utf8_general_ci" 
mysql -u root --password='123456' -e "create database hue default character set utf8 DEFAULT COLLATE utf8_general_ci" 
mysql -u root --password='123456' -e "create database sentry default character set utf8 DEFAULT COLLATE utf8_general_ci" 
mysql -u root --password='123456' -e "create user 'scm'@'%' identified by '123456'" 
mysql -u root --password='123456' -e "create user 'navms'@'%' identified by '123456'" 
mysql -u root --password='123456' -e "create user 'nav'@'%' identified by '123456'" 
mysql -u root --password='123456' -e "create user 'amon'@'%' identified by '123456'" 
mysql -u root --password='123456' -e "create user 'rman'@'%' identified by '123456'" 
mysql -u root --password='123456' -e "create user 'hive'@'%' identified by '123456'" 
mysql -u root --password='123456' -e "create user 'oozie'@'%' identified by '123456'" 
mysql -u root --password='123456' -e "create user 'hue'@'%' identified by '123456'" 
mysql -u root --password='123456' -e "create user 'sentry'@'%' identified by '123456'" 
mysql -u root --password='123456' -e "grant all privileges on scm.* to 'scm'@'%'" 
mysql -u root --password='123456' -e "grant all privileges on navms.* to 'navms'@'%'" 
mysql -u root --password='123456' -e "grant all privileges on nav.* to 'nav'@'%'"

root@kylin:~# mysql -u root --password='123456' -e "select host,user,authentication_string,plugin from mysql.user;"
mysql: [Warning] Using a password on the command line interface can be insecure.
+-----------+------------------+-------------------------------------------+-----------------------+
| host      | user             | authentication_string                     | plugin                |
+-----------+------------------+-------------------------------------------+-----------------------+
| localhost | root             | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | mysql_native_password |
| localhost | mysql.session    | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password |
| localhost | mysql.sys        | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password |
| localhost | debian-sys-maint | *A34B9AEB851FB45CDAB91FD4AFE2DE0F455D9672 | mysql_native_password |
| %         | scm              | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | mysql_native_password |
| %         | navms            | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | mysql_native_password |
| %         | nav              | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | mysql_native_password |
| %         | amon             | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | mysql_native_password |
| %         | rman             | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | mysql_native_password |
| %         | hive             | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | mysql_native_password |
| %         | oozie            | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | mysql_native_password |
| %         | hue              | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | mysql_native_password |
| %         | sentry           | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | mysql_native_password |
+-----------+------------------+-------------------------------------------+-----------------------+
root@kylin:~# mysql -u root --password='123456' -e "flush privileges;"
mysql: [Warning] Using a password on the command line interface can be insecure.
root@kylin:~# netstat -an | grep 3306
tcp6       0      0 :::3306                 :::*                    LISTEN

2.2.5 设置Cloudera Manager数据库

#root@kylin:~# sudo /opt/cloudera/cm/schema/scm_prepare_database.sh -h cloudera mysql scm scm 123456
root@kylin:~# sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm 123456
JAVA_HOME=/usr/java/jdk1.8.0_181
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/java/jdk1.8.0_181/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Fri May 15 08:11:35 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2020-05-15 08:11:37,129 [main] INFO  com.cloudera.enterprise.dbutil.DbCommandExecutor  - Successfully connected to database.
All done, your SCM database is configured correctly!

2.3 安装CDH及其他外围应用

2.3.1 配置相关资源库

## cdh 3.6.2 and lower
sudo mkdir -p /var/www/html/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/cdh6/6.3.2/parcels/ -P /var/www/html/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/gplextras6/6.3.2/parcels/ -P /var/www/html/cloudera-repos
sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cdh6
sudo chmod -R ugo+rX /var/www/html/cloudera-repos/gplextras6

## Spark2
sudo mkdir -p /var/www/html/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/spark2/parcels/2.3.0.cloudera3/ -P /var/www/html/cloudera-repos
sudo chmod -R ugo+rX /var/www/html/cloudera-repos/spark2

## sqoop
sudo mkdir -p /var/www/html/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories http://archive.cloudera.com/sqoop-connectors/parcels/latest/ -P /var/www/html/cloudera-repos
sudo chmod -R ugo+rX /var/www/html/cloudera-repos/sqoop-connectors

需要修改.sha1为.sha

2.3.2 cloudera-scm-server启动

root@kylin:~# sudo service cloudera-scm-server start

root@kylin:~# sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

#查看启动日志，等待Jetty启动完成
#2020-05-17 04:39:43,307 INFO WebServerImpl:org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@212279a1{HTTP/1.1,[http/1.1]}{0.0.0.0:7180}
#2020-05-17 04:39:43,311 INFO WebServerImpl:org.eclipse.jetty.server.Server: Started @609468ms
#2020-05-17 04:39:43,311 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

2.3.3 CM安装外围应用

浏览器访问：http：// <服务器主机>：7180
username：admin
passwd：admin

登录后，安装向导将启动。

ddttoop
关注
专栏目录
Ubuntu Bionic 18.04安装docker
网络架构
01-19 205
Ubuntu Bionic 18.04安装docker操作系统要求卸载旧版本 ：设置存储库添加 Docker 的阿里云 GPG 密钥：向 source.list 中添加Docker软件阿里云 源安装引擎库列出存储库中可用的版本：使用第二列中的版本字符串安装特定版本，例如 `5:18.09.1~3-0~ubuntu-xenial`通过运行映像验证 Docker 引擎安装是否正确。hello-world 操作系统要求 要安装 Docker 引擎，您需要这些 Ubuntu 版本的 64 位版本： Ubuntu
【Ubuntu 18.04】安装 显卡驱动+CUDA+CuDNN 完全记录 (以 RTX 3090 为例)
闻韶
11-02 8302
新机安装 RTX 3090 显卡驱动 + CUDA + CuDNN 小记
ubuntu18安装CDH6.3.1
zhdan~的专栏
02-02 3455
ubuntu18安装CDH6.3.1
linux ubuntu bionic,在Ubuntu 18.04 Bionic Beaver Linux上安装和设置KVM
weixin_31766871的博客
05-15 260
目的在Ubuntu 18.04 Bionic Beaver上通过桥接网络和virt-manager安装KVM发行版Ubuntu 18.04要求具有root特权的Ubuntu 18.04的有效安装困难介质约定＃-要求linux命令可以直接以root用户身份或通过使用root特权以root特权执行sudo命令$-要求linux命令以普通非特权用户身份执行介绍KVM是Linux内核自己的虚拟化解决方案。...
在Ubuntu18.04(Bionic)上安装ROS Melodic
随遇而安的dandelion的博客
11-04 3094
本文介绍了在Ubuntu18.04（Bionic）操作系统上安装ROS Melodic的过程，以及在安装过程中遇到的问题和解决方法。
今天开始，ubuntu 18.04 bionic 主机不能从 ppa:deadsnakes 仓库下载 Python 包
LeoHsiao的博客
06-06 1274
ppa:deadsnakes 仓库今天停止维护旧版本 Ubuntu ，删除了针对 Ubuntu bionic 18.04 的所有 Python 预编译包。2023年6月6日 官方公告因此主机操作系统是 Ubuntu 18.04 bionic 的用户，不能从 ppa:deadsnakes 仓库下载 Python 包。而主机操作系统是 Ubuntu 20.04 focal、Ubuntu 22.04 jammy 的用户，可以继续从 ppa:deadsnakes 仓库下载 Python 包。
Ubuntu18.04 部署cdh6.3.2文档
最新发布
07-04
### Ubuntu18.04部署CDH6.3.2文档知识点详解 #### 环境要求与资源准备 **环境要求：** - 内存：至少16GB，建议32GB或更高。 - 操作用户：root。 **资源准备：** 1. **CM6.3.2 (Ubuntu18.04)安装包获取：** - ...
ubuntu18.04+安装GPU++CUDA+cuDNN
09-07
在本部分内容中，将对如何在Ubuntu 18.04系统上安装GPU、CUDA以及cuDNN进行详细介绍，内容涵盖了必要的编译器安装、显卡驱动安装、CUDA与cuDNN的版本选择与安装过程，以及开发环境IDE的安装。 首先，为了进行GPU...
Ubuntu18.04安装anaconda+CUDA+cudnn+pytorch过程记录
SarahYu1997的博客
07-10 2948
Ubuntu18.04安装anaconda+CUDA+pytorch过程记录anacondaCUDA anaconda 问题：安装完anaconda后，在terminal中输入anconda，查无此命令。 原因：没有配置环境变量 解决：在桌面进入terminal，输入vim .bashrc，编辑此文件。进入文件后，按i进入插入模式，将光标移到文件末尾，另起一行，输入export PATH=~/anaconda3/bin:$PATH，按esc退出插入模式，输入:wq!，保存并退出文件。 注：vim的相关操作，
Ubuntu18.04安装之后的配置+优化-附件资源
03-05
Ubuntu18.04安装之后的配置+优化-附件资源
cdh6.3.1+cm6.3.1安装包-cdh安装包大全
09-05
cdh6.3.1 + cm6.3.1版本网盘地址，操作系统为Redhat7和Centos7及以上，链接失效可私信，其他版本也可以私信
CDH6.3.2安装包及安装教程网盘下载地址.txt
04-10
从2021年1月31日开始，所有Cloudera软件都需要有效的订阅，并且只能通过付费墙进行访问。也就是说CDH各版本都不能免费获取了。 安装教程中详细说明了安装过程中遇到的各种问题，并且整个过程不需要依赖外网，全部为离线安装。 网盘下载《CDH6.3.2安装包及安装教程网盘下载地址.txt》，包括： manifest.json CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256 cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel.sha CentOS7.6下CDH6.3.2离线安装教程(正式环境).docx
cdh6.3.1离线完整版（百度网盘下载）
08-04
CDH安装包
cdh6.3.2+cm6.3.1.zip下载
07-07
CDH官方的网站已经无法直接下载安装包了（需要账号密码）， 此处版本为：【CM-6.3.1】，【CDH-6.3.2】 包含所有安装所需文件（含allkeys.asc以及manifest.json）。
ubuntu的镜像源+bionic版本
m0_69253695的博客
04-27 939
查找和你自己ubuntu版本匹配的版本号。可以看到我这个版本号的代号是jammy。每个版本的镜像文件都是有规律的。bionic版本的源如下。查看自己系统上的版本号。
Ubuntu18.04LTS搭建CDH6.3.0环境-版本二
ITMAN009的博客
07-30 1415
CM在线安装版本 注意事项 0.最好是干净的环境 1.root身份 2.建议主server在线安装 下载好资源后(默认下载到的位置/var/cache/apt/archives) 其他节点直接发过去安装包.deb 然后进行本地安装 dpek --install xxx.deb 4.sudo apt-get install openjdk-8-jdk(server节点被这个替换了 不然总是提示找不打java) 3.mysql-connector-java-8.20.jar 最好
CDH6.3.1安装指南
Aisen_bear的博客
04-21 2707
CDH安装指南！！！！ CDH简介 CDH基于Web的用户界面,支持大多数Hadoop组件，包括HDFS、MapReduce、Hive、Pig、 HBase、Zookeeper、Sqoop，简化了大数据平台的安装、使用难度。 Cloudera Manager的功能： 管理：对集群进行管理，如添加、删除节点等操作。 监控：监控集群的健康情况，对设置的各种指标和系统运行情况进行全面监控。 诊断：对集群出现的问题进行诊断，对出现的问题给出建议解决方案。 集成：多组件进行整合 《准备工作》 CDH安装包网盘
linux ubuntu bionic,如何升级Ubuntu到18.04 LTS Bionic Beaver
weixin_42101056的博客
05-15 358
目的将现有的Ubuntu安装升级至18.04 LTS Bionic Beaver版本您需要一个现有的Ubuntu 16.04 LTS或17.10安装。要求现有的Ubuntu 16.04 LTS或17.10以root权限安装。困难简单约定＃ - 要求以root权限直接以root用户或使用sudo命令执行给定的命令$ - 给定命令将作为普通非特权用户执行介绍最新的Ubuntu发布版，18.04，标志着...
Ubuntu 18.04 安装GPU+CUDA+cuDNN指南
"该资源主要介绍了如何在Ubuntu 18.04系统上进行GPU+CUDA+cuDNN的安装和配置，特别是针对CUDA 9.1版本的安装，包括GCC降级、CUDA主文件及补丁的安装，以及catia VBA二次开发中的编译连接设置。" 在编译连接CATIA ...
评论 36
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值