大数据入门第六天总结

1.MySQL的部署

注：一定要注意部署过程中，不同命令使用时的用户、目录是否正确
不同服务最好使用专用的用户来维护
如：
mysql服务 --》mysql用户去维护
hadoop服务 -》hadoop用户

cdh的hdfs服务–》hdfs用户
hbase服务 --》hbase用户

1.上传安装包

[root@ruozedata001 local]# cd /usr/local
[root@ruozedata001 local]# rz

2.检查是否有以前存在的正在运行的mysql进程

[root@ruozedata001 local]#ps -ef|grep mysqld

无进程，然后搜索已存在的安装包及依赖包

[root@ruozedata001 local]rpm -qa |grep -i mysql

3.解压以及建立软连接

[root@ruozedata001 local]# tar xzvf mysql-5.6.23-linux-glibc2.5-x86_64.tar.gz
[root@ruozedata001 local]# ln -s mysql-5.6.23-linux-glibc2.5-x86_64.targz mysql

4.创建用户及用户组

[root@ruozedata001 local]# groupadd -g 101 dba
[root@ruozedata001 local]# useradd -u 514 -g dba -G root -d /usr/local/mysql mysqladmin
[root@ruozedata001 local]# id mysqladmin
uid=514(mysqladmin) gid=101(dba) groups=101(dba),0(root)

-u 指定用户id
-g 主组
-G 附加组
-d 用户的家目录
此时，切换至新用户，命令行样式会丢失，此时应拷贝原用户家目录的个人环境变量配置文件至新用户家目录文件夹，再修改权限，用户及用户组

[root@ruozedata001 local]# cp /etc/skel/.* /usr/local/mysql/
[root@ruozedata001 local]# chown mysqladmin：dba .bash

5.配置文件

执行时按照此路径来寻找文件，因此一般配置在第一个文件夹中
#defualt start: /etc/my.cnf->/etc/mysql/my.cnf->SYSCONFDIR/my.cnf->$MYSQL_HOME/my.cnf-> --defaults-extra-file->~/my.cnf
①在etc目录创建新文件

[root@ruozedata001 local]# vi /etc/my.cnf

②此时文本中有默认数据，为免以后使用，先将此文件拷贝一份

[root@ruozedata001 local]# cp /etc/my.cnf /etc/my.cnf2020

③ 将需要的配置文件覆盖此文件夹

[root@ruozedata001 local]# vi /etc/my.cnf
命令行 gg dG 然后进入编辑模式粘贴以下文件
[client]
port            = 3306
socket          = /usr/local/mysql/data/mysql.sock
 
[mysqld]
port            = 3306
socket          = /usr/local/mysql/data/mysql.sock

skip-external-locking
key_buffer_size = 256M
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 4M
query_cache_size= 32M
max_allowed_packet = 16M
myisam_sort_buffer_size=128M
tmp_table_size=32M

table_open_cache = 512
thread_cache_size = 8
wait_timeout = 86400
interactive_timeout = 86400
max_connections = 600

# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 32

#isolation level and default engine 
default-storage-engine = INNODB
transaction-isolation = READ-COMMITTED

server-id  = 1
basedir     = /usr/local/mysql
datadir     = /usr/local/mysql/data
pid-file     = /usr/local/mysql/data/hostname.pid

#open performance schema
log-warnings
sysdate-is-now

binlog_format = MIXED
log_bin_trust_function_creators=1
log-error  = /usr/local/mysql/data/hostname.err
log-bin=/usr/local/mysql/arch/mysql-bin
#other logs
#general_log =1
#general_log_file  = /usr/local/mysql/data/general_log.err
#slow_query_log=1
#slow_query_log_file=/usr/local/mysql/data/slow_log.err

#for replication slave
#log-slave-updates 
#sync_binlog = 1

#for innodb options 
innodb_data_home_dir = /usr/local/mysql/data/
innodb_data_file_path = ibdata1:500M:autoextend
innodb_log_group_home_dir = /usr/local/mysql/arch
innodb_log_files_in_group = 2
innodb_log_file_size = 200M

innodb_buffer_pool_size = 2048M
innodb_additional_mem_pool_size = 50M
innodb_log_buffer_size = 16M

innodb_lock_wait_timeout = 100
#innodb_thread_concurrency = 0
innodb_flush_log_at_trx_commit = 1
innodb_locks_unsafe_for_binlog=1

#innodb io features: add for mysql5.5.8
performance_schema
innodb_read_io_threads=4
innodb-write-io-threads=4
innodb-io-capacity=200
#purge threads change default(0) to 1 for purge
innodb_purge_threads=1
innodb_use_native_aio=on

#case-sensitive file names and separate tablespace
innodb_file_per_table = 1
lower_case_table_names=1

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[mysqlhotcopy]
interactive-timeout

[myisamchk]
key_buffer_size = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

然后，若是服务器内存较小，可以酌情修改一个参数
#TUN product
innodb_buffer_pool_size =1024M

6.修改权限

先修改配置文件的权限、用户及用户组

[root@ruozedata001 local]# chown mysqladmin:dba /etc/my.cnf
[root@ruozedata001 local]# chmod 640 /etc/my.cnf

再修改目录的权限、用户及用户组

[root@ruozedata001 local]# chown -R mysqladmin:dba /usr/local/mysql
[root@ruozedata001 local]# chown -R mysqladmin:dba /usr/local/mysql/*
[root@ruozedata001 local]# chown -R mysqladmin:dba /usr/local/mysql-5.6.23-linux-glibc2.5-x86_64
[root@ruozedata001 local]# chmod -R 755 /usr/local/mysql
[root@ruozedata001 local]# chmod -R 755 /usr/local/mysql/*
[root@ruozedata001 local]# chmod -R 755 /usr/local/mysql-5.6.23-linux-glibc2.5-x86_64

切换用户 su - mysqladmin
存放binlog文件

[mysqladmin@ruozedata001 ~]$ mkdir arch
[mysqladmin@ruozedata001 ~]$  scripts/mysql_install_db \
--user=mysqladmin \
--basedir=/usr/local/mysql \
--datadir=/usr/local/mysql/data

切回root用户，安装缺少的组件

[root@ruozedata001 ~]# yum install -y perl
[root@ruozedata001 ~]#yum install -y autoconf
[root@ruozedata001 ~]#yum install -y libaio

赋予权限

[root@ruozedata001 ~]# cd /usr/local/mysql
[root@ruozedata001 ~]# cp support-files/mysql.server /etc/rc.d/init.d/mysql 
[root@ruozedata001 ~]# chmod +x /etc/rc.d/init.d/mysql
[root@ruozedata001 ~]# chkconfig --del mysql
[root@ruozedata001 ~]# chkconfig --add mysql
[root@ruozedata001 ~]# chkconfig --level 345 mysql on
[root@ruozedata001 ~]# vi /etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

**su - mysqladmin -c "/etc/init.d/mysql start --federated"**

"/etc/rc.local" 9L, 278C written

在touch /var/lock/subsys/local 这个命令后边添加一行
su - mysqladmin -c “/etc/init.d/mysql start --federated”
然后保存退出
切回mysqladmin用户

[mysqladmin@ruozedata001 ~]$ rm -rf my.cnf my.new.cnf
[mysqladmin@ruozedata001 ~]$ service mysql start
[mysqladmin@ruozedata001 ~]$ service mysql status

然后就进入Mysql进程了
查看进程与端口号
进程可以在当前用户直接查询 ps -ef |grep mysqld
端口号则必须在root用户才能查询
登陆用户

[mysqladmin@ruozedata001 ~]$ mysql -uroot -p
show databases；

输入密码可进入，命令行不能直接带上密码，否则会泄露
使用show databases;查询是否可用
查询密码 select host,user,password from user
修改密码

[mysqladmin@ruozedata001 ~]$ mysql -uroot -p
mysql> use mysql
mysql> update user set password=password('设置的密码') where user='root';

删除空账号

mysql> delete from user where user='';
mysql> select host,user,password from user;

删除后记得检查，并刷权限

mysql> flush privileges;

修改环境变量
退出mysql进程，在.bashrc中操作

[mysqladmin@ruozedata001 ~]$ vi .bashrc
 # .bashrc
 # Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

 # Uncomment the following line if you don't like systemctl's auto-paging feature:
 # export SYSTEMD_PAGER=

 # User specific aliases and functions

export MYSQL_HOME=/usr/local/mysql
export PATH=$MYSQL_HOME/bin:$PATH


PS1=`uname -n`":"'$USER'":"'$PWD'":>"; export PS1
[mysqladmin@ruozedata001 ~]$ source .bashrc
[mysqladmin@ruozedata001 ~]$ echo $MYSQL_HOME

.bashrc文件的最后一行可以将命令行直接显示路径.

部署完成

部署出现问题要查看命令执行的目录、用户

重新部署

①rm -rf arch/* data/*
arch文件夹就是存放binlog文件的

②scripts/mysql_install_db
–user=mysqladmin
–basedir=/usr/local/mysql
–datadir=/usr/local/mysql/data

注：binlog文件不会满，每隔一段时间会自动清理，自动删除机制

host 标识 user+password允许从哪个机器来访问本mysql的实例

注：[root@ruozedata001 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

前两行不能动

192.168.0.3 ruozedata001
内网ip hostname
ping hostname
shell脚本 程序 CDH部署 统统使用hostname
window 访问，IDEA开发 都是外网ip 统一的配置hostname