CentOS 7，软件测试面试题及答案2024

2401_84240742

于 2024-04-13 05:14:42 发布

阅读量830

点赞数 19

分类专栏：程序员文章标签： centos linux 运维

本文链接：https://blog.csdn.net/2401_84240742/article/details/137702753

版权

程序员专栏收录该内容

17 篇文章 0 订阅

订阅专栏

Changed password for user beats_system
PASSWORD beats_system = rY90aBHjAdidQPwgX87u

Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = 0VxaGROqo255y60P1kBV

Changed password for user elastic
PASSWORD elastic = NvOBRGpUE3DoaSbYaUp3

echo “##### 测试es加密，查看集群状态是否为green #####”
[elkuser@es83 elasticsearch-7.2.0]$ curl --tlsv1 -XGET “https://192.168.100.83:9200/_cluster/health?pretty” --user elastic:NvOBRGpUE3DoaSbYaUp3 -k

05Kafka集群部署

Kafka 是最初由Linkedin公司开发，是一个分布式、分区的、多副本的、多订阅者，基于zookeeper协调的分布式消息系统；它具有高吞吐量、低延迟、可扩展性、持久性、可靠性、容错性和高并发等特点，可以处理几十万条消息，延迟只有几毫秒，集群式部署支持热扩展，消息可被持久化到本地磁盘，防止数据丢失，而且支持数千个客户端同时读写。

在本文的架构中，kafka是用作缓存消息队列，用来实时接收日志和发送日志到logstash，实现解耦和流量削峰，解决logstash消费能力跟不上导致的数据丢失问题；笔者采用的是kafka内置的zookeeper，也是以集群方式部署，无需再单独搭建zookeeper集群服务。
注意：kafka的集群配置信息，状态维护是存储在zookeeper这个进程里的，所以kafka在启动前需要先配置启动zookeeper！

笔者为zookeeper服务分配了4G内存，为kafka服务分配了31G内存和5块SSD磁盘，关于硬件配置大家根据实际情况来分配使用！

下载方式：wget https://archive.apache.org/dist/kafka/2.3.0/kafka_2.12-2.3.0.tgz

echo “##### 解压Kafka #####”
[root@es83 ~]# cd /opt/
[root@es83 opt]# tar -xvf ./kafka_2.12-2.3.0.tgz

echo “##### 修改zookeeper配置文件 #####”
[root@es83 opt]# cd ./kafka_2.12-2.3.0/
[root@es83 kafka_2.12-2.3.0]# cat > ./config/zookeeper.properties <<EOF
dataDir=/opt/zookeeper
clientPort=2181
maxClientCnxns=0
tickTime=2000
initLimit=10
syncLimit=5
server.1=192.168.100.83:2888:3888
server.2=192.168.100.86:2888:3888
server.3=192.168.100.87:2888:3888

authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000
4lw.commands.whitelist=
EOF

echo “##### 创建zookeeper数据目录和对应的myid文件 #####”
[root@es83 kafka_2.12-2.3.0]# mkdir /opt/zookeeper
[root@es83 kafka_2.12-2.3.0]# echo 1 > /opt/zookeeper/myid

echo “##### 修改kafka配置文件 #####”
[root@es83 kafka_2.12-2.3.0]# cat > ./config/server.properties <<EOF
broker.id=83
listeners=SASL_PLAINTEXT://192.168.100.83:9092
advertised.listeners=SASL_PLAINTEXT://192.168.100.83:9092
num.network.threads=5
num.io.threads=8
socket.send.buffer.bytes=1024000
socket.receive.buffer.bytes=1024000
socket.request.max.bytes=1048576000
log.dirs=/logdata/kfkdata1,/logdata/kfkdata2,/logdata/kfkdata3,/logdata/kfkdata4,/logdata/kfkdata5
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=72
log.segment.delete.delay.ms=1000
log.cleaner.enable=true
log.cleanup.policy=delete
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.100.83:2181,192.168.100.86:2181,192.168.100.87:2181
zookeeper.connection.timeout.ms=60000
group.initial.rebalance.delay.ms=0
delete.topic.enable=true

security.inter.broker.protocol=SASL_PLAINTEXT
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=true
super.users=User:admin;User:kafka
EOF

echo “##### 创建zk和kafka的sasl jaas文件 #####”
[root@es83 kafka_2.12-2.3.0]# cat > ./config/zk_server_jaas.conf <<EOF
Server {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=“admin”
password=“chilu@rljie”
user_kafka=“chilu@rljie”
user_producer=“chilu@rljie”;
};
EOF

[root@es83 kafka_2.12-2.3.0]# cat > ./config/kafka_server_jaas.conf <<EOF
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=“admin”
password=“chilu@rljie”
user_admin=“chilu@rljie”
user_producer=“chilu@rljie”
user_consumer=“chilu@rljie”;
};

KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=“kafka”
password=“chilu@rljie”;
};

Client {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=“kafka”
password=“chilu@rljie”;
};
EOF

echo “##### 修改zk和kafka的启动文件（增加SASL的环境配置） #####”
[root@es83 kafka_2.12-2.3.0]# sed -i -e ‘s/512M/4G/g’ -e ‘s#Xms4G#Xms4G -Djava.security.auth.login.config=/opt/kafka_2.12-2.3.0/config/zk_server_jaas.conf#’ ./bin/zookeeper-server-start.sh
[root@es83 kafka_2.12-2.3.0]# sed -i -e ‘s/1G/31G/g’ -e ‘s#Xms31G#Xms31G -Djava.security.auth.login.config=/opt/kafka_2.12-2.3.0/config/kafka_server_jaas.conf#’ ./bin/kafka-server-start.sh

echo “##### 将相关目录复制到其他两台节点上，并进行修改 #####”
[root@es83 kafka_2.12-2.3.0]# cd …/
[root@es83 opt]# scp -r ./zookeeper ./kafka_2.12-2.3.0 192.168.100.86:/opt/
[root@es83 opt]# scp -r ./zookeeper ./kafka_2.12-2.3.0 192.168.100.87:/opt/
[root@es83 opt]# ssh 192.168.100.86 “echo 2 > /opt/zookeeper/myid ; sed -i ‘1,3s/83/86/’ /opt/kafka_2.12-2.3.0/config/server.properties”
[root@es83 opt]# ssh 192.168.100.87 “echo 3 > /opt/zookeeper/myid ; sed -i ‘1,3s/83/87/’ /opt/kafka_2.12-2.3.0/config/server.properties”

echo “##### 后台启动zookeeper服务 #####”
[root@es83 opt]# cd ./kafka_2.12-2.3.0/
[root@es83 kafka_2.12-2.3.0]# ./bin/zookeeper-server-start.sh -daemon ./config/zookeeper.properties
[root@es83 kafka_2.12-2.3.0]# ssh 192.168.100.86 “/opt/kafka_2.12-2.3.0/bin/zookeeper-server-start.sh -daemon /opt/kafka_2.12-2.3.0/config/zookeeper.properties”
[root@es83 kafka_2.12-2.3.0]# ssh 192.168.100.87 “/opt/kafka_2.12-2.3.0/bin/zookeeper-server-start.sh -daemon /opt/kafka_2.12-2.3.0/config/zookeeper.properties”

echo “##### 后台启动kafka服务 #####”
[root@es83 kafka_2.12-2.3.0]# ./bin/kafka-server-start.sh -daemon ./config/server.properties
[root@es83 kafka_2.12-2.3.0]# ssh 192.168.100.86 “/opt/kafka_2.12-2.3.0/bin/kafka-server-start.sh -daemon /opt/kafka_2.12-2.3.0/config/server.properties”
[root@es83 kafka_2.12-2.3.0]# ssh 192.168.100.87 “/opt/kafka_2.12-2.3.0/bin/kafka-server-start.sh -daemon /opt/kafka_2.12-2.3.0/config/server.properties”

当zk和kafka服务都启动后，可以先检查下相关端口状态是否正常

[root@es83 kafka_2.12-2.3.0]# netstat -antlp | grep -E “2888|3888|2181|9092”

当集群服务一切正常后，即可在其中一台kafka节点上配置ACL访问控制权限，对生产者producer和消费者consumer的主题topic和组group设置访问权限，可以限制只允许指定的机器访问。

提示：下面的mykafka是通过/etc/hosts自定义一个IP的域名，例如：192.168.100.83 mykafka；如果写成localhost可能没有权限，执行命令后会报NoAuth；如果写成IP地址会报CONNECT !!!

echo “##### 编写配置ACL访问权限脚本 #####”
[root@es83 kafka_2.12-2.3.0]# cat > ./kfkacls.sh <<EOF
#!/bin/bash
/opt/kafka_2.12-2.3.0/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=mykafka:2181 --add --allow-principal User:producer --allow-host 0.0.0.0 --operation Read --operation Write --topic elk
/opt/kafka_2.12-2.3.0/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=mykafka:2181 --add --allow-principal User:producer --topic elk --producer --group chilu
/opt/kafka_2.12-2.3.0/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=mykafka:2181 --add --allow-principal User:consumer --allow-host 0.0.0.0 --operation Read --operation Write --topic elk
/opt/kafka_2.12-2.3.0/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=mykafka:2181 --add --allow-principal User:consumer --topic elk --consumer --group chilu
EOF

echo “##### 执行脚本 #####”
[root@es83 kafka_2.12-2.3.0]# bash ./kfkacls.sh

echo “##### 查看ACL权限列表 #####”
[root@es83 kafka_2.12-2.3.0]# ./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=mykafka:2181 --list

提示：下面是交互式的命令配置

echo “##### 增加ACL访问权限 #####”
[root@es83 kafka_2.12-2.3.0]# ./bin/zookeeper-shell.sh mykafka:2181
Welcome to ZooKeeper!
JLine support is disabled

WATCHER::

WatchedEvent state:SyncConnected type:None path:null

此时可以直接在这个控制台输入命令
如ls / 查看ZK的目录

检查默认权限
getAcl /

默认所有人可以查看
添加权限命令为：（仅添加kafka主机的IP）
setAcl / ip:192.168.100.83:cdrwa,ip:192.168.100.86:cdrwa,ip:192.168.100.87:cdrwa
setAcl /kafka-acl ip:192.168.100.83:cdrwa,ip:192.168.100.86:cdrwa,ip:192.168.100.87:cdrwa

检查是否生效
getAcl /
输出：
'ip,'192.168.100.83
: cdrwa
'ip,'192.168.100.86
: cdrwa
'ip,'192.168.100.87
: cdrwa

退出
quit

06Logstash服务部署

Logstash 是免费且开放的服务器端数据处理管道，采用的是可插拔框架，拥有200多个插件，支持各种输入和输出选择，能够实时解析和转换数据，具有可伸缩性、弹性和灵活性；但是它比较消耗资源，运行时占用较高的CPU和内存，如果缺少消息队列缓存，会有数据丢失的隐患，所以小伙伴们要结合自身情况来使用！

笔者在生产环境上，也为Logstash分配了30G内存，关于硬件配置大家根据实际情况来分配使用！

下载方式：wget https://artifacts.elastic.co/downloads/logstash/logstash-7.2.0.tar.gz

echo “##### 解压Logstash #####”
[root@es83 ~]# cd /home/elkuser/
[root@es83 elkuser]# tar -xvf ./logstash-7.2.0.tar.gz

echo “##### 修改启动内存 #####”
[root@es83 elkuser]# cd ./logstash-7.2.0/
[root@es83 logstash-7.2.0]# sed -i -e ‘s/1g/30g/g’ ./config/jvm.options

echo “##### 复制相关所需证书到logstash目录下 #####”
[root@es83 elkuser]# cd ./logstash-7.2.0/config/
[root@es83 config]# cp /home/elkuser/elasticsearch-7.2.0/root.pem ./

echo “##### 修改logstash配置文件 #####”
[root@es83 config]# cat > ./logstash.yml <<EOF
http.host: “192.168.100.83”
node.name: “logstash83”
xpack.monitoring.elasticsearch.hosts: [ “https://192.168.100.83:9200” ]
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: “elastic”
xpack.monitoring.elasticsearch.password: “NvOBRGpUE3DoaSbYaUp3”
xpack.monitoring.elasticsearch.ssl.certificate_authority: config/root.pem
xpack.monitoring.elasticsearch.ssl.verification_mode: certificate
xpack.monitoring.collection.interval: 30s
xpack.monitoring.collection.pipeline.details.enabled: true
EOF

提示：配置的用户名和密码要跟kafka配置的一致！

echo “##### 配置接入kafka的客户端文件 #####”
[root@es83 config]# cat > ./kafka-client-jaas.conf <<EOF
KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=“consumer”
password=“chilu@rljie”;
};
EOF

echo “##### input和ouput的配置示例 #####”
[root@es83 config]# cat > ./test.cfg <<EOF
input {
kafka {
bootstrap_servers => “192.168.100.83:9092,192.168.100.86:9092,192.168.100.87:9092”
client_id => “chilu83”
auto_offset_reset => “latest”
topics => “elk”
group_id => “chilu”
security_protocol => “SASL_PLAINTEXT”
sasl_mechanism => “PLAIN”
jaas_path => “/home/elkuser/logstash-7.2.0/config/kafka-client-jaas.conf”
}
}

filter {
}

output {
elasticsearch {
hosts => [“192.168.4.1:9200”,“192.168.4.2:9200”,“192.168.4.3:9200”]
user => “elastic”
password => “NvOBRGpUE3DoaSbYaUp3”
ssl => true
cacert => “/home/elkuser/logstash-7.2.0/config/root.pem”
index => “chilu_elk%{+YYYY.MM.dd}”
}
}
EOF

echo “##### 启动logstash服务 #####”
[root@es83 config]# …/bin/logstash -r -f ./test.cfg

07Kibana服务部署

Kibana 是一个开源的分析和可视化平台，可以为Logstash和ElasticSearch提供的日志数据进行高效的搜索、可视化汇总和多维度分析，并且与Elasticsearch索引中的数据进行交互；它基于浏览器的界面操作可以快速创建动态仪表板，实时监控ElasticSearch的数据状态与更改等。

笔者在生产环境上，为Kibana分配了8G内存，关于硬件配置大家根据实际情况来分配使用！

下载方式：wget https://artifacts.elastic.co/downloads/kibana/kibana-7.2.0-linux-x86_64.tar.gz

echo “##### 解压Kibana #####”
[root@es83 ~]# cd /home/elkuser/
[root@es83 elkuser]# tar -xvf kibana-7.2.0-linux-x86_64.tar.gz

echo “##### 修改启动内存 #####”
[root@es83 elkuser]# cd ./kibana-7.2.0-linux-x86_64/
[root@es83 kibana-7.2.0-linux-x86_64]# sed -i ‘s/warnings/warnings --max_old_space_size=8096/’ ./bin/kibana

echo “##### 复制相关所需证书到kibana目录下 #####”
[root@es83 kibana-7.2.0-linux-x86_64]# cd ./config/
[root@es83 config]# cp /home/elkuser/elasticsearch-7.2.0/client.p12 ./

echo “##### 利用client.p12证书生成其他所需证书 #####”
[root@es83 config]# openssl pkcs12 -in client.p12 -nocerts -nodes > client.key
Enter Import Password: 回车Enter
MAC verified OK

[root@es83 config]# openssl pkcs12 -in client.p12 -clcerts -nokeys > client.cer
Enter Import Password: 回车Enter
MAC verified OK

[root@es83 config]# openssl pkcs12 -in client.p12 -cacerts -nokeys -chain > client-ca.cer
Enter Import Password: 回车Enter
MAC verified OK

echo “##### 升级kibana的web界面为https访问 #####”
[root@es83 config]# cd …/
[root@es83 kibana-7.2.0-linux-x86_64]# openssl req -newkey rsa:2048 -nodes -keyout server.key -x509 -days 3650 -out server.crt -subj “/C=CN/ST=guangzhou/L=rljie/O=chilu/OU=linux/”

echo “##### 修改kibana的配置文件 #####”
[root@es83 kibana-7.2.0-linux-x86_64]# cat > ./config/kibana.yml <<EOF
server.name: kibana
server.host: “192.168.100.83”
elasticsearch.hosts: [ “https://192.168.100.83:9200” ]
xpack.monitoring.ui.container.elasticsearch.enabled: true
elasticsearch.username: “elastic”
elasticsearch.password: “NvOBRGpUE3DoaSbYaUp3”
xpack.security.enabled: true
elasticsearch.ssl.certificateAuthorities: config/client-ca.cer
elasticsearch.ssl.verificationMode: certificate
xpack.security.encryptionKey: “4297f44b13955235245b2497399d7a93”
xpack.reporting.encryptionKey: “4297f44b13955235245b2497399d7a93”
server.ssl.enabled: true
server.ssl.certificate: server.crt
server.ssl.key: server.key
EOF

echo “##### nohup后台启动kibana服务（自行选择后台方式） #####”
[root@es83 kibana-7.2.0-linux-x86_64]# nohup ./bin/kibana --allow-root &

完成以上操作后，可使用浏览器访问kibana地址https://192.168.100.83，输入elastic用户密码即可！

在这里插入图片描述

curl 示例

curl --tlsv1 -XGET ‘https://192.168.100.83:9200/_cluster/health?pretty’ --cacert ‘/home/elkuser/elasticsearch-7.2.0/root.pem’ --user elastic:NvOBRGpUE3DoaSbYaUp3