配置需求
- 操作系统:CentOS7.X
- 内存:128GB
- CPU:32核*2
- 硬盘:10TB
第三方软件
- JDK(http://jdk.java.net/)
- Kafka(https://kafka.apache.org/)
- Elasticsearch(https://www.elastic.co/)
- Neo4j(https://neo4j.com/)
- Redis(https://redis.io/)
- MySQL(https://www.mysql.com/)
- MinIO(https://min.io/)
- Flink(https://flink.apache.org/)
- vector(https://vector.dev/)
- rsyslog(https://www.rsyslog.com/)
- Spring Boot(https://spring.io/projects/spring-boot)
- Caddy Server(https://caddyserver.com/)
- Docker(https://docs.docker.com/engine/install/centos/)
安装步骤
环境准备
常用工具安装
sudo yum install vim
sudo yum install net-tools
sudo yum install unzip
新建用户
adduser test
设置密码
passwd test
切换test用户
su test
新建目录结构
mkdir ~/software
mkdir ~/soar
mkdir ~/data
配置Host
sudo vim /etc/hosts
增加如下内容
127.0.0.1 soar01
约定目录结构
将所有的软件、数据放到用户家目录下:
-
~/software:存放所有第三放软件
-
~/data:存放所有数据
- ~/data/es-data:Elasticsearch数据目录
- ~/data/kafka-data:Kafka数据目录
- ~/data/zk-data:Zookeeper数据目录
- ~/data/minio-data:MinIO数据目录
- ~/data/rsyslog-data:rsyslog数据目录
-
~/soar:存放所有Spring Boot项目和Flink Job项目
-
~/soar/soar-admin:soar-admin项目
- ~/soar/soar-admin/bin:启动脚本
- ~/soar/soar-admin/config:配置文件
-
~/soar/soar-persistent:soar-persistent项目
-
JDK安装
下载JDK11
https://download.java.net/openjdk/jdk11/ri/openjdk-11+28_linux-x64_bin.tar.gz
下载JDK17
https://download.oracle.com/java/17/latest/jdk-17_linux-x64_bin.tar.gz
移动到~/software目录
mv openjdk-11+28_linux-x64_bin.tar.gz /home/test/software/
解压
cd /home/test/software
tar zxvf openjdk-11+28_linux-x64_bin.tar.gz
配置JAVA_HOME环境变量
sudo vim /etc/profile
在文件最后添加如下行
export JAVA_HOME=/home/test/software/jdk-11
export PATH=$JAVA_HOME/bin:$PATH
环境变量生效
source /etc/profile
Zookeeper单机部署
下载
https://dlcdn.apache.org/zookeeper/zookeeper-3.7.0/apache-zookeeper-3.7.0-bin.tar.gz
移动到~/software目录
mv apache-zookeeper-3.7.0-bin.tar.gz /home/test/software/
解压
cd /home/test/software/
tar zxvf apache-zookeeper-3.7.0-bin.tar.gz
修改配置
cd apache-zookeeper-3.7.0-bin
cp conf/zoo_sample.cfg conf/zoo.cfg
vim conf/zoo.cfg
修改如下配置
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/test/data/zoo-data
clientPort=2181
server.1=soar01:2888:3888
新建数据存储目录
mkdir -p /home/test/data/zoo-data
创建myid
echo 1 > /home/test/data/zoo-data/myid
启动
bin/zkServer.sh start
bin/zkServer.sh status
Kafka单机部署
下载
https://archive.apache.org/dist/kafka/2.8.1/kafka_2.13-2.8.1.tgz
移动到~/software目录
mv kafka_2.13-2.8.1.tgz /home/test/software/
解压
cd /home/test/software/
tar zxvf kafka_2.13-2.8.1.tgz
修改配置
cd kafka_2.13-2.8.1
vim config/server.properties
修改如下配置
broker.id=1
host.name=soar01
log.dirs=/home/test/data/kafka-data
log.retention.hours=1
zookeeper.connect=soar01:2181/kafka
创建数据目录
mkdir -p /home/test/data/kafka-data
启动
bin/kafka-server-start.sh -daemon config/server.properties
Elasticsearch单机部署
下载
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.15.1-linux-x86_64.tar.gz
移动到~/software目录
mv elasticsearch-7.15.1-linux-x86_64.tar.gz /home/test/software/
解压
cd /home/test/software/
tar zxvf elasticsearch-7.15.1-linux-x86_64.tar.gz
修改ES_JAVA_HOME环境变量
vim ~/.bashrc
在最后添加
export ES_JAVA_HOME=/home/test/software/elasticsearch-7.15.1/jdk
环境变量生效
source ~/.bashrc
修改配置
cd elasticsearch-7.15.1
vim config/elasticsearch.yml
cluster.name: soar-es
node.name: soar01
path.data: /home/test/data/elastic-data
network.host: soar01
discovery.seed_hosts: ["soar01"]
cluster.initial_master_nodes: ["soar01"]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
创建数据目录
mkdir -p /home/test/data/elastic-data
系统配置修改
- max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
每个进程最大同时打开文件数太小,修改/etc/security/limits.conf
文件,增加配置,用户退出后重新登录生效
vim /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
- max number of threads [3818] for user [es] is too low, increase to at least [4096]
问题同上,最大线程个数太低。修改配置文件/etc/security/limits.conf
,增加配置
* soft nproc 4096
* hard nproc 4096
可通过命令查看
ulimit -Hu
ulimit -Su
- max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
修改/etc/sysctl.conf
文件,增加配置
vim /etc/sysctl.conf
vm.max_map_count = 262144
执行命令sysctl -p
生效
sysctl -p
生成证书
bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""
启动
bin/elasticsearch -d
设置密码
bin/elasticsearch-setup-passwords auto
这将会为不同的内部堆栈用户生成随机密码。或者,您也可以跳过 auto 参数,改为通过 interactive 参数手动定义密码。请记录这些密码,我们很快就会再次用到这些密码。
Changed password for user apm_system
PASSWORD apm_system = IRyTBui8lHx3DIrAddaN
Changed password for user kibana_system
PASSWORD kibana_system = kWje2eiWafU11WOwBUPr
Changed password for user kibana
PASSWORD kibana = kWje2eiWafU11WOwBUPr
Changed password for user logstash_system
PASSWORD logstash_system = nz5TL5iJonlRBJJto1zn
Changed password for user beats_system
PASSWORD beats_system = 04eczUb33BMz2BtLKwsC
Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = 7rKANkqHq1KNzUdZjwFj
Changed password for user elastic
PASSWORD elastic = HEj0u3t5gKNsLPE41gsK
安装Neo4j
解压
cd ~/soar/software
tar zxvf neo4j-community-4.4.4-unix.tar.gz
修改配置
cd neo4j-community-4.4.4
vim conf/neo4j.conf
修改配置为
dbms.directories.data
为Neo4j数据目录配置为家目录下的soar/soar-data/neo4j-data
目录。
dbms.directories.data=/zkqa/soar/soar-data/neo4j-data
dbms.memory.heap.initial_size=8g
dbms.memory.heap.max_size=8g
dbms.default_listen_address=0.0.0.0
启动
bin/neo4j start
修改密码
通过浏览器访问
http://soar02:7474
默认用户名:neo4j
默认密码:neo4j
修改密码
lsjdfl*lkjldf32VBN
Redis单机部署
下载
https://download.redis.io/releases/redis-6.2.6.tar.gz
cd redis-6.2.5
make
vim redis.conf
bind soar01
requirepass 4klsadfIE832.sda
daemonize yes
MySQL单机部署
下载
https://dev.mysql.com/get/Downloads/MySQL-8.0/mysql-8.0.27-1.el7.x86_64.rpm-bundle.tar
解压
tar xvf mysql-8.0.27-1.el7.x86_64.rpm-bundle.tar
安装
sudo rpm -ivh *.rpm --force --nodeps
初始化
mysqld --initialize
ALTER USER 'root'@'localhost' IDENTIFIED BY 'slmyq^8002IST';
MinIO单机部署
下载
https://dl.min.io/server/minio/release/linux-amd64/minio
修改环境变量
export MINIO_ACCESS_KEY=minioadmin
export MINIO_SECRET_KEY=sdlf#ljfjdasl.@
启动
minio server minio-data/ &
Flink部署
下载
https://www.apache.org/dyn/closer.lua/flink/flink-1.14.0/flink-1.14.0-bin-scala_2.12.tgz
修改配置
vim flink-conf.yaml
taskmanager.host: localhost
Vector部署
下载
https://packages.timber.io/vector/0.17.3/vector-0.17.3-x86_64-unknown-linux-gnu.tar.gz
配置文件
- dns.toml(dns日志推送配置)
data_dir = ".dns"
[sources.file]
type = "file"
include = ["/data/data-logs/dns/**"]
read_from = "beginning"
remove_after_secs = 60
max_line_bytes = 3097152
[sinks.kafka]
bootstrap_servers = "soar01:9092,soar02:9092,soar03:9092"
group_id = "soar01-dns"
topic = "dns-log"
type = "kafka"
inputs = ["file"]
encoding.codec = "text"
- dpi.toml(dpi日志推送配置)
data_dir = ".dpi"
[sources.file]
type = "file"
include = ["/data/test/data-logs/dpi/**"]
read_from = "beginning"
remove_after_secs = 60
max_line_bytes = 3097152
[sinks.kafka]
bootstrap_servers = "soar01:9092,soar02:9092,soar03:9092"
group_id = "soar01-dpi"
topic = "dpi-log"
type = "kafka"
inputs = ["file"]
encoding.codec = "text"
启动
nohup vector -c dns.toml &
nohup vector -c dpi.toml &
网络策略
开放端口
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.181" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.182" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.183" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.184" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.185" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.186" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.187" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.188" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.189" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.190" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.173.6.234" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.173.6.235" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.173.100.101" port protocol="tcp" port="9092" accept"
firewall-cmd --reload
开放端口
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.10.2.120" port protocol="tcp" port="9300" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.10.2.121" port protocol="tcp" port="9300" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.10.2.122" port protocol="tcp" port="9300" accept"
firewall-cmd --zone=public --add-port=9200/tcp --permanent
firewall-cmd --reload
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.184" port protocol="tcp" port="6379" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.185" port protocol="tcp" port="6379" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.171.80.186" port protocol="tcp" port="6379" accept"
firewall-cmd --reload
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.174.220.107" port protocol="tcp" port="8383" accept"
firewall-cmd --reload
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.174.220.106" port protocol="tcp" port="8383" accept"
firewall-cmd --reload
firewall-cmd --zone=public --add-port=8081/tcp --permanent
firewall-cmd --reload
firewall-cmd --zone=public --add-port=7474/tcp --permanent
firewall-cmd --reload
firewall-cmd --zone=public --add-port=7687/tcp --permanent
firewall-cmd --reload
firewall-cmd --zone=public --remove-port=8081/tcp --permanent
firewall-cmd --reload
firewall-cmd --zone=public --remove-port=7474/tcp --permanent
firewall-cmd --reload
firewall-cmd --zone=public --remove-port=7687/tcp --permanent
firewall-cmd --reload
firewall-cmd --zone=public --add-port=514/udp --permanent
firewall-cmd --zone=public --add-port=514/tcp --permanent
firewall-cmd --zone=public --add-port=9092/tcp --permanent
firewall-cmd --zone=public --add-port=22/tcp --permanent
firewall-cmd --reload
firewall-cmd --zone=public --remove-port=9200/tcp --permanent
firewall-cmd --reload
flink run formatter-test-http-alert-0.0.1.jar -bootstrap.servers 10.211.88.68:9092
flink run formatter-nsfocus-waf-0.0.1.jar -bootstrap.servers 10.211.88.68:9092
flink run accuracy-mark-0.0.1.jar -bootstrap.servers 10.211.88.68:9092
flink run control-mark-0.0.1.jar -bootstrap.servers 10.211.88.68:9092
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.174.220.107" port protocol="tcp" port="8081" accept"
firewall-cmd --reload
firewall-cmd --permanent --remove-rich-rule="rule family="ipv4" source address="10.174.220.107" port protocol="tcp" port="8081" accept"
firewall-cmd --reload
日志接收
rsyslog配置
日志存放目录结构
/home/test/data/data-logs/dns/
/home/test/data/data-logs/dpi/
Docker部署
Docker安装使用root,客户端访问使用普通用户
配置IPv4 forwarding
vim /etc/sysctl.conf
配置转发
net.ipv4.ip_forward=1
重启服务,让配置生效
systemctl restart network
查看是否成功,如果返回为“net.ipv4.ip_forward = 1”则表示成功
sysctl net.ipv4.ip_forward
重启docker服务
service docker restart
Spring Boot项目部署
Spring Boot项目部署
部署目录结构
- springboot-test
- bin
- startup.sh
- shutdown.sh
- config
- application.properties
- logs
- info/info.log
- warn/warn.log
- error/error.log
- trace/trace.log
- springboot-test.jar
启动
cd soar-persistence
bin/startup.sh
startup.sh
#!/bin/bash
JAVA_HOME=/root/software/jdk11
basepath=$(cd `dirname $0`; pwd)
cd $basepath
cd ..
nohup $JAVA_HOME/bin/java -Xmx1G -Xms1G -server -XX:+UseG1GC -jar *.jar > /dev/null 2>&1 &
exit
停止
cd soar-persistence
bin/shutdown.sh
shutdown.sh
#!/bin/sh
basepath=$(cd `dirname $0`; pwd)
cd $basepath
cd ..
NAME=`ls *.jar`
if [ -z "$NAME" ]; then
echo "NAME is empty"
exit 0
fi
ID=`ps -ef | grep "$NAME" | grep -v "$0" | grep -v "grep" | awk '{print $2}'`
echo "Stopping $NAME..."
for id in $ID
do
kill -9 $id
echo "$NAME Stopped!"
done