1、硬件环境
1.1 、网络拓扑图
1.2、对各个服务器的要求
2、实验一:验证单机mosquitto能够支持的最大定阅端数量
2.1、Linux 进程级开启最大文件描述符 调优
2.1.1、开启最大文件数
系统可以开启的最大文件描述符(可同时开启最多的文件数),最大开启1024,可根据需求进行调优。
2.1.2、查看系统当前可开启最大文件描述符数
ulimit -n
[root@localhost ~]# ulimit -n
65535
2.1.3、修改最大文件描述符数
# 在limits.conf文件内添加如下行“*”标识用户,*为所有用户,可改为用户名。
echo -ne
“
* soft nofile 65535
* hard nofile 65535
”
>>/etc/security/limits.conf
#/etc/security/limits.conf允许所有用户打开999,999个
#<domain> <type> <item> <value>
* - nofile 999999
注:该操作在50万并发下可使用。
2.1.4、设置登录自动读取限制参数
修改/etc/pam.d/login文件,在文件中添加如下行
session required /lib/security/pam_limits.so
注:这是告诉Linux在用户完成系统登录后,应该调用pam_limits.so模块来设置系统对该用户可使用的各种资源数量的最大限制(包括用户可打开的最大文件数限制),而pam_limits.so模块就会从/etc/security/limits.conf文件中读取配置来设置这些限制值。修改完后保存此文件。
2.1.5、参考网址:
https://www.cnblogs.com/xiangsikai/p/9496479.html
2.2、修改mosquitto能够支持的最大客户端连接数
vi /etc/mosquitto/mosquitto.conf
#添加内容如下
# The maximum number of client connections to allow. This is
# a per listener setting.
# Default is -1, which means unlimited connections.
# Note that other process limits mean that unlimited connections
# are not really possible. Typically the default maximum number of
# connections possible is around 1024.
#max_connections -1
max_connections 65535
2.3、多进程测试实例 shell脚本启动
#!/bin/bash
#python3 mqtt_sub.py 1 &
#python3 mqtt_sub.py 2 &
counter=1
loop=5000
b=0
hostip="10.10.40.242"#broker 服务端IP
while true
do
echo "counter==" $counter
b=$(( $counter % 5 ))
#sleep 1
#python3 mqtt_sub.py $counter &
topic="caoft/test/"$counter
echo $topic
#mosquitto_pub -h $hostip -t $topic -m "Hello1111 MQTT2"
mosquitto_sub -h $hostip -t $topic &
counter=$[counter+1]
if [ $counter == $loop ] ; then
break;
fi
if [ $b == 0 ] ; then
sleep 1
fi
done
2.4、结论:
①ubuntu默认安装的mosquito 1.4.8 最大支持订阅端的数量为1024左右,我实际测试位1118
mosquitto -v
1592383135: mosquitto version 1.4.8 (build date Tue, 18 Jun 2019 11:59:34 -0300) starting
1592383135: Using default config.
1592383136: Opening ipv4 listen socket on port 1883.
1592383136: Error: Address already in use
ubuntu通过apt-get install 安装Mosquittohttps://blog.csdn.net/caofengtao1314/article/details/106686329
②ubuntu默认安装的mosquito 1.6.9 最大支持订阅端的数量为10000左右,我实际测试位10000,
mosquitto -v
1592382811: mosquitto version 1.6.9 starting
1592382811: Using default config.
1592382811: Opening ipv4 listen socket on port 1883.
1592382811: Error: Address already in use
ubuntu通过手动下载编译mosquittohttps://blog.csdn.net/caofengtao1314/article/details/106686329
3、实验二:每个客户端定义唯一的topic,验证发布端发布第一个定阅端到最后一个定阅端 时间差
3.1、多进程测试实例 shell脚本启动
#!/bin/bash
#python3 mqtt_sub.py 1 &
#python3 mqtt_sub.py 2 &
counter=1
loop=5000
b=0
hostip="10.10.40.242"#broker 服务端IP
while true
do
echo "counter==" $counter
b=$(( $counter % 5 ))
#sleep 1
#python3 mqtt_sub.py $counter &
topic="caoft/test/"$counter
echo $topic
#mosquitto_pub -h $hostip -t $topic -m "Hello1111 MQTT2"
mosquitto_sub -h $hostip -t $topic &
counter=$[counter+1]
if [ $counter == $loop ] ; then
break;
fi
if [ $b == 0 ] ; then
sleep 1
fi
done
3.2、使用发布端服务器MQTTBox发布消息统计时间
发布主题caoft/test/1
发布主题 caoft/test/9999
订阅端收到的消息没有时间差
4、实验三:使用mosquito_sub 定阅1万个不同主题,使用mosquito_pub发布1万个对应的不同主题,所需要的时间消耗
4.1、订阅端服务器start_mosquito_sub.sh
#!/bin/bash
counter=1
loop=10000
b=0
hostip="10.10.40.242"
endT=0
consume=0
startT=$[$(date +%s%N)/1000000]
while true
do
echo "counter==" $counter
b=$(( $counter % 5 ))
#sleep 1
#python3 mqtt_sub.py $counter &
topic="caoft/test/"$counter
echo $topic
#mosquitto_pub -h $hostip -t $topic -m "Hello1111 MQTT2"
mosquitto_sub -h $hostip -t $topic &
counter=$[counter+1]
if [ $counter == $loop ] ; then
break;
fi
if [ $b == 0 ] ; then
sleep 1
fi
done
endT=$[$(date +%s%N)/1000000]
consume=$[endT-startT]
echo "sub start ===" $startT "end ===" $endT "consume ==" $consume "counter==" $counter
4.2、发布端服务器start_mosquito_pub.sh
#!/bin/bash
counter=1
loop=10000
b=0
hostip="10.10.40.242"
endT=0
consume=0
startT=$[$(date +%s%N)/1000000]
while true
do
echo "counter==" $counter
b=$(( $counter % 5 ))
#sleep 1
#python3 mqtt_sub.py $counter &
topic="caoft/test/"$counter
echo $topic
mosquitto_pub -h $hostip -t $topic -m $topic
counter=$[counter+1]
if [ $counter == $loop ] ; then
break;
fi
#if [ $b == 0 ] ; then
# sleep 1
# fi
done
mosquitto_pub -h $hostip -t $topic -m $topic
endT=$[$(date +%s%N)/1000000]
consume=$[endT-startT]
echo "start ===" $startT "end ===" $endT "consume ==" $consume "counter==" $counter
4.3、测试方法
4.3.1 首先启动start_mosquito_sub.sh 至少需要半个小时
4.3.2 启动start_mosquito_pub.sh的时候启动秒表计数等待订阅端全部接受完数据
4.3.3 测试结果
发布端消耗时间统计 70.142s
订阅端消耗时间统计 71.29s
5、实验四:使用mosquito_sub 定阅一个组主题,但是启动1万个定阅客户端,使用mosquito_pub发布1个组主题,所需要的时间消耗
5.1、订阅端服务器start_mosquito_sub.sh
#!/bin/bash
counter=1
loop=10000
b=0
hostip="10.10.40.242"
endT=0
consume=0
startT=$[$(date +%s%N)/1000000]
while true
do
echo "counter==" $counter
b=$(( $counter % 5 ))
#sleep 1
#python3 mqtt_sub.py $counter &
topic="caoft/test/#"
echo $topic
#mosquitto_pub -h $hostip -t $topic -m "Hello1111 MQTT2"
mosquitto_sub -h $hostip -t $topic &
counter=$[counter+1]
if [ $counter == $loop ] ; then
break;
fi
if [ $b == 0 ] ; then
sleep 1
fi
done
endT=$[$(date +%s%N)/1000000]
consume=$[endT-startT]
echo "sub start ===" $startT "end ===" $endT "consume ==" $consume "counter==" $counter
5.2、发布端服务器使用MQTTBox发布一个组主题
MQTTBox发布caoft/test/111
5.3、测试方法
5.3.1 首先启动start_mosquito_sub.sh 至少需要半个小时
5.3.2 启动MQTTBox发布caoft/test/111 的时候启动秒表计数等待订阅端全部接受完数据
5.3.3 测试结果
发布端消耗时间统计 延迟3左右秒,订阅端才有响应
订阅端消耗时间统计 7秒
6、实验五:使用mosquito_sub 定阅一个组主题,但是启动1万个定阅客户端,使用mosquito_pub发布10个组主题,所需要的时间消耗
6.1、订阅端服务器start_mosquito_sub.sh
#!/bin/bash
counter=1
loop=10000
b=0
hostip="10.10.40.242"
endT=0
consume=0
startT=$[$(date +%s%N)/1000000]
while true
do
echo "counter==" $counter
b=$(( $counter % 5 ))
#sleep 1
#python3 mqtt_sub.py $counter &
#topic="caoft/test/"$counter
topic="caoft/test/#"
echo $topic
#mosquitto_pub -h $hostip -t $topic -m "Hello1111 MQTT2"
mosquitto_sub -h $hostip -t $topic &
counter=$[counter+1]
if [ $counter == $loop ] ; then
break;
fi
if [ $b == 0 ] ; then
sleep 1
fi
done
endT=$[$(date +%s%N)/1000000]
consume=$[endT-startT]
echo "sub start ===" $startT "end ===" $endT "consume ==" $consume "counter==" $counter
6.2、发布端服务器start_mosquito_pub.sh
#!/bin/bash
try_times=0
init_counter=0
loop=10000
loop_try_times=10
b=0
hostip="10.10.40.242"
endT=0
consume=0
startT=$[$(date +%s%N)/1000000]
counter=$init_counter
while true
do
while true
do
echo "counter==" $counter
b=$(( $counter % 5 ))
#sleep 1
#python3 mqtt_sub.py $counter &
topic="caoft/test/"$counter
echo $topic
mosquitto_pub -h $hostip -t $topic -m $topic
counter=$[counter+1]
if [ $counter == $loop ] ; then
break;
fi
#if [ $b == 0 ] ; then
# sleep 1
# fi
done
try_times=$[try_times+1]
counter=$init_counter
echo "counter==" $counter "try_times==" $try_times
if [ $try_times == $loop_try_times ] ; then
break;
fi
done
mosquitto_pub -h $hostip -t $topic -m $topic
endT=$[$(date +%s%N)/1000000]
consume=$[endT-startT]
echo "start ===" $startT "end ===" $endT "consume ==" $consume "counter==" $counter "try_times==" $try_times
6.3、测试方法
6.3.1 首先启动start_mosquito_sub.sh 至少需要半个小时
6.3.2 启动start_mosquito_pub.sh发布caoft/test/111 的时候启动秒表计数等待订阅端全部接受完数据
6.3.3 测试结果
发布端消耗时间统计 延迟3左右秒,订阅端才有响应
订阅端消耗时间统计 预计消耗至少19个小时,实际测试系统已经崩溃
7、实验六:订阅端使用mosquito_sub 一个不同的主题,使用python mqtt_pub 一万个对应的订阅端主题的信息,时间测试
7.1、订阅端服务器start_mosquito_sub.sh
#!/bin/bash
counter=1
loop=10000
b=0
hostip="10.10.40.242"
endT=0
consume=0
startT=$[$(date +%s%N)/1000000]
while true
do
echo "counter==" $counter
b=$(( $counter % 5 ))
#sleep 1
#python3 mqtt_sub.py $counter &
topic="caoft/test/"$counter
#topic="caoft/test/#"
echo $topic
#mosquitto_pub -h $hostip -t $topic -m "Hello1111 MQTT2"
mosquitto_sub -h $hostip -t $topic &
counter=$[counter+1]
if [ $counter == $loop ] ; then
break;
fi
if [ $b == 0 ] ; then
sleep 1
fi
done
endT=$[$(date +%s%N)/1000000]
consume=$[endT-startT]
echo "sub start ===" $startT "end ===" $endT "consume ==" $consume "counter==" $counter
7.2、订阅端服务器start_mosquito_pub.sh
# -*- coding: utf-8 -*-
import paho.mqtt.client as mqtt
import sys
import psutil
import os
#import datetime
import time
#from datetime import *
MQTTHOST = "10.10.40.242"
MQTTPORT = 1883
mqttClient = mqtt.Client()
#def get_head_info():
# try:
# raise Exception
# except:
# f = sys.exc_info()[2].tb_frame.f_back
# return '%s, %s, %s, %s, %d' % (str(datetime.now()), f.f_code.co_filename, f.f_code.co_name, str(f.f_lineno),os.getpid())
# 连接MQTT服务器
def on_mqtt_connect():
mqttClient.connect(MQTTHOST, MQTTPORT, 60)
mqttClient.loop_start()
# publish 消息
def on_publish(topic, payload, qos):
mqttClient.publish(topic, payload, qos)
# 消息处理函数
def on_message_come(lient, userdata, msg):
print(msg.topic + " " + ":" + str(msg.payload))
# subscribe 消息
def on_subscribe():
mqttClient.subscribe("caoft/test/#", 1)
mqttClient.on_message = on_message_come # 消息到来处理函数
def main():
on_mqtt_connect()
i = 0
#print(get_head_info() + " start publish")
print("start publish")
#on_publish("caoft/test/1","caoftestt111111", 1)
while True:
topic="caoft/test/"+str(i)
on_publish(topic,topic, 1)
i += 1
if i > 10000:
break
#on_mqtt_connect()
#on_publish("caoft/test/1", "Hello Python!", 1)
#print(get_head_info() + " end publish")
print("end publish")
if __name__ == '__main__':
main()
while True:
time.sleep( 5 )
pass
7.3、测试方法
7.3.1 首先启动start_mosquito_sub.sh 至少需要半个小时
7.3.2 启动start_mosquito_pub.sh发布c的时候启动秒表计数等待订阅端全部接受完数据
7.3.3 测试结果
发布端消耗时间统计 消耗时间为1.31秒
start publish
end publish
订阅端消耗时间统计7.30秒 对比 4、实验三:使用mosquito_sub 定阅1万个不同主题,使用mosquito_pub发布1万个对应的不同主题,所需要的时间消耗 71.29s 节省64秒
8、试验七:使用python 发布一个组主题,开启1万个订阅端
8.1、发布端mqtt_pub.py
# -*- coding: utf-8 -*-
import paho.mqtt.client as mqtt
import sys
import psutil
import os
#import datetime
import time
#from datetime import *
MQTTHOST = "10.10.40.242"
MQTTPORT = 1883
mqttClient = mqtt.Client()
#def get_head_info():
# try:
# raise Exception
# except:
# f = sys.exc_info()[2].tb_frame.f_back
# return '%s, %s, %s, %s, %d' % (str(datetime.now()), f.f_code.co_filename, f.f_code.co_name, str(f.f_lineno),os.getpid())
# 连接MQTT服务器
def on_mqtt_connect():
mqttClient.connect(MQTTHOST, MQTTPORT, 60)
mqttClient.loop_start()
# publish 消息
def on_publish(topic, payload, qos):
mqttClient.publish(topic, payload, qos)
# 消息处理函数
def on_message_come(lient, userdata, msg):
print(msg.topic + " " + ":" + str(msg.payload))
# subscribe 消息
def on_subscribe():
mqttClient.subscribe("caoft/test/#", 1)
mqttClient.on_message = on_message_come # 消息到来处理函数
def main():
on_mqtt_connect()
i = 0
#print(get_head_info() + " start publish")
print("start publish")
#on_publish("caoft/test/1","caoftestt111111", 1)
while True:
topic="caoft/test/"+str(i)
on_publish(topic,topic, 1)
i += 1
if i > 1:
break
#on_mqtt_connect()
#on_publish("caoft/test/1", "Hello Python!", 1)
#print(get_head_info() + " end publish")
print("end publish")
if __name__ == '__main__':
main()
while True:
time.sleep( 5 ) #不加入延时会导致CPU占有率100%
pass
8.2、订阅端start_mosquito_sub.sh
!/bin/bash
#python3 mqtt_sub.py 1 &
#python3 mqtt_sub.py 2 &
counter=1
loop=10000
b=0
#hostip="114.115.223.53"
hostip="10.10.40.242"
endT=0
consume=0
startT=[(date +%s%N)/1000000]
while true
do
echo "counter==" counterb=(( countercounter &
#topic="caoft/test/"You can't use 'macro parameter character #' in math modetopic
#mosquitto_pub -h hostip−t topic -m "Hello1111 MQTT2"
mosquitto_sub -h hostip−t topic &
counter=[counter+1]if[counter == loop];thenbreak;fiif[b == 0 ] ; then
sleep 1
fi
done
endT=[(date +%s%N)/1000000]
consume=[endT−startT]echo"substart===" startT "end ===" endT"consume=="consume "counter==" $counter
8.3、测试方法
8.3.1 首先启动start_mosquito_sub.sh 至少需要半个小时
8.3.2 启动MQTTBox发布caoft/test/111 的时候启动秒表计数等待订阅端全部接受完数据
8.3.3 测试结果
发布端消耗时间统计 延迟1左右秒,订阅端才有响应
订阅端消耗时间统计 7秒左右
9、结论:
①ubuntu默认安装的mosquito 1.4.8 最大支持订阅端的数量为1024左右,我实际测试位1118
ubuntu通过apt-get install 安装MosquittoUbuntu安装使用mosquitto
②ubuntu默认安装的mosquito 1.6.9 最大支持订阅端的数量为10000左右,我实际测试位10000,
ubuntu通过手动下载编译mosquittoUbuntu安装使用mosquitto
③在局域网内如果每个客户端定义一个唯一的topic,总共启动1万个客户端,通过发布唯一的topic第一个和第1万个topic的时间是相同的。
④对比试验三与试验四:对每一个客户端发布一个消息耗时要比对一个组发布消息慢很多,但是订阅端收到的消息要比发布端发送的消息数量多 ,这是Qos=1导致的
⑤对比试验四与实验五:发布10万条组消息,没有导致broker死机,但是订阅端收到的消息要比发布端发送的消息数量多 ,这是Qos=1导致的
⑥实验六与实验三对比 确少了MQTT的connect阶段节省了64秒
⑦实验四,实验六 可以证明发布一个组的效率基本和发布不同的主题实践基本差不多
⑧如果要想解决LOT设备的海量连接、海量主题、海量会话的问题,需要引入集群
MQTT简介之十五--Ubuntu下Mosquitto 集群搭建
https://blog.csdn.net/caofengtao1314/article/details/106826840