MQTT-mosquitto学习笔记

CodeStarr

已于 2022-05-06 16:22:41 修改

阅读量4.7k

点赞数 1

分类专栏： IOT 文章标签： iot

于 2022-04-12 20:06:55 首次发布

本文链接：https://blog.csdn.net/Ga4ra/article/details/124132864

版权

IOT 专栏收录该内容

14 篇文章 2 订阅

订阅专栏

文章目录

1. 编译源码
- 安装依赖
- 编译安装
2. 基本使用
3. 指定QoS
4. 源码分析
- mosquitto
5. 参考资料

1. 编译源码

官网：https://mosquitto.org/

本次实验编译的版本是v3.1.1。

安装依赖

源码目录README-compileing.md。

我在编译时缺少了以下库：

# cJson
# https://github.com/DaveGamble/cJSON
unzip cJSON-1.7.15.zip
cd cJSON-1.7.15/
mkdir build
cd build
cmake ..
make
make install

# xsltproc 
# docbook-xsl
sudo apt-get install xsltproc docbook-xsl

编译安装

make && make install

或者用源码目录下的cmake, 如果要调试的话，需要在CMakeLists.txt里面加上可调式配置：

SET(CMAKE_CXX_FLAGS_DEBUG "$ENV{CXXFLAGS} -O0 -Wall -g -ggdb")
SET(CMAKE_CXX_FLAGS_RELEASE "$ENV{CXXFLAGS} -O3 -Wall")

然后编译安装就好了：

mkdir build && cd build 
cmake -DCMAKE_BUILD_TYPE=Debug ..
make
make install

经过测试，cmake的方法调试起来比较顺利一些，直接用makefile编译的话调试起来源码行数不太匹配。

主要是获取了以下可执行文件：

$ ll /usr/local/bin/mos*
-rwxr-xr-x 1 root root 198672 4月  11 14:52 /usr/local/bin/mosquitto_ctrl*	# 管理工具
-rwxr-xr-x 1 root root  88584 4月  11 14:52 /usr/local/bin/mosquitto_passwd*	# 管理账户密码
-rwxr-xr-x 1 root root 158952 4月  11 14:52 /usr/local/bin/mosquitto_pub*	# 发布客户端
-rwxr-xr-x 1 root root 192912 4月  11 14:52 /usr/local/bin/mosquitto_rr*		# 发送请求、接收响应的客户端
-rwxr-xr-x 1 root root 189368 4月  11 14:52 /usr/local/bin/mosquitto_sub*	# 订阅客户端
$ ll /usr/local/sbin/mos*
-rwxr-xr-x 1 root root 1809264 4月  11 14:52 /usr/local/sbin/mosquitto*		# broker

安装完后需要执行一下sudo ldconfig，不然搜不到/usr/loca/lib下新安装的so。

2. 基本使用

2.1 配置

$ ll /etc/mosquitto/
total 68
drwxr-xr-x   2 root root  4096 4月  11 14:52 ./
drwxr-xr-x 137 root root 12288 4月  11 14:52 ../
-rw-r--r--   1 root root   230 4月  11 14:52 aclfile.example	# 权限配置，
-rw-r--r--   1 root root 40142 4月  11 14:52 mosquitto.conf.example
-rw-r--r--   1 root root    23 4月  11 14:52 pskfile.example
-rw-r--r--   1 root root   355 4月  11 14:52 pwfile.example

执行mosquitto代理时 -c 指定mosquitto.conf即可，不指定则默认监听1883端口。

2.2 Demo

Broker:

$ mosquitto
1649669075: mosquitto version 2.0.14 starting
1649669075: Using default config.
1649669075: Starting in local only mode. Connections will only be possible from clients running on this machine.
1649669075: Create a configuration file which defines a listener to allow remote access.
1649669075: For more details see https://mosquitto.org/documentation/authentication-methods/
1649669075: Opening ipv4 listen socket on port 1883.
1649669075: Opening ipv6 listen socket on port 1883.
1649669075: mosquitto version 2.0.14 running

订阅（-t指定topic）：

mosquitto_sub -t foobar [-h localhost] -v

发布（-m指定message）：

mosquitto_pub -t foobar [-h localhost] [-p 1883] -m "hello"

然后订阅端就会收到foobar hello消息（格式 Topic Msg），再看看服务端日志：

1649669420: New connection from 127.0.0.1:43066 on port 1883.
1649669420: New client connected from 127.0.0.1:43066 as auto-11ADBEA9-7454-58EF-31EF-03D7916A9C06 (p2, c1, k60).
1649669448: New connection from 127.0.0.1:43068 on port 1883.
1649669448: New client connected from 127.0.0.1:43068 as auto-B93201BC-7CD1-F179-ECCA-4F33181AF365 (p2, c1, k60).

2.3 非匿名

可以参数指定：

-u username
-P password

也可以改配置mosquitto.conf：

# Defaults to false, unless there are no listeners defined in the configuration
# file, in which case it is set to true, but connections are only allowed from
# the local machine.
allow_anonymous false

# See the TLS client require_certificate and use_identity_as_username options
# for alternative authentication options. If a plugin is used as well as
# password_file, the plugin check will be made first.
password_file /etc/mosquitto/pwfile

# If an plugin is used as well as acl_file, the plugin check will be
# made first.
acl_file /etc/mosquitto/aclfile

新增账户：

$ sudo mosquitto_passwd -b /etc/mosquitto/pwfile foo 123
$ sudo mosquitto_passwd -b /etc/mosquitto/pwfile bar 123
$ cat /etc/mosquitto/pwfile
foo:$7$101$cEsUq8j7617mzd+6$jT757TUWXKg3MlNCAhi5wCHATk8I6pNMG7dumaA2GdExL60v9FmwtC8nMuXb0YmM1bx+myGCCN82hD2w1HF3xw==
bar:$7$101$mgWeseFJ6T7wDkpk$K8/S4/Kmr70q4uFVUfu3MgdPjDd8boARSK8uguOyLmEmuvmXq6mRPbhOmPnokNQFarum4YVmMj0z/b+SQNTk6Q==

访问权限aclfile:

# read 订阅权限 、write 发布权限、# 通配符表示所有的
user foo
topic read Topic

user bar
topic write Topic

启动broker：

$ mosquitto -c /etc/mosquitto/mosquitto.conf

使用错误的账户订阅，会失败：

$ mosquitto_sub -t Topic -v -d -u aaa
Client null sending CONNECT
Client null received CONNACK (5)
Connection error: Connection Refused: not authorised.
Client null sending DISCONNECT

必须用acl允许的账户密码（foo/123）才行：

$ mosquitto_sub -t Topic -v -d -u foo -P 123
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK

发布也是:

$ mosquitto_pub -t Topic -m "hello" -u bar
Connection error: Connection Refused: not authorised.
Error: The connection was refused.
$ mosquitto_pub -t Topic -m "hello" -u bar -P 123

订阅者成功收到消息：

Client null received PUBLISH (d0, q0, r0, m0, 'Topic', ... (5 bytes))
Topic hello

2.4 TLS安全通信

MQTT安全通信有3种方法：

网络层采用VPN专线；
传输层使用TLS加密，实现数据加密和身份认证；
应用层采用账户密码验证。

本节讲解TLS的方法，需要安装openssl：

sudo apt install openssl libssl-dev

在/etc/mosquitto/执行脚本make_tls.sh：

# * Redistributions in binary form must reproduce the above copyright
#   notice, this list of conditions and the following disclaimer in the
#   documentation and/or other materials provided with the distribution.
# * Neither the name of the axTLS project nor the names of its
#   contributors may be used to endorse or promote products derived
#   from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#
# Generate the certificates and keys for testing.
#

PROJECT_NAME="TLS Project"
  
# Generate the openssl configuration files.
cat > ca_cert.conf << EOF 
[ req ]
distinguished_name     = req_distinguished_name
prompt                 = no
[ req_distinguished_name ]
 O                      = $PROJECT_NAME Dodgy Certificate Authority
EOF

  
cat > server_cert.conf << EOF 
[ req ]
distinguished_name     = req_distinguished_name
prompt                 = no
[ req_distinguished_name ]
 O                      = $PROJECT_NAME
 CN                     = 192.168.56.101
EOF

cat > client_cert.conf << EOF 
[ req ]
distinguished_name     = req_distinguished_name
prompt                 = no
[ req_distinguished_name ]
 O                      = $PROJECT_NAME Device Certificate
 CN                     = 192.168.56.101
EOF


mkdir ca
mkdir server
mkdir client
mkdir certDER


# private key generation
openssl genrsa -out ca.key 1024
openssl genrsa -out server.key 1024
openssl genrsa -out client.key 1024


# 想从证书颁发机构certificate authority（CA）那里获得 SSL 证书，就必须生成一个证书签署请求certificate signing request（CSR）
openssl req -out ca.csr -key ca.key -new \
            -config ./ca_cert.conf
openssl req -out server.csr -key server.key -new \
            -config ./server_cert.conf
openssl req -out client.csr -key client.key -new \
            -config ./client_cert.conf
  
# generate the actual certs.
openssl x509 -req -in ca.csr -out ca.crt \
            -sha1 -days 5000 -signkey ca.key
openssl x509 -req -in server.csr -out server.crt \
            -sha1 -CAcreateserial -days 5000 \
            -CA ca.crt -CAkey ca.key
openssl x509 -req -in client.csr -out client.crt \
            -sha1 -CAcreateserial -days 5000 \
            -CA ca.crt -CAkey ca.key

# 将 PEM 编码的证书.crt转换为 DER 编码的证书.der
# DER 格式通常与 Java 一起使用。
openssl x509 -in ca.crt -outform DER -out ca.der
openssl x509 -in server.crt -outform DER -out server.der
openssl x509 -in client.crt -outform DER -out client.der


mv ca.crt ca.key ca/
mv server.crt server.key server/
mv client.crt client.key client/
mv ca.der server.der client.der certDER/

chmod 644 ca/ca.key
chmod 644 server/server.key
chmod 644 client/client.key

rm *cert.conf
rm *.csr
rm *.srl

生成的文件有证书（.crt），私钥（.key）。

测试一下：

$ openssl verify -CAfile ca/ca.crt server/server.crt
server/server.crt: OK
$ openssl verify -CAfile ca/ca.crt client/client.crt
client/client.crt: OK

单向验证

单向的意思是，仅客户端验证服务端返回的ca证书。

再次配置/etc/mosquitto/mosquitto.conf:

# Note that the recommended port for MQTT over TLS is 8883,
# but this must be set manually.
port 8883

cafile /etc/mosquitto/ca/ca.crt

# Path to the PEM encoded server certificate.
certfile /etc/mosquitto/server/server.crt

# Path to the PEM encoded keyfile.
keyfile /etc/mosquitto/server/server.key

use_identity_as_username false
require_certificate false

启动broker：

$ sudo mosquitto -c /etc/mosquitto/mosquitto.conf -v

$ mosquitto_sub -t Topic -d -u foo -P 123
Error: Connection refused
$ mosquitto_sub -t Topic -d -u foo -P 123 --cafile /etc/mosquitto/ca/ca.crt
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0

发布：

mosquitto_pub -t Topic -m "hello" -u bar -P 123 --cafile /etc/mosquitto/ca/ca.crt

双向验证

Client验证服务器端的证书，Server验证客户端证书。

改mosquitto.conf的两个字段：

use_identity_as_username true
require_certificate true

启动Broker:

$ sudo mosquitto -c /etc/mosquitto/mosquitto.conf -v

$ mosquitto_sub -t Topic -d -u foo -P 123 --cafile /etc/mosquitto/ca/ca.crt
Client null sending CONNECT
Error: Protocol error
$ mosquitto_sub -t Topic -d -u foo -P 123 --cafile /etc/mosquitto/ca/ca.crt --cert /etc/mosquitto/client/client.crt --key /etc/mosquitto/client/client.key
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0

发布：

$ mosquitto_pub -t Topic -m "hello" -u bar -P 123 --cafile /etc/mosquitto/ca/ca.crt --cert /etc/mosquitto/client/client.crt --key /etc/mosquitto/client/client.key

3. 指定QoS

接下来会通过抓包来分析mqtt，所以要把tls去掉，只保留用户信息即可，mosquitto.conf如下：

keyfile /etc/mosquitto/server/server.key
password_file /etc/mosquitto/pwfile
acl_file /etc/mosquitto/aclfile

启动Broker：

sudo mosquitto -c /etc/mosquitto/mosquitto.conf -v

这部分内容，需要结合文档第3章来分析。

QoS 0

mosquitto_pub -q参数可以指定QoS（默认0）。

$ mosquitto_sub -t Topic -d -u foo -P 123 -v -q 0
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0

在这里插入图片描述

第一个由订阅端发出的CONNECT包是最长的，注意Msg Len这22个字节是指【可变头(截图两个框中间的部分)+Payload】。Payload主要就是客户端ID，明文账户名和口令。

简单说下Connect Flags中的will flag，可以翻译成遗嘱消息，如果这个bit设为1，那么在网络连接非正常关闭的情况下，服务端就会发布遗嘱消息。

订阅的主题和QoS，则是在第3个包，SUBSCRIBE 里：

MQ Telemetry Transport Protocol, Subscribe Request
    Header Flags: 0x82, Message Type: Subscribe Request
        1000 .... = Message Type: Subscribe Request (8)
        .... 0010 = Reserved: 2
    Msg Len: 10
    Message Identifier: 1
    Topic Length: 5
    Topic: Topic
    Requested QoS: At most once delivery (Fire and Forget) (0)

发布：

$ mosquitto_pub -t foobar -m "hello" -d
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending PUBLISH (d0, q0, r0, m1, 'foobar', ... (5 bytes))
Client null sending DISCONNECT

在这里插入图片描述

增加了7个包，最后两个心跳包是碰巧抓到的，排除掉。

前两个是发布端与Broker的connect交互，和订阅时差不多；
然后是两个publish发布包，第一个是发布端给Broker的（因为三方都在一个虚拟机里，所以都是127.0.0.1），第二个是Broker发给订阅端的；
最后是发布端与Broker断开连接的包，断开的理由官网提供了很多，但我这里只有一个Fixed Header，没有Variable Header，以后碰到再说吧。

QoS 1

mosquitto_sub -t Topic -d -u foo -P 123 -v -q 1

仍然是4个数据包，只不过SUBSCRIBE包的QoS字节变成了1。

发布：

$ mosquitto_pub -t Topic -m "hello" -u bar -P 123 -d -q 1
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending PUBLISH (d0, q1, r0, m1, 'Topic', ... (5 bytes))
Client null received PUBACK (Mid: 1, RC:0)
Client null sending DISCONNECT

从调试信息可以看出，多出了一个PUBACK响应。

在这里插入图片描述

从抓到的数据包来看，也是多了两个PUBACK响应包。

从broker日志可以看出，

1649749139: New connection from 127.0.0.1:52304 on port 1883.

1649749139: New client connected from 127.0.0.1:52304 as auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (p2, c1, k60, u'bar').
1649749139: No will message specified.
1649749139: Sending CONNACK to auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (0, 0)
1649749139: Received PUBLISH from auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (d0, q1, r0, m1, 'Topic', ... (5 bytes))	# 收到发布端消息

1649749139: Sending PUBLISH to auto-60541609-17C4-B477-7348-50276571D3C1 (d0, q1, r0, m1, 'Topic', ... (5 bytes))	# 发给订阅端

1649749139: Sending PUBACK to auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (m1, rc0)
# 发给发布端PUBACK

1649749139: Received PUBACK from auto-60541609-17C4-B477-7348-50276571D3C1 (Mid: 1, RC:0)
# 收到订阅端的PUBACK

1649749139: Received DISCONNECT from auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278
1649749139: Client auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 disconnected.
# 断开与发布端的连接

QoS 2

QoS 2因为往来的数据包太多，为了方便分析，订阅端使用QoS 0。

发布：

mosquitto_pub -t Topic -m "hello" -u bar -P 123 -d -q 2

先看下broker日志：

1649749959: New connection from 127.0.0.1:53004 on port 1883.
1649749959: New client connected from 127.0.0.1:53004 as auto-FF7F8A46-058D-A421-0856-B629C0247399 (p2, c1, k60, u'foo').
1649749959: No will message specified.
1649749959: Sending CONNACK to auto-FF7F8A46-058D-A421-0856-B629C0247399 (0, 0)
1649749959: Received SUBSCRIBE from auto-FF7F8A46-058D-A421-0856-B629C0247399
1649749959:     Topic (QoS 0)
1649749959: auto-FF7F8A46-058D-A421-0856-B629C0247399 0 Topic
1649749959: Sending SUBACK to auto-FF7F8A46-058D-A421-0856-B629C0247399
# 以上是订阅端


1649749969: New connection from 127.0.0.1:53006 on port 1883.
1649749969: New client connected from 127.0.0.1:53006 as auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (p2, c1, k60, u'bar').
1649749969: No will message specified.
1649749969: Sending CONNACK to auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (0, 0)
1649749969: Received PUBLISH from auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (d0, q2, r0, m1, 'Topic', ... (5 bytes))
# 收到发布端消息

1649749969: Sending PUBREC to auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (m1, rc0)
1649749969: Received PUBREL from auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (Mid: 1)
# PUBREL 是发布端对 PUBREC 的响应

1649749969: Sending PUBLISH to auto-FF7F8A46-058D-A421-0856-B629C0247399 (d0, q0, r0, m0, 'Topic', ... (5 bytes))
# 发给订阅端

1649749969: Sending PUBCOMP to auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (m1)
# PUBCOMP 是Broker对 PUBREL 的响应

1649749969: Received DISCONNECT from auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD
1649749969: Client auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD disconnected.
# 与发布端断开连接

再看下数据包，与日志对的上：

在这里插入图片描述

这里有个细节，两个publish message的长度不一样（82和80），这两个字节是可变头（Variable header）的Packet Identifier标识符字段，在QoS 1和2两个服务等级里才存在，用来匹配类似PUBLISH和PUBACK这样的请求/响应包，具体可参考文档2.3.1.

整理一下QoS2的流程图：

4. 源码分析

mosquitto

主函数里的主循环及响应处理：

rc = mosquitto_main_loop(listensock, listensock_count);

int mosquitto_main_loop(struct mosquitto__listener_sock *listensock, int listensock_count)
{
    //...
	rc = mux__handle(listensock, listensock_count);
    //...
}

调试过程中肯定会超时，比如我用订阅端发送订阅请求，QoS为0，那么超时后订阅端会多次发送CONNECT请求。

Broker响应CONNACK的函数栈：

net__write(struct mosquitto * mosq, const void * buf, size_t count) (\home\starr\Documents\CProject\mosquitto\lib\net_mosq.c:1045)
packet__write(struct mosquitto * mosq) (\home\starr\Documents\CProject\mosquitto\lib\packet_mosq.c:252)
packet__queue(struct mosquitto * mosq, struct mosquitto__packet * packet) (\home\starr\Documents\CProject\mosquitto\lib\packet_mosq.c:172)
send__connack(struct mosquitto * context, uint8_t ack, uint8_t reason_code, const mosquitto_property * properties) (\home\starr\Documents\CProject\mosquitto\src\send_connack.c:108)
connect__on_authorised(struct mosquitto * context, void * auth_data_out, uint16_t auth_data_out_len) (\home\starr\Documents\CProject\mosquitto\src\handle_connect.c:314)
handle__connect(struct mosquitto * context) (\home\starr\Documents\CProject\mosquitto\src\handle_connect.c:929)
handle__packet(struct mosquitto * context) (\home\starr\Documents\CProject\mosquitto\src\read_handle.c:64)
packet__read(struct mosquitto * mosq) (\home\starr\Documents\CProject\mosquitto\lib\packet_mosq.c:553)
loop_handle_reads_writes(struct mosquitto * context, uint32_t events) (\home\starr\Documents\CProject\mosquitto\src\mux_epoll.c:295)
mux_epoll__handle() (\home\starr\Documents\CProject\mosquitto\src\mux_epoll.c:207)
mux__handle(struct mosquitto__listener_sock * listensock, int listensock_count) (\home\starr\Documents\CProject\mosquitto\src\mux.c:76)
mosquitto_main_loop(struct mosquitto__listener_sock * listensock, int listensock_count) (\home\starr\Documents\CProject\mosquitto\src\loop.c:205)
main(int argc, char ** argv) (\home\starr\Documents\CProject\mosquitto\src\mosquitto.c:562)
libc.so.6!__libc_start_main(int (*)(int, char **, char **) main, int argc, char ** argv, int (*)(int, char **, char **) init, void (*)(void) fini, void (*)(void) rtld_fini, void * stack_end) (\build\glibc-uZu3wS\glibc-2.27\csu\libc-start.c:310)
_start (Unknown Source:0)

5. 参考资料

CodeStarr

关注

1
点赞
踩
6

收藏

觉得还不错? 一键收藏
0
评论
MQTT-mosquitto学习笔记

@toc]1. 编译源码官网：https://mosquitto.org/本次实验编译的版本是v3.1.1。安装依赖源码目录README-compileing.md。我在编译时缺少了以下库：# cJson# https://github.com/DaveGamble/cJSONunzip cJSON-1.7.15.zipcd cJSON-1.7.15/mkdir buildcd buildcmake ..makemake install# xsltproc # docb
复制链接

扫一扫