分布式FastDFS存储1

已于 2023-06-10 11:36:05 修改
阅读量226 收藏
点赞数
文章标签： 分布式 数据库
于 2023-06-04 01:07:03 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_37070988/article/details/130938357
版权

分布式FastDFS集群部署参见：FastDFS分布式集群部署

部署情况如下：在A机器上部署了2个tracker，通过使用2个不同的端口，创建2个tracker服务；在A机器上部署了1个storage，storage不能通过使用不同的端口实现不同的storage服务，因此还要在B机器上部署1个storage。通过在A和B机器的storage的.conf文件添加2个tracker服务的IP和Port信息，实现每个storage可以能跟每个tracker连接。如果需要增加tracker，则需要在每个storage配置文件中添加新增的tracker的IP和Port信息。

调试storage服务

先查看storage服务运行情况：

root@ubuntu:/home/user/Downloads# ps -ef | grep tracker
root      79273   1418  0 03:41 ?        00:00:22 /usr/bin/fdfs_trackerd /etc/fdfs/tracker_22122.conf
root      79293   1418  0 03:42 ?        00:00:24 /usr/bin/fdfs_trackerd /etc/fdfs/tracker_22123.conf
root      80806  74826  0 09:16 pts/0    00:00:00 grep --color=auto tracker

然后停止storage服务：

root@ubuntu:/home/user/Downloads# fdfs_storaged /etc/fdfs/storage_group1_23000.conf stop
waiting for pid [80375] exit ...
pid [80375] exit.

调试storage服务：

root@ubuntu:/home/user/Downloads# gdb fdfs_storaged 
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from fdfs_storaged...done.
(gdb) 
(gdb) b main
Breakpoint 1 at 0x5560: file fdfs_storaged.c, line 87.
(gdb) set args /etc/fdfs/storage_group1_23000.conf 
(gdb) r
Starting program: /usr/bin/fdfs_storaged /etc/fdfs/storage_group1_23000.conf 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, main (argc=2, argv=0x7fffffffe4f8) at fdfs_storaged.c:87
87	{
(gdb) b ::fork
Breakpoint 2 at 0x7ffff7654700: file ../sysdeps/nptl/fork.c, line 49.
(gdb) c
Continuing.

Breakpoint 2, __libc_fork () at ../sysdeps/nptl/fork.c:49
49	../sysdeps/nptl/fork.c: No such file or directory.
(gdb) bt
#0  __libc_fork () at ../sysdeps/nptl/fork.c:49
#1  0x00007ffff79706cd in daemon_init (bCloseFiles=<optimized out>) at shared_func.c:454
#2  0x000055555555975e in main (argc=2, argv=0x7fffffffe4f8) at fdfs_storaged.c:162
(gdb)

当然，我们还可以通过另一种方式调试storaged服务:gdb attach pid

文件上传

文件上传协议

服务之间的通信协议，遵循 header + body的设计原则。具体可参考《FastDFS通信协议详解》

header结构体封装如下：

#define FDFS_PROTO_PKG_LEN_SIZE        8
typedef struct
{
	char pkg_len[FDFS_PROTO_PKG_LEN_SIZE];  //body length, not including header
	char cmd;    //command code
	char status; //status code for response
} TrackerHeader;

字段解释：
FastDFS协议头部由10个字节组成：
● pck_len：一个int64_t的整型，除去TrackerHeader长度的报文长度。FDFS_PROTO_PKG_LEN_SIZE=8
● cmd：命令
● status：返回时的状态码，发送时设置为0

注意，各个端之间的信令交互，header的格式是固定的，data是不固定的（根据cmd实际情况为准）。

文件上传流程

1.上传连接请求

在fdfs_upload_file.c的main()函数中，调用tracker_connect_server()查询tracker集群中可用的tracker server，然后从连接池中取出一个连接和客户端建立连接。

2.查询可用的storage

fdfs_upload_file进程在fdfs_upload_file.c的main()函数中调用tracker_query_storage_store()函数（实际是tracker_query_storage_store_without_group()的宏定义）。然后调用tcpsenddata_nb()发送请求，通过fdfs_recv_response()接收tracker server的响应。tracker_query_storage_store_without_group()最终的作用就是获取可用的storage的IP和Port。

我们现在同时打开两个窗口，调试fdfs_upload_file客户端和tracker server服务端。先进如fdfs_upload_file调试：

root@ubuntu:/home/user/Downloads/fastdfs# gdb fdfs_upload_file 
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from fdfs_upload_file...done.
(gdb) b main
Breakpoint 1 at 0x233a: file fdfs_upload_file.c, line 26.
(gdb) b tracker_client.c:924    #查询storage的IP和Port
Breakpoint 2 at 0x55555555decd: file ../client/tracker_client.c, line 924.
(gdb) set args /etc/fdfs/client.conf  /home/user/Downloads/test.txt #设置main的参数
(gdb) r
Starting program: /usr/bin/fdfs_upload_file /etc/fdfs/client.conf  /home/user/Downloads/test.txt 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, main (argc=3, argv=0x7fffffffe4c8) at fdfs_upload_file.c:26
26	{
(gdb) c
Continuing.

Breakpoint 2, tracker_query_storage_store_without_group (pTrackerServer=0x55555577c7c8, pStorageServer=0x7fffffffe2c0, 
    group_name=0x7fffffffe300 "group1", store_path_index=0x7fffffffe2bc) at ../client/tracker_client.c:925
925		*(group_name + FDFS_GROUP_NAME_MAX_LEN) = '\0';
(gdb) 
(gdb) p in_buff 
$1 = "group1\000\000\000\000\000\000\000\000\000\000\061\071\062.168.221.152\000\000\000\000\000\000Y\330\000\353\302UUUU\000\000\000"
(gdb) p group_name 
$2 = 0x7fffffffe300 "group1"
(gdb) n
926		memcpy(pStorageServer->ip_addr, in_buff + \
(gdb) n
928		pStorageServer->port = (int)buff2long(in_buff + \
(gdb) n
930		*store_path_index = *(in_buff + FDFS_GROUP_NAME_MAX_LEN + \
(gdb) p pStorageServer->ip_addr
$4 = "192.168.221.152", '\000' <repeats 30 times>
(gdb) pStorageServer->port
Undefined command: "pStorageServer->port".  Try "help".
(gdb) p pStorageServer->port
$5 = 23000
(gdb) 
(gdb) p *store_path_index 
$7 = 32767
(gdb) 
(gdb) q
A debugging session is active.

	Inferior 1 [process 98990] will be killed.

Quit anyway? (y or n) y

调试查询可用的storage:

        tracker端的处理函数是：tracker_service.c: tracker_deal_task()。tracker_deal_task()函数内部又调用tracker_deal_service_query_storage().

下面是对tracker进行调试：

root@ubuntu:/home/user/Downloads/fastdfs# ps aux | grep tracker
root      99149  0.0  0.1  90548  5964 ?        Sl   08:35   0:00 /usr/bin/fdfs_trackerd /etc/fdfs/tracker_22122.conf
root      99176  0.0  0.0  14432  1068 pts/3    S+   08:36   0:00 grep --color=auto tracker
root@ubuntu:/home/user/Downloads/fastdfs# gdb attach 99149
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
attach: No such file or directory.
Attaching to process 99149
[New LWP 99150]
[New LWP 99151]
[New LWP 99152]
[New LWP 99153]
[New LWP 99154]
[New LWP 99155]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f6add1987c7 in __libc_accept (fd=fd@entry=5, addr=addr@entry=..., len=len@entry=0x7ffc103ba19c)
    at ../sysdeps/unix/sysv/linux/accept.c:26
26	../sysdeps/unix/sysv/linux/accept.c: No such file or directory.
(gdb) bt
#0  0x00007f6add1987c7 in __libc_accept (fd=fd@entry=5, addr=addr@entry=..., len=len@entry=0x7ffc103ba19c)
    at ../sysdeps/unix/sysv/linux/accept.c:26
#1  0x000055c77ad8b332 in accept_thread_entrance (arg=arg@entry=0x5) at tracker_service.c:243
#2  0x000055c77ad8d4b1 in tracker_accept_loop (server_sock=5) at tracker_service.c:315
#3  0x000055c77ad8083e in main (argc=<optimized out>, argv=<optimized out>) at fdfs_trackerd.c:377
(gdb) b tracker_deal_task 
Breakpoint 1 at 0x55c77ad8d4d5: file tracker_service.c, line 3908.
(gdb) c
Continuing.
[Switching to Thread 0x7f6add508700 (LWP 99152)]

Thread 4 "fdfs_trackerd" hit Breakpoint 1, tracker_deal_task (pTask=pTask@entry=0x7f6add58b010) at tracker_service.c:3908
3908	{
(gdb) c
Continuing.

Thread 4 "fdfs_trackerd" hit Breakpoint 1, tracker_deal_task (pTask=pTask@entry=0x7f6add58b010) at tracker_service.c:3908
3908	{
(gdb) c
Continuing.

Thread 4 "fdfs_trackerd" hit Breakpoint 1, tracker_deal_task (pTask=pTask@entry=0x7f6add58b010) at tracker_service.c:3908
3908	{
(gdb) n
3912		pHeader = (TrackerHeader *)pTask->data;
(gdb) n
3913		switch(pHeader->cmd)
(gdb) p pHeader->cmd
$1 = 84 'T'
(gdb) c
Continuing.
[Switching to Thread 0x7f6add4c7700 (LWP 99153)]

Thread 5 "fdfs_trackerd" hit Breakpoint 1, tracker_deal_task (pTask=pTask@entry=0x7f6add58b010) at tracker_service.c:3908
3908	{
(gdb) n
3912		pHeader = (TrackerHeader *)pTask->data;
(gdb) n
3913		switch(pHeader->cmd)
(gdb) p pHeader->cmd
$3 = 81 'Q'
(gdb) 
# client_sock_read()里面的while(1)里一直调用tracker_deal_task()
(gdb) disable 1    #关闭断点，让程序能接着向里面执行
(gdb) b tracker_deal_service_query_storage    #对查询可用的storage函数打断点
Breakpoint 2 at 0x55c77ad8c168: file tracker_service.c, line 2661.
(gdb) c
Continuing.    #tracker此时卡在这个位置，我们在另一个窗口上传文件：（root@ubuntu:/home/user/Downloads/fastdfs# /usr/bin/fdfs_upload_file   /etc/fdfs/client.conf /home/user/Downloads/test.txt，然后该窗口被卡住执行，等待tracker的响应 ）
[Switching to Thread 0x7f6add508700 (LWP 99152)]

Thread 4 "fdfs_trackerd" hit Breakpoint 2, tracker_deal_service_query_storage (pTask=0x7f6add58b220, cmd=cmd@entry=101 'e')
    at tracker_service.c:2661
2661	{
(gdb) 
(gdb) p cmd
$11 = 101 'e'
(gdb) p g_groups.store_lookup
$12 = 2 '\002'
(gdb) p pStoreGroup->storage_port
$2 = 23000
(gdb) p pTask->client_ip
$3 = "192.168.221.152"
(gdb) b tracker_service.c:4076
Breakpoint 1 at 0x55c77ad8de1a: file tracker_service.c, line 4076.
(gdb) c
Continuing.

Thread 3 "fdfs_trackerd" hit Breakpoint 1, tracker_deal_task (pTask=pTask@entry=0x7f6add58e500) at tracker_service.c:4076
4076		pHeader = (TrackerHeader *)pTask->data;
(gdb) p pTask->data
$1 = 0x55c77b7f5790 ""
(gdb) p *pTask->data
$2 = 0 '\000'
(gdb) n
[Switching to Thread 0x7f6add508700 (LWP 99152)]

Thread 4 "fdfs_trackerd" hit Breakpoint 1, tracker_deal_task (pTask=pTask@entry=0x7f6add58e5b0) at tracker_service.c:4076
4076		pHeader = (TrackerHeader *)pTask->data;
(gdb) n
4077		pHeader->status = result;
(gdb) p *pHeader 
$3 = {pkg_len = "\000\000\000\000\000\000\000", cmd = 101 'e', status = 0 '\000'}
(gdb) n
[Switching to Thread 0x7f6add549700 (LWP 99151)]

Thread 3 "fdfs_trackerd" hit Breakpoint 1, tracker_deal_task (pTask=pTask@entry=0x7f6add58e500) at tracker_service.c:4076
4076		pHeader = (TrackerHeader *)pTask->data;
(gdb) n
4077		pHeader->status = result;
(gdb) n
4078		pHeader->cmd = TRACKER_PROTO_CMD_RESP;
(gdb) n
4079		long2buff(pTask->length - sizeof(TrackerHeader), pHeader->pkg_len);
(gdb) n
4081		send_add_event(pTask);
(gdb) p *pTask 
$4 = {event = {timer = {expires = 1685846827, prev = 0x55c77bf66ec0, next = 0x0, slot_index = 0, rehash = true}, fd = 21, 
    callback = 0x55c77ad949bf <client_sock_read>}, {server_ip = "192.168.221.152", client_ip = "192.168.221.152"}, 
  arg = 0x7f6add58e598, data = 0x55c77b7f5790 "", size = 8192, length = 10, offset = 10, port = 0, nio_stages = {
    current = 0 '\000', notify = 0 '\000'}, continue_callback = 0x0, reffer_count = 0 '\000', canceled = 0 '\000', 
  connect_timeout = 0, network_timeout = 0, req_count = 2945, finish_callback = 0x0, thread_data = 0x55c77bf5f370, 
  ctx = 0x0, next = 0x7f6add58e450}
(gdb)

下图是tracker server处理客户端请求的过程图：

 下图是tracker server处理请求完毕后，返回客户端响应的函数说明：

int tracker_deal_task(struct fast_task_info *pTask)
{
    .....
        
	pHeader = (TrackerHeader *)pTask->data;
	pHeader->status = result;
	pHeader->cmd = TRACKER_PROTO_CMD_RESP; // 不管是什么命令的请求，都是以TRACKER_PROTO_CMD_RESP命令返回
	long2buff(pTask->length - sizeof(TrackerHeader), pHeader->pkg_len);

	send_add_event(pTask);    #回复客户端

	return 0;
}
全局变量 FDFSGroups g_groups; 记录group以及对应storage的信息

3. 返回Storage信息

Tracker选择完可用的Storage服务器后，向客户端返回信息。返回的报文data内容（十六进制）如下（为了便于观察，做了内容换行）：

00000000000000286400				//10个字节67726f75703200000000000000000000 
0x67, 0x72, 0x6f, 0x75,  0x70, 0x31,  0x0,  0x0,   0x0,  0x0,   0x0,   0x0,    0x0,  0x0,  0x0, 0x0, 	//16个字节
0x31, 0x32, 0x30, 0x2e, 0x32, 0x37,  0x2e, 0x31, 0x33, 0x31, 0x2e, 0x31, 0x39, 0x37, 0x0, 0x0,  //16个字节
000000000059d8						//7个字节， 0x59d8 -> 即是23000
00									//1个字节

解释：data共50个字节

● 第一行10个字节为TrackerHeader，其中前8个字节为报文体长度，0x28（十六进制）=40（十进制），对应的接下来的报文体40个字节；0x64（十六进制）=100（十进制），100命令对应源码中的定义：
#define TRACKER_PROTO_CMD_RESP					100
● 第二行16个字节为Storage的组名（group name），翻译为ASCII为group1，正好是我Storage中的一个组名。
● 第三行16个字节为Storage的ip，		翻译为ASCII为120.27.131.197
● 第四行7个字节为Storage的port，		翻译为ASCII为23000
● 第五行1个字节为storage_index               翻译为ASCII为 0

4. 上传文件

fdfs_upload_file.c:100行调用storage_upload_by_filename1函数，最终调用storage_do_upload_file。FastDFS不保存文件名。

下面是storage_do_upload_file的调用过程：

 开始调试：

root@ubuntu:~# gdb fdfs_upload_file
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from fdfs_upload_file...done.
(gdb) b storage_do_upload_file
Breakpoint 1 at 0xd76c: file ../client/storage_client.c, line 816.
(gdb) b storage_client.c:853
Breakpoint 2 at 0xd84d: file ../client/storage_client.c, line 853.
(gdb) b storage_client.c:940
Breakpoint 3 at 0xdd6f: file ../client/storage_client.c, line 940.
(gdb) b storage_client.c:985
Breakpoint 4 at 0xddd8: file ../client/storage_client.c, line 985.
(gdb) c
The program is not being run.
(gdb) set args /etc/fdfs/client.conf  /home/user/Downloads/test.txt
(gdb) r
Starting program: /usr/bin/fdfs_upload_file /etc/fdfs/client.conf  /home/user/Downloads/test.txt
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, storage_do_upload_file (pTrackerServer=0x55555577c7c8, pStorageServer=0x7fffffffe2e0, store_path_index=0, 
    cmd=11 '\v', upload_type=2, file_buff=0x7fffffffe76d "/home/user/Downloads/test.txt", arg=0x0, file_size=24, 
    master_filename=0x0, prefix_name=0x0, file_ext_name=0x7fffffffe787 "txt", meta_list=0x0, meta_count=0, 
    group_name=0x7fffffffe1c0 "group1", remote_filename=0x7fffffffe1e0 "\310\307wUUU") at ../client/storage_client.c:816
816	{
(gdb) c
Continuing.

Breakpoint 3, storage_do_upload_file (pTrackerServer=0x55555577c7c8, pStorageServer=<optimized out>, 
    store_path_index=<optimized out>, cmd=<optimized out>, upload_type=2, 
    file_buff=0x7fffffffe76d "/home/user/Downloads/test.txt", arg=0x0, file_size=24, master_filename=<optimized out>, 
    prefix_name=<optimized out>, file_ext_name=<optimized out>, meta_list=0x0, meta_count=0, group_name=0x7fffffffe1c0 "", 
    remote_filename=0x7fffffffe1e0 "") at ../client/storage_client.c:940
940		if ((result=tcpsenddata_nb(pStorageServer->sock, out_buff, \
(gdb) p bUploadSlave
$1 = <optimized out>
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /usr/bin/fdfs_upload_file /etc/fdfs/client.conf  /home/user/Downloads/test.txt
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, storage_do_upload_file (pTrackerServer=0x55555577c7c8, pStorageServer=0x7fffffffe2e0, store_path_index=0, 
    cmd=11 '\v', upload_type=2, file_buff=0x7fffffffe76d "/home/user/Downloads/test.txt", arg=0x0, file_size=24, 
    master_filename=0x0, prefix_name=0x0, file_ext_name=0x7fffffffe787 "txt", meta_list=0x0, meta_count=0, 
    group_name=0x7fffffffe1c0 "group1", remote_filename=0x7fffffffe1e0 "\310\307wUUU") at ../client/storage_client.c:816
816	{
(gdb) b storage_client.c:852
Breakpoint 5 at 0x555555561840: file ../client/storage_client.c, line 852.
(gdb) c
Continuing.

Breakpoint 3, storage_do_upload_file (pTrackerServer=0x55555577c7c8, pStorageServer=<optimized out>, 
    store_path_index=<optimized out>, cmd=<optimized out>, upload_type=2, 
    file_buff=0x7fffffffe76d "/home/user/Downloads/test.txt", arg=0x0, file_size=24, master_filename=<optimized out>, 
    prefix_name=<optimized out>, file_ext_name=<optimized out>, meta_list=0x0, meta_count=0, group_name=0x7fffffffe1c0 "", 
    remote_filename=0x7fffffffe1e0 "") at ../client/storage_client.c:940
940		if ((result=tcpsenddata_nb(pStorageServer->sock, out_buff, \
(gdb) p bUploadSlave
$2 = <optimized out>
(gdb) b storage_client.c:833
Breakpoint 6 at 0x5555555617f2: file ../client/storage_client.c, line 833.
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /usr/bin/fdfs_upload_file /etc/fdfs/client.conf  /home/user/Downloads/test.txt
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, storage_do_upload_file (pTrackerServer=0x55555577c7c8, pStorageServer=0x7fffffffe2e0, store_path_index=0, 
    cmd=11 '\v', upload_type=2, file_buff=0x7fffffffe76d "/home/user/Downloads/test.txt", arg=0x0, file_size=24, 
    master_filename=0x0, prefix_name=0x0, file_ext_name=0x7fffffffe787 "txt", meta_list=0x0, meta_count=0, 
    group_name=0x7fffffffe1c0 "group1", remote_filename=0x7fffffffe1e0 "\310\307wUUU") at ../client/storage_client.c:816
816	{
(gdb) c
Continuing.

Breakpoint 6, storage_do_upload_file (pTrackerServer=0x55555577c7c8, pStorageServer=0x7fffffffe2e0, store_path_index=0, 
    cmd=11 '\v', upload_type=2, file_buff=0x7fffffffe76d "/home/user/Downloads/test.txt", arg=0x0, file_size=24, 
    master_filename=0x0, prefix_name=0x0, file_ext_name=0x7fffffffe787 "txt", meta_list=0x0, meta_count=0, 
    group_name=0x7fffffffe1c0 "group1", remote_filename=0x7fffffffe1e0 "") at ../client/storage_client.c:833
833		new_store_path = store_path_index;
(gdb) n
834		if (master_filename != NULL)
(gdb) n
840			master_filename_len = 0;
(gdb) n
843		if (prefix_name != NULL)
(gdb) n
862		else if ((result=storage_get_upload_connection(pTrackerServer, \
(gdb) p bUploadSlave
$3 = false
(gdb) c
Continuing.

Breakpoint 3, storage_do_upload_file (pTrackerServer=0x55555577c7c8, pStorageServer=<optimized out>, 
    store_path_index=<optimized out>, cmd=<optimized out>, upload_type=2, 
    file_buff=0x7fffffffe76d "/home/user/Downloads/test.txt", arg=0x0, file_size=24, master_filename=<optimized out>, 
    prefix_name=<optimized out>, file_ext_name=<optimized out>, meta_list=0x0, meta_count=0, group_name=0x7fffffffe1c0 "", 
    remote_filename=0x7fffffffe1e0 "") at ../client/storage_client.c:940
940		if ((result=tcpsenddata_nb(pStorageServer->sock, out_buff, \
(gdb) p pHeader 
$4 = (TrackerHeader *) 0x7fffffffddf0
(gdb) p* pHeader 
$5 = {pkg_len = "\000\000\000\000\000\000\000'", cmd = 11 '\v', status = 0 '\000'}
(gdb) n
951		if (upload_type == FDFS_UPLOAD_BY_FILE)
(gdb) p upload_type
$6 = 2
(gdb) n
953			if ((result=tcpsendfile(pStorageServer->sock, file_buff, \
(gdb) n
984		pInBuff = in_buff;
(gdb) n

Breakpoint 4, storage_do_upload_file (pTrackerServer=0x55555577c7c8, pStorageServer=<optimized out>, 
    store_path_index=<optimized out>, cmd=<optimized out>, upload_type=2, 
    file_buff=0x7fffffffe76d "/home/user/Downloads/test.txt", arg=0x0, file_size=24, master_filename=<optimized out>, 
    prefix_name=<optimized out>, file_ext_name=<optimized out>, meta_list=0x0, meta_count=0, group_name=0x7fffffffe1c0 "", 
    remote_filename=0x7fffffffe1e0 "") at ../client/storage_client.c:985
985		if ((result=fdfs_recv_response(pStorageServer, \
(gdb) p *pInBuff
$7 = 0 '\000'
(gdb) n
994		if (in_bytes <= FDFS_GROUP_NAME_MAX_LEN)
(gdb) p *pInBuff
$8 = 103 'g'
(gdb) n
1006		in_buff[in_bytes] = '\0';
(gdb) n
1007		memcpy(group_name, in_buff, FDFS_GROUP_NAME_MAX_LEN);
(gdb) n
1008		group_name[FDFS_GROUP_NAME_MAX_LEN] = '\0';
(gdb) n
1011			in_bytes - FDFS_GROUP_NAME_MAX_LEN + 1);
(gdb) n
1010		memcpy(remote_filename, in_buff + FDFS_GROUP_NAME_MAX_LEN, \
(gdb) n
1015		if (result == 0 && meta_count > 0)
(gdb) n
1030		if (new_connection)
(gdb) p remote_filename
$9 = 0x7fffffffe1e0 "M00/00/00/wKjdmGR8d4iAOze_AAAAGFA603w825.txt"
(gdb) p *file_buff
$10 = 47 '/'
(gdb) p group_name
$11 = 0x7fffffffe1c0 "group1"
(gdb) 
(gdb) p new_connection
$12 = true
(gdb) p /x *out_buff@25
$14 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x27, 0xb, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x18, 0x74, 0x78, 0x74, 
  0x0, 0x0, 0x0}
(gdb)

发送的时候header + data，实际是发送了25字节

(gdb) p /x *out_buff@25

$14 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x27, 0xb, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x18, 0x74, 0x78, 0x74, 0x0, 0x0, 0x0}

0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x27, 0xb, 0x0,  10字节 header
0x0, 										        storage_index 1字节
0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x18,    		文件大小，0x18即是24个字节
0x74, 0x78, 0x74, 0x0, 0x0, 0x0                     扩展名6字节，这里是txt

5.选择存储路径、生成文件id并存储文件

        storage响应函数storage_service.c: storage_deal_task，然后调用

int storage_upload_file(struct fast_task_info *pTask, bool bAppenderFile)。

storage处理流程：

  gdb对storage调试：

root@ubuntu:~# gdb attach 117909
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
attach: No such file or directory.
Attaching to process 117909
[New LWP 117925]
[New LWP 117926]
[New LWP 117927]
[New LWP 117928]
[New LWP 117929]
[New LWP 117930]
[New LWP 117931]
[New LWP 117932]
[New LWP 117933]
[New LWP 117934]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f01418f57c7 in __libc_accept (fd=fd@entry=5, addr=addr@entry=..., len=len@entry=0x7ffc78000770)
    at ../sysdeps/unix/sysv/linux/accept.c:26
26	../sysdeps/unix/sysv/linux/accept.c: No such file or directory.
(gdb) bt
#0  0x00007f01418f57c7 in __libc_accept (fd=fd@entry=5, addr=addr@entry=..., len=len@entry=0x7ffc78000770)
    at ../sysdeps/unix/sysv/linux/accept.c:26
#1  0x000055cb29adcd18 in accept_thread_entrance (arg=arg@entry=0x5) at storage_service.c:1796
#2  0x000055cb29ae66df in storage_accept_loop (server_sock=5) at storage_service.c:1914
#3  0x000055cb29ad013a in main (argc=<optimized out>, argv=0x7ffc78000ca8) at fdfs_storaged.c:287
(gdb) b storage_deal_task
Breakpoint 1 at 0x55cb29ae9c12: file storage_service.c, line 8319.
(gdb) b storage_service.c:4581
Breakpoint 2 at 0x55cb29ae6c5b: file storage_service.c, line 4581.
(gdb) b storage_write_to_file
Breakpoint 3 at 0x55cb29ade35e: file storage_service.c, line 7422.
(gdb) b b storage_service.c:7440
No source file named b storage_service.c.
Make breakpoint pending on future shared library load? (y or [n]) n
(gdb) b storage_service.c:7440
Breakpoint 4 at 0x55cb29ade3bd: file storage_service.c, line 7440.
(gdb) c
Continuing.
[Switching to Thread 0x7f013ccb3700 (LWP 117928)]

Thread 5 "fdfs_storaged" hit Breakpoint 1, storage_deal_task (pTask=pTask@entry=0x7f013ccb4010) at storage_service.c:8319
8319	{
(gdb) c
Continuing.

Thread 5 "fdfs_storaged" hit Breakpoint 2, storage_upload_file (pTask=pTask@entry=0x7f013ccb4010, 
    bAppenderFile=bAppenderFile@entry=false) at storage_service.c:4581
4581		store_path_index = *p++;
(gdb) p store_path_index
$1 = 1019948048
(gdb) n
4583		if (store_path_index == -1)
(gdb) n
4595		else if (store_path_index < 0 || store_path_index >= \
(gdb) n
4581		store_path_index = *p++;
(gdb) n
4605		file_bytes = buff2long(p);
(gdb) n
4607		if (file_bytes < 0 || file_bytes != nInPackLen - \
(gdb) n
4619		memcpy(file_ext_name, p, FDFS_FILE_EXT_NAME_MAX_LEN);
(gdb) n
4620		*(file_ext_name + FDFS_FILE_EXT_NAME_MAX_LEN) = '\0';
(gdb) p *file_ext_name
$2 = 116 't'
(gdb) n
4622		if ((result=fdfs_validate_filename(file_ext_name)) != 0)
(gdb) n
4631		pFileContext->calc_crc32 = true;
(gdb) n
4632		pFileContext->calc_file_hash = g_check_file_duplicate;
(gdb) n
4633		pFileContext->extra_info.upload.start_time = g_current_time;
(gdb) n
4635		strcpy(pFileContext->extra_info.upload.file_ext_name, file_ext_name);
(gdb) n
4637				pFileContext->extra_info.upload.formatted_ext_name);
(gdb) p *pFileContext->extra_info.upload.file_ext_name
$3 = 116 't'
(gdb) n
4636		storage_format_ext_name(file_ext_name, \
(gdb) n
4639					store_path_index = store_path_index;
(gdb) n
4640		pFileContext->extra_info.upload.file_type = _FILE_TYPE_REGULAR;
(gdb) n
4641		pFileContext->sync_flag = STORAGE_OP_TYPE_SOURCE_CREATE_FILE;
(gdb) n
4642		pFileContext->timestamp2log = pFileContext->extra_info.upload.start_time;
(gdb) n
4643		pFileContext->op = FDFS_STORAGE_FILE_OP_WRITE;
(gdb) n
4644		if (bAppenderFile)
(gdb) n
4651			if (g_if_use_trunk_file && trunk_check_size( \
(gdb) n
4686			if (!storage_check_reserved_space_path(g_fdfs_store_paths.paths \
(gdb) n
4706			crc32 = rand();
(gdb) n
4707			*filename = '\0';
(gdb) n
4708			filename_len = 0;
(gdb) n
4709			pFileContext->extra_info.upload.if_sub_path_alloced = false;
(gdb) n
4710			if ((result=storage_get_filename(pClientInfo, \
(gdb) n
4714				pFileContext->filename)) != 0)
(gdb) n
4710			if ((result=storage_get_filename(pClientInfo, \
(gdb) n
4721	    pFileContext->extra_info.upload.if_gen_filename = true;
(gdb) n
4722			pFileContext->extra_info.upload.before_open_callback = NULL;
(gdb) n
4723			pFileContext->extra_info.upload.before_close_callback = NULL;
(gdb) n
4725							| g_extra_open_file_flags;
(gdb) n
4724			pFileContext->open_flags = O_WRONLY | O_CREAT | O_TRUNC \
(gdb) n
4720			file_offset = 0;
(gdb) n
4719			clean_func = dio_write_finish_clean_up;
(gdb) n
4728	    pFileContext->continue_callback = storage_nio_notify;
(gdb) n
4621		p += FDFS_FILE_EXT_NAME_MAX_LEN;
(gdb) c
Continuing.

Thread 5 "fdfs_storaged" hit Breakpoint 3, storage_write_to_file (pTask=pTask@entry=0x7f013ccb4010, 
    file_offset=file_offset@entry=0, upload_bytes=upload_bytes@entry=24, buff_offset=25, 
    deal_func=0x55cb29af1a77 <dio_write_file>, 
    done_callback=done_callback@entry=0x55cb29ae8dd5 <storage_upload_file_done_callback>, 
    clean_func=0x55cb29af1ef4 <dio_write_finish_clean_up>, store_path_index=0) at storage_service.c:7422
7422	{
(gdb) n
7427		pClientInfo = (StorageClientInfo *)pTask->arg;
(gdb) b storage_service.c:7447
Breakpoint 5 at 0x55cb29ade3d9: file storage_service.c, line 7447.
(gdb) c
Continuing.

Thread 5 "fdfs_storaged" hit Breakpoint 4, storage_write_to_file (pTask=pTask@entry=0x7f013ccb4010, 
    file_offset=file_offset@entry=0, upload_bytes=upload_bytes@entry=24, buff_offset=<optimized out>, 
    deal_func=<optimized out>, done_callback=done_callback@entry=0x55cb29ae8dd5 <storage_upload_file_done_callback>, 
    clean_func=0x55cb29af1ef4 <dio_write_finish_clean_up>, store_path_index=0) at storage_service.c:7440
7440		pFileContext->done_callback = done_callback;
(gdb) n
7442		if (pFileContext->calc_crc32)
(gdb) n
7444			pFileContext->crc32 = CRC32_XINIT;
(gdb) n

Thread 5 "fdfs_storaged" hit Breakpoint 5, storage_write_to_file (pTask=pTask@entry=0x7f013ccb4010, 
    file_offset=file_offset@entry=0, upload_bytes=upload_bytes@entry=24, buff_offset=<optimized out>, 
    deal_func=<optimized out>, done_callback=done_callback@entry=0x55cb29ae8dd5 <storage_upload_file_done_callback>, 
    clean_func=0x55cb29af1ef4 <dio_write_finish_clean_up>, store_path_index=0) at storage_service.c:7447
7447		if (pFileContext->calc_file_hash)
(gdb) p *pFileContext 
$4 = {
  filename = "/home/fastdfs/storage_group1_23000/data/00/00/wKjdmGR8kBmAMxhWAAAAGFoujyI220.txt", '\000' <repeats 303 times>, fname2log = '\000' <repeats 130 times>, op = 87 'W', sync_flag = 67 'C', calc_crc32 = true, calc_file_hash = false, 
  open_flags = 577, file_hash_codes = {0, 0, 0, 0}, crc32 = 4294967295, md5_context = {state = {0, 0, 0, 0}, count = {0, 
      0}, buffer = '\000' <repeats 63 times>}, extra_info = {upload = {if_gen_filename = true, file_type = 8 '\b', 
      if_sub_path_alloced = true, master_filename = '\000' <repeats 127 times>, file_ext_name = "txt\000\000\000", 
      formatted_ext_name = "220.txt", prefix_name = '\000' <repeats 16 times>, group_name = '\000' <repeats 16 times>, 
      start_time = 1685884953, trunk_info = {status = 0 '\000', path = {store_path_index = 0 '\000', 
          sub_path_high = 0 '\000', sub_path_low = 0 '\000'}, file = {id = 0, offset = 0, size = 0}}, 
      before_open_callback = 0x0, before_close_callback = 0x0}, setmeta = {op_flag = 1 '\001', meta_buff = 0x0, 
      meta_bytes = 0}}, dio_thread_index = 1, timestamp2log = 1685884953, delete_flag = 0, create_flag = 0, 
  buff_offset = 25, fd = -1, start = 0, end = 24, offset = 0, continue_callback = 0x55cb29adce22 <storage_nio_notify>, 
  done_callback = 0x55cb29ae8dd5 <storage_upload_file_done_callback>, log_callback = 0x0, tv_deal_start = {tv_sec = 0, 
    tv_usec = 0}}
(gdb) b storage_dio_queue_push
Breakpoint 6 at 0x55cb29af14fc: file storage_dio.c, line 147.
(gdb) b blocked_queue_push
Breakpoint 7 at 0x7f01416c4bf0: file fast_blocked_queue.c, line 62.
(gdb) c
Continuing.

Thread 5 "fdfs_storaged" hit Breakpoint 6, storage_dio_queue_push (pTask=pTask@entry=0x7f013ccb4010) at storage_dio.c:147
147	{
(gdb) c
Continuing.

Thread 5 "fdfs_storaged" hit Breakpoint 7, blocked_queue_push (pQueue=0x55cb2aa65b48, pTask=pTask@entry=0x7f013ccb4010)
    at fast_blocked_queue.c:62
62	{
(gdb) n
66		if ((result=pthread_mutex_lock(&(pQueue->lock))) != 0)
(gdb) n
62	{
(gdb) n
66		if ((result=pthread_mutex_lock(&(pQueue->lock))) != 0)
(gdb) n
62	{
(gdb) n
66		if ((result=pthread_mutex_lock(&(pQueue->lock))) != 0)
(gdb) n
76		if (pQueue->tail == NULL)
(gdb) n
75		pTask->next = NULL;
(gdb) n
76		if (pQueue->tail == NULL)
(gdb) n
78			pQueue->head = pTask;
(gdb) n
86		pQueue->tail = pTask;
(gdb) n
88		if ((result=pthread_mutex_unlock(&(pQueue->lock))) != 0)
(gdb) n
98	        pthread_cond_signal(&(pQueue->cond));
(gdb) n
102	}
(gdb) n
storage_dio_queue_push (pTask=pTask@entry=0x7f013ccb4010) at storage_dio.c:165
165	}
(gdb) n
storage_write_to_file (pTask=pTask@entry=0x7f013ccb4010, file_offset=file_offset@entry=0, 
    upload_bytes=upload_bytes@entry=24, buff_offset=<optimized out>, deal_func=<optimized out>, 
    done_callback=done_callback@entry=0x55cb29ae8dd5 <storage_upload_file_done_callback>, 
    clean_func=0x55cb29af1ef4 <dio_write_finish_clean_up>, store_path_index=0) at storage_service.c:7464
7464		return STORAGE_STATUE_DEAL_FILE;
(gdb) n
7465	}
(gdb) n
storage_upload_file (pTask=pTask@entry=0x7f013ccb4010, bAppenderFile=bAppenderFile@entry=false) at storage_service.c:4733
4733	}
(gdb) n
storage_deal_task (pTask=pTask@entry=0x7f013ccb4010) at storage_service.c:8346
8346				STORAGE_ACCESS_LOG(pTask, \
(gdb) n
8482		if (result != STORAGE_STATUE_DEAL_FILE)
(gdb) n
8500	}
(gdb) n
client_sock_read (sock=19, event=<optimized out>, arg=0x7f013ccb4010) at storage_nio.c:424
424	}
(gdb) n
deal_ioevents (ioevent=0x55cb2aa6a7d8) at ioevent_loop.c:26
26	            iterator.count; ioevent->iterator.index++)
(gdb) n
25		for (ioevent->iterator.index=0; ioevent->iterator.index < ioevent->
(gdb) n
26	            iterator.count; ioevent->iterator.index++)
(gdb) n
25		for (ioevent->iterator.index=0; ioevent->iterator.index < ioevent->
(gdb) n
ioevent_loop (pThreadData=pThreadData@entry=0x55cb2aa6a7d8, recv_notify_callback=<optimized out>, 
    clean_up_callback=0x55cb29af07de <task_finish_clean_up>, continue_flag=0x55cb29d2c368 <g_continue_flag>)
    at ioevent_loop.c:147
147				while (pThreadData->deleted_list != NULL)
(gdb) n
158			if (g_current_time - last_check_time > 0)
(gdb) n
160				last_check_time = g_current_time;
(gdb) n
161				count = fast_timer_timeouts_get(
(gdb) n
162					&pThreadData->timer, g_current_time, &head);
(gdb) n
161				count = fast_timer_timeouts_get(
(gdb) n
163				if (count > 0)
(gdb) n
165					deal_timeouts(&head);
(gdb) n
169	        if (pThreadData->notify.enabled)
(gdb) n
184	        if (pThreadData->thread_loop_callback != NULL)
(gdb) n
123		while (*continue_flag)
(gdb) n
125			pThreadData->ev_puller.iterator.count = ioevent_poll(
(gdb) n
127			if (pThreadData->ev_puller.iterator.count > 0)
(gdb) n
125			pThreadData->ev_puller.iterator.count = ioevent_poll(
(gdb) n
127			if (pThreadData->ev_puller.iterator.count > 0)
(gdb) n
129				deal_ioevents(&pThreadData->ev_puller);
(gdb) n

Thread 5 "fdfs_storaged" received signal SIGPIPE, Broken pipe.
0x00007f01418f5a9e in __libc_send (fd=fd@entry=19, buf=0x7f013ccb44a8, len=70, flags=flags@entry=0)
    at ../sysdeps/unix/sysv/linux/send.c:28
28	../sysdeps/unix/sysv/linux/send.c: No such file or directory.
(gdb) n
32	in ../sysdeps/unix/sysv/linux/send.c
(gdb) n
client_sock_write (sock=19, event=event@entry=4, arg=arg@entry=0x7f013ccb4010) at storage_nio.c:468
468			if (bytes < 0)
(gdb) n
470				if (errno == EAGAIN || errno == EWOULDBLOCK)
(gdb) n
474				else if (errno == EINTR)
(gdb) n
484						errno, STRERROR(errno));
(gdb) n
480					logError("file: "__FILE__", line: %d, " \
(gdb) n
483						__LINE__, pTask->client_ip, \
(gdb) n
480					logError("file: "__FILE__", line: %d, " \
(gdb) n
486					task_finish_clean_up(pTask);
(gdb) n
542	}
(gdb) n
storage_send_add_event (pTask=pTask@entry=0x7f013ccb4010) at storage_nio.c:234
234	}
(gdb)

client客户端：storage_client.c:951 ，然后调用 tcpsendfile_ex 上传文件（走1488行逻辑），本质是调用 sendfile 发送（sendfile函数在两个文件描述符之间传递数据（完全在内核中操作），从而避免了内核缓冲区和用户缓冲区之间的数据拷贝，效率很高，被称为零拷贝。）

storage服务器：dio_write_file函数负责写入数据

 6. 上传成功，返回访问路径

最后通过storage_upload_file_done_callback 返回。

Storage将文件写入磁盘后，返回客户端文件的路径和文件名，报文data内容（十六进制）如下：

000000000000003c6400 //10个字节

67726f75703100000000000000000000 //16个字节

4d30302f30302f30302f774b67714856344f51517941626f3959414141415f666453706d673835352e747874  //44个字节

文件下载流程

文件下载命令：

fdfs_download_file /etc/fdfs/client.conf group1/M00/00/00/eBuDxWCuAbOAJAJnAAAA-3Qtcs8577.txt

1.上传请求连接

发起 storage_do_download_file_ex

2 查询可用的storage

服务器响应：tracker_deal_service_query_fetch_update

3 返回storage信息

和上传文件是类似的

4 下载文件

client客户端请求：storage_do_download_file_ex， 然后STORAGE_PROTO_CMD_DOWNLOAD_FILE请求下载。

storage服务响应：storage_server_download_file

storage_read_from_file

storage_download_file_done_callback

真正读取文件的函数：dio_read_file

网络IO模型

默认：g_accept_threads 1

g_work_threads 4

g_disk_reader_threads默认1

以storage为例：

accept线程：入口accept_thread_entrance

IO工作线程： work_thread_entrance，

文件线程：dio_thread_entrance

队列：

storage_dio_queue 文件处理队列

task_queue 任务对象池队列

管道

thread_data.pipe_fds[2]：IO工作线程的触发

storage_nio_notify 触发取读取网络io数据

accept_thread_entrance：

  • 本身是一个线程
  • 调用accept获取新的连接 incomesock = accept(server_sock, (struct sockaddr*)&inaddr, \ &sockaddr_len);
  • 获取对方ip地址：client_addr = getPeerIpaddr(incomesock, \ szClientIp, IP_ADDRESS_SIZE);
  • 从对象池取一个task对象：pTask = free_queue_pop();
  • task对象里面有client的封装信息，需要设置：pClientInfo = (StorageClientInfo *)pTask->arg;
    • pTask->event.fd = incomesock;
    • pClientInfo->stage = FDFS_STORAGE_STAGE_NIO_INIT;
    • 轮询线程：pClientInfo->nio_thread_index = pTask->event.fd % g_work_threads;
    • 通知io线程有新的连接：write(pThreadData->thread_data.pipe_fds[1], &task_addr, \ sizeof(task_addr)) != sizeof(task_addr)

work_thread_entrance：

  • 本身是一个线程
  • 核心调用 ioevent_loop(&pThreadData->thread_data, storage_recv_notify_read, task_finish_clean_up, &g_continue_flag);
  • storage_recv_notify_read 当数据可读时触发
  • 进入到ioevent_loop核心：
  • 实际是调用epoll_wait：pThreadData->ev_puller.iterator.count = ioevent_poll(&pThreadData->ev_puller); 返回可处理事件循环处理事件：deal_ioevents
  • 调用storage_recv_notify_read

文件IO线程dio_thread_entrance：

  • 本身是一个线程
  • 任务由storage_dio_queue_push 投递
  • 从blocked_queue_pop读取任务
  • 通过回调写入文件dio_write_file 或者 读取文件dio_read_file

(gdb) b storage_dio.c:748Breakpoint 1 at 0x55c62441f084: file storage_dio.c, line 748.

(gdb) b client_sock_read

Breakpoint 2 at 0x55c62441e852: file storage_nio.c, line 245.

(gdb) b client_sock_write

Breakpoint 3 at 0x55c62441eb80: file storage_nio.c, line 434.

(gdb) b storage_recv_notify_read

Breakpoint 4 at 0x55c62441ee7f: file storage_nio.c, line 121.

(gdb) b dio_write_file

Breakpoint 5 at 0x55c62441fa46: file storage_dio.c, line 405.

(gdb) b storage_nio_notify

Breakpoint 6 at 0x55c62440adf1: file storage_service.c, line 1918.

(gdb) b storage_upload_file_done_callback

Breakpoint 7 at 0x55c624416da4: file storage_service.c, line 1131.

该用户真的不存在
关注
剖析分布式文件存储系统 FastDFS
AI天才研究院
08-06 1662
FastDFS是一个开源的高性能分布式文件系统，它对文件进行管理、存储、调度等操作。它支持文件的切片、同步加速、文件访问统计等功能。FastDFS客户端库对接语言包括C/C++, Java, PHP, Python, Ruby, Node.js, Go等。FastDFS存储集群可以部署在互联网、公司网络环境中，提供高可靠性和容灾能力。它设计了简单的设计理念和开发模式，非常容易使用和集成。由于其简单易用、性能优秀、自身特色鲜明等特点，使得越来越多的人选择使用FastDFS作为其文件存储系统。
FastDFS安装中出现的问题
xxwwh的博客
10-29 2488
fdfs服务器部署时出错：Failed to start fdfs_trackerd.service: Unit fdfs_trackerd.service not found. 解决方案： sudo /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf start sudo /usr/bin/fdfs_storaged /etc/fdfs/storage.conf start 采用上面的方法，发现fdfs_storaged服务启动不起来 解决方安 ...
Linux 安装 VMware Player
aLingYun的博客
09-14 2273
1. 从官网下载安装包 https://customerconnect.vmware.com/en/downloads/details?downloadGroup=WKST-PLAYER-1612&productId=1039&rPId=66621 2. 执行以下指令去安装 sh VMware-Player-16.1.2-17966106.x86_64.bundle 3. 如果安装完重启后可以直接使用，那么恭喜你，太幸运了。我运行时遇到如下问题： vmplayer //运行 v
ERROR - file: tracker_proto.c, line: 48, server: XXX:22122, response status 2 != 0
热门推荐
lin1234qq1234的博客
08-17 1万+
ERROR - file: tracker_proto.c, line: 48, server: XXX:22122, response status 2 != 0 同一台服务器下安装storage和tracker遇到的问题 遇到该问题的原因:之前注册过一台storage在tracker下面,所以只需要删除之前一台就可以 /usr/bin/fdfs_monitor /etc/fdfs/storage.conf 自行匹配fdfs_monitor 所在目录 查看返回结果中 Storage 1: ip_addr
fastDFS测试上传报错：tracker_proto.c, line: 48, server: x.x.x.x:22122, response status 2 != 0
weixin_44185736的博客
04-12 6675
该错误是 /etc/fdfs 的storage.conf配置文件出错。 修改storage.conf中的store_path0=xx ，xx为你自定义的文件存储位置，路径必须存在 store_path0=/home/fastdfs/storage 改 store_path0=你的路径（必须存在） 配置完需重启。 ...
FastDFS分布式存储实战 - FastDFS.pdf
11-15
FastDFS分布式存储介绍
分布式存储FastDFS
12-24
FastDFS是一个开源的轻量级分布式文件系统，它对文件进行管理，功能包括：文件存储、文件同步、文件访问（文件上传、文件下载）等，解决了大容量存储和负载均衡的问题。特别适合以文件为载体的在线服务，如相册网站...
分布式文件存储FastDFS
weixin_42430543的博客
10-28 84
分布式文件存储FastDFS 学习目标理 理解FastDFS工作流程 分布式文件管理系统 文件上传 文件下载 文件删除 文件缓存控制 搭建文件上传微服务 相册管理(实战) 规格参数管理(实战) 商品分类管理(实战) 1 FastDFS 1.1 FastDFS简介 1.1.1 FastDFS体系结构 FastDFS是一个开源的轻量级分布式文件系统，它对文件进行管理，功能包括：文件存储、文件同步、文件访问（文件上传、文件下载）等，解决了大容量存储和负载均衡的问题。特别适合以文件
2分布式存储fastDFS.zip
08-19
FastDFS是一个开源的轻量级分布式文件系统，它对文件进行管理，功能包括：文件存储、文件同步、文件访问（文件上传、文件下载）等，解决了大容量存储和负载均衡的问题。特别适合以文件为载体的在线服务，如相册网站...
fastdfs遇到的问题汇总
Brother_Wei的博客
09-28 2133
1、上传文件时错误 错误原因：Tracker服务请求调度Storage服务时错误 Tracker错误日志： file: tracker_mem.c, line: 1535, the format of the file "/data/fast_data/data/storage_sync_timestamp.dat" is invalid, group_name: group1, colums: 5 > 4 解决方案： storage_sync_timestamp.dat文件中数据错误
LDD3 scull 编译出错，解决过程【转】
12-24 162
Download ldd_src.rar解压后得ldd_src,进入ldd_src/scull目录，执行 $ make 后报错如下： make-C/lib/modules/2.6.38-8-generic/build M=/home/eric/Downloads/ldd_src/scull LDDINC=/home/eric/Downloads/ldd_src/scull/.....
FastDFS分布式文件系统
blll0的博客
11-24 1174
分布式文件系统FastDFS
FastDFS通信协议详解
Huazie
08-20 1487
FastDFS通信协议详解 FastDFS 采用二进制 TCP 通信协议。
项目实战，集成Redis分布式锁
初听不知曲中意，再听已是曲中人。
03-07 2403
今天我们就来把基于Redis实现的分布式锁，集成到我们的项目中，分布式锁历来都受到大家的关注。不管是工作中、面试中，分布式锁永远是个不老的话题。分布式锁背景为什么要有分布式锁呢？不是已经有synchronized、ReantrantLock等相关锁了吗？是的，我们在开发应用的时候，如果需要对某一个共享变量进行多线程同步访问的时候，可以使用我们学到的锁进行处理，并且可以完美的运行，毫无Bug！注意：这是单机应用，后来业务发展，需要做集群，一个应用需要部署到几台机器上然后做负载均衡 ：上图可以看到，变量A存在三
fastDFS测试上传报错：[2020-05-21 19:40:15] ERROR - file: tracker_proto.c, line: 50, server: 192.168.89.188:
bushitaiben的博客
05-21 3208
[2020-05-21 19:40:15] ERROR - file: tracker_proto.c, line: 50, server: 192.168.89.188:22122, response status 2 != 0 [2020-05-21 19:40:15] ERROR - file: …/client/tracker_client.c, line: 899, fdfs_recv_response fail, result: 2 tracker_query_storage fail, err
fastdfs上传文件时候报错
微电子学与固体电子学-俞驰
02-14 2899
在上传文件的时候报错如下; [2018-10-22 18:50:14] ERROR - file: tracker_proto.c, line: 48, server: 192.168.2.111:22122, response status 2 != 0 [2018-10-22 18:50:14] ERROR - file: ../client/tracker_client.c, line: ...
java 分布式fastdfs删除文件
最新发布
05-13
在本例中，存储组名是“group1”，而文件名是从文件URL中提取出来的。 此外，`delete_file`方法返回一个整数值，表示删除操作的结果。如果结果为0，则表示成功删除文件，否则表示删除文件失败。 最后，需要注意的...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值