容器化部署fastdfs文件存储

目录

一、软件信息

二、构建fastdfs镜像

三、docker 启动fdfs服务

四、k8s部署fdfs服务

1、fdfs部署文件

五、外部服务访问

一、软件信息

  1. fastdfs版本:fastdfs:V5.11

  2. libfastcommon版本: V1.0.36

  3. fastdfs-nginx-module版本:V1.20

  4. nginx版本:1.18.0

二、构建fastdfs镜像

        为了减少操作了文件数,我是直接把启动文件和dockerfile写一起了,也可以把启动脚本 start.sh 单独于dockerfile;然后在构建镜像的时候把启动文件 COPY 到镜像里。

        以下是包括了启动文件为一起的dockerfile构建文件。直接使用该dockerfile即可完成镜像构建。然后按照自己的实际情况来修改配置文件和挂载,即可完成容器化运行fastdfs件存储服务。

FROM centos:7

LABEL creator=tudou date="2024-06-02"

ENV FASTDFS_PATH=/var/fastdfs \
    TZ=Asia/Shanghai \
    TRACKER_PORT= \
    STORAGE_PORT= \
    TRACKER_SERVER= \
    GROUP_NAME= 
    

#get all the dependences
RUN yum install -y git wget zip unzip gcc make \
    perl openssl openssl-devel pcre pcre-devel zlib zlib-devel libevent libevent-devel \
 && yum clean all \
 && mkdir -p ${FASTDFS_PATH}/files

RUN git clone --branch V1.0.36 --depth 1 https://github.com/happyfish100/libfastcommon.git ${FASTDFS_PATH}/libfastcommon \
 && cd ${FASTDFS_PATH}/libfastcommon \
 && ./make.sh \
 && ./make.sh install \
 && rm -rf ${FASTDFS_PATH}/libfastcommon

RUN git clone --branch V5.11 --depth 1 https://github.com/happyfish100/fastdfs.git ${FASTDFS_PATH}/fastdfs \
 && cd ${FASTDFS_PATH}/fastdfs  \
 && ./make.sh \
 && ./make.sh install \
 && cp -r  conf/*.* /etc/fdfs/ \
 && rm -rf ${FASTDFS_PATH}/fastdfs 

RUN git clone --branch V1.20 https://github.com/happyfish100/fastdfs-nginx-module.git /usr/local/fastdfs-nginx-module \
 && cp /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so \
 && sed -i "s@/usr/local/include@/usr/include/fastdfs /usr/include/fastcommon@g" /usr/local/fastdfs-nginx-module/src/config \
 && cd ${FASTDFS_PATH} \
 && wget https://nginx.org/download/nginx-1.18.0.tar.gz \
 && tar -xzvf nginx-1.18.0.tar.gz \
 && cd nginx-1.18.0 \
 && ./configure --prefix=/usr/local/nginx --add-module=/usr/local/fastdfs-nginx-module/src \
 && make \
 && make install \
 && cp /usr/local/fastdfs-nginx-module/src/mod_fastdfs.conf  /etc/fdfs/ \
 && rm -rf ${FASTDFS_PATH}/nginx-1.18.0* 

WORKDIR ${FASTDFS_PATH}

# 启动脚本 start.sh
RUN cat > /usr/bin/start.sh <<-EOF
#!/bin/bash

cd \$(dirname \$0)

# start.sh trackerd 
# start.sh storaged

# # 自定义环境变量初始化,若无自定义环境变量,则直接跳过使用配置文件的默认值
function init_config(){
  if [[ -n \${STORAGE_PORT} ]];then
     sed -i "s|^port=.*$|port=\${STORAGE_PORT}|g" /etc/fdfs/storage.conf
  fi
  if [[ -n \${TRACKER_PORT} ]];then
     sed -i "s|^port=.*$|port=\${TRACKER_PORT}|g" /etc/fdfs/tracker.conf
  fi
  
  if [[ -n \${GROUP_NAME} ]];then
     sed -i "s|group_name=.*$|group_name=\${GROUP_NAME}|g" /etc/fdfs/storage.conf
     sed -i "s|group_name=.*$|group_name=\${GROUP_NAME}|g" /etc/fdfs/mod_fastdfs.conf
     sed -i "s|url_have_group_name =.*$|url_have_group_name = true|g" /etc/fdfs/mod_fastdfs.conf
  fi
  
  if [[ -n \${TRACKER_SERVER} ]];then
     sed -i "s|tracker_server=.*$|tracker_server=\${TRACKER_SERVER}|g" /etc/fdfs/storage.conf
     sed -i "s|tracker_server=.*$|tracker_server=\${TRACKER_SERVER}|g" /etc/fdfs/client.conf
     sed -i "s|tracker_server=.*$|tracker_server=\${TRACKER_SERVER}|g" /etc/fdfs/mod_fastdfs.conf
  fi
  
  sed -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/tracker.conf
  sed -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/storage.conf
  sed -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/client.conf
  sed -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/mod_fastdfs.conf
}

init_config

case \$1 in
     trackerd)
        echo "start trackerd ..."
        fdfs_trackerd /etc/fdfs/tracker.conf start  && tail -f ${FASTDFS_PATH}/logs/trackerd.log
     ;;
	
     storaged)
        sed -i "s|store_path0.*$|store_path0=\${FASTDFS_PATH}/files|g" /etc/fdfs/storage.conf
        sed -i "s|store_path0.*$|store_path0=\${FASTDFS_PATH}/files|g" /etc/fdfs/mod_fastdfs.conf

        echo "start storaged ..."
        fdfs_storaged /etc/fdfs/storage.conf start \
     && /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf \
     && tail -f ${FASTDFS_PATH}/logs/storaged.log
     ;;

     *)
        echo "USAGE sh $0 [ trackerd|storaged ]"
        exit 1
     ;;
esac

EOF
# 启动脚本 start.sh

# 默认fastdfs端口
EXPOSE 22122 23000  8888 80

RUN chmod a+x /usr/bin/start.sh 

ENTRYPOINT ["/usr/bin/start.sh"]
CMD ["trackerd"]
#CMD ["storaged"]
# 构建 fdfs 应用镜像

# docker build -t fdfs:v1.0 -f dockerfile . --no-cache

docker build -t fdfs:v1.0 -f dockerfile . 

三、docker 启动fdfs服务

# 容器启动 trackerd 服务,并且自定义环境变量值,可不指定环境变量值

docker run -itd --name trackerd -e TRACKER_PORT=22122 -e TRACKER_SERVER=192.168.159.132:22122 -p 22122:22122   fdfs:v1.0 trackerd


# 容器启动 storaged 服务,并且自定义环境变量值,可不指定环境变量值

docker run -itd --name storage -e STORAGE_PORT=23000 -e TRACKER_SERVER=192.168.159.132:22122 -p 23000:23000   fdfs:v1.0 storaged

# 分别查看容器应用的日志

docker logs trackerd


docker logs storage


四、k8s部署fdfs服务

1、配置文件

  1.  tracker服务需要修改的配置文件是:tracker.conf,其他文件默认但不能删。
  2. storage服务需要修改的配置文件有: storage.conf、mod_fastdfs.conf 以及安装在storage端的 nginx配置文件 nginx.conf。需要给storage的数据目录做软链接,要不然,有可能无法通过nginx访问 文件,其他配置保持默认即可,但不能删。
  3. client.conf 配置文件不是必须的,可放在任何服务端。
# 配置文件
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: fdfs-config
  namespace: fdfs
  annotations:
    kubesphere.io/creator: admin
    kubesphere.io/description: client.conf 可以不挂载。
data:
  client.conf: |-
    connect_timeout=30
    network_timeout=60
    base_path=/var/fastdfs
    tracker_server=tracker-svc:22122
    log_level=info
    use_connection_pool = false
    connection_pool_max_idle_time = 3600
    load_fdfs_parameters_from_tracker=false
    use_storage_id = false
    storage_ids_filename = storage_ids.conf
    http.tracker_server_port=8080
  mod_fastdfs.conf: |-
    connect_timeout=2
    network_timeout=30
    base_path=/var/fastdfs/files
    load_fdfs_parameters_from_tracker=true
    storage_sync_file_max_delay = 86400
    use_storage_id = false
    storage_ids_filename = storage_ids.conf
    tracker_server=tracker-svc:22122
    storage_server_port=23000
    group_name=group1
    url_have_group_name = true
    store_path_count=1
    store_path0=/var/fastdfs/files
    log_level=info
    log_filename=
    response_mode=proxy
    if_alias_prefix=
    flv_support = true
    flv_extension = flv
    group_count = 1
    [group1]
    group_name=group1
    storage_server_port=23000
    store_path_count=1
    store_path0=/var/fastdfs/files

    #[group2]
    #group_name=group2
    #storage_server_port=23000
    #store_path_count=2
    #store_path0=/var/fastdfs/files
    #store_path1=/var/fastdfs/files1
  nginx.conf: |
    user root;
    worker_processes  2;
    events {
        worker_connections  65535;
         use epoll;
    }
    http {
        include       mime.types;
        default_type  application/octet-stream;
        sendfile        on;
        keepalive_timeout  65;
        charset utf-8;
        server_tokens off;

        server {
            listen       80;
            server_name  localhost;
            location / {
                root   html;
                index  index.html index.htm;
            }
            error_page   500 502 503 504  /50x.html;
            location = /50x.html {
                root   html;
            }
        }
        server {
            listen       8888;
            server_name  localhost;
            location ~/group[1-2]/M00 {
                root /data/fastdfs/store/data;
                # storage 服务要创建软连接: ln -s /var/fastdfs/files/data /data/fastdfs/files/data/M00
                 ngx_fastdfs_module;
            }
            error_page   500 502 503 504  /50x.html;
            location = /50x.html {
                root   html;
            }
        }
    }
  storage.conf: |-
    disabled=false
    group_name=group1
    bind_addr=
    client_bind=true
    port=23000
    connect_timeout=30
    network_timeout=60
    heart_beat_interval=30
    stat_report_interval=60
    base_path=/var/fastdfs/files
    max_connections=256
    buff_size = 256KB
    accept_threads=1
    work_threads=4
    disk_rw_separated = true
    disk_reader_threads = 1
    disk_writer_threads = 1
    sync_wait_msec=50
    sync_interval=0
    sync_start_time=00:00
    sync_end_time=23:59
    write_mark_file_freq=500
    store_path_count=1
    store_path0=/var/fastdfs/files
    subdir_count_per_path=256
    tracker_server=tracker-svc:22122
    log_level=info
    run_by_group=
    run_by_user=
    allow_hosts=*
    file_distribute_path_mode=0
    file_distribute_rotate_count=100
    fsync_after_written_bytes=0
    sync_log_buff_interval=10
    sync_binlog_buff_interval=10
    sync_stat_file_interval=300
    thread_stack_size=512KB
    upload_priority=10
    if_alias_prefix=
    check_file_duplicate=0
    file_signature_method=hash
    key_namespace=FastDFS
    keep_alive=0
    use_access_log = false
    rotate_access_log = false
    access_log_rotate_time=00:00
    rotate_error_log = false
    error_log_rotate_time=00:00
    rotate_access_log_size = 0
    rotate_error_log_size = 0
    log_file_keep_days = 0
    file_sync_skip_invalid_record=false
    use_connection_pool = false
    connection_pool_max_idle_time = 3600
    http.domain_name=
    http.server_port=8888
  tracker.conf: |-
    disabled=false
    bind_addr=
    port=22122
    connect_timeout=30
    network_timeout=60
    base_path=/var/fastdfs
    max_connections=256
    accept_threads=1
    work_threads=4
    min_buff_size = 8KB
    max_buff_size = 128KB
    store_lookup=2
    store_group=group2
    store_server=0
    store_path=0
    download_server=0
    reserved_storage_space = 10%
    log_level=info
    run_by_group=
    run_by_user=
    allow_hosts=*
    sync_log_buff_interval = 10
    check_active_interval = 120
    thread_stack_size = 64KB
    storage_ip_changed_auto_adjust = true
    storage_sync_file_max_delay = 86400
    storage_sync_file_max_time = 300
    use_trunk_file = false
    slot_min_size = 256
    slot_max_size = 16MB
    trunk_file_size = 64MB
    trunk_create_file_advance = false
    trunk_create_file_time_base = 02:00
    trunk_create_file_interval = 86400
    trunk_create_file_space_threshold = 20G
    trunk_init_check_occupying = false
    trunk_init_reload_from_binlog = false
    trunk_compress_binlog_min_interval = 0
    use_storage_id = false
    storage_ids_filename = storage_ids.conf
    id_type_in_filename = ip
    store_slave_file_use_link = false
    rotate_error_log = false
    error_log_rotate_time=00:00
    rotate_error_log_size = 0
    log_file_keep_days = 0
    use_connection_pool = false
    connection_pool_max_idle_time = 3600
    http.server_port=8080
    http.check_alive_interval=30
    http.check_alive_type=tcp
    http.check_alive_uri=/status.html
---

2、fdfs部署文件

         由于storage存储是各自独立的,所以要使用有状态部署。tracker是透明无状态的,可以使用无状态负载deploy部署,也可以使用有状态负载StatefulSet部署。
# kubectl apply -f fdfs-trackerd.yaml -n fdfs
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: fdfs-trackerd
  name: fdfs-trackerd
  namespace: fdfs
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fdfs-trackerd
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: fdfs-trackerd
    spec:
      containers:
      - args:
        - trackerd
        command:
        - /usr/bin/start.sh
        image: swr.cn-south-1.myhuaweicloud.com/tudou/fastdfs:latest
        imagePullPolicy: IfNotPresent
        name: trackerd
        ports:
        - containerPort: 22122
          name: http-0
          protocol: TCP
        - containerPort: 8080
          name: http-1
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/fdfs/tracker.conf
          name: volume-b8e30i
          readOnly: true
          subPath: tracker.conf
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: default
      serviceAccountName: default
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          defaultMode: 420
          name: fdfs-config
        name: volume-b8e30i
---
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: fdfs-trackerd
  name: fdfs-trackerd
  namespace: fdfs
spec:
  ports:
  - name: port-1
    port: 22122
    protocol: TCP
    targetPort: 22122
  - name: port-2
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: fdfs-trackerd
  type: ClusterIP
---



# kubectl apply -f fdfs-storaged.yaml -n fdfs
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: fdfs-storaged
  name: fdfs-storaged
  namespace: fdfs
spec:
  podManagementPolicy: OrderedReady
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: fdfs-storaged
  serviceName: fdfs-storaged
  template:
    metadata:
      labels:
        app: fdfs-storaged
    spec:
      containers:
      - args:
        - storaged
        command:
        - /usr/bin/start.sh
        image: swr.cn-south-1.myhuaweicloud.com/tudou/fastdfs:latest
        imagePullPolicy: IfNotPresent
        name: storage
        ports:
        - containerPort: 23000
          name: http-0
          protocol: TCP
        - containerPort: 8888
          name: http-1
          protocol: TCP
        - containerPort: 80
          name: http-2
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/fdfs/storage.conf
          name: volume-ssmj62
          readOnly: true
          subPath: storage.conf
        - mountPath: /etc/fdfs/mod_fastdfs.conf
          name: volume-zlknkz
          readOnly: true
          subPath: mod_fastdfs.conf
        - mountPath: /usr/local/nginx/conf/nginx.conf
          name: volume-q4ylwb
          readOnly: true
          subPath: nginx.conf
        - mountPath: /var/fastdfs
          name: fdfs-storage-data
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: default
      serviceAccountName: default
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          defaultMode: 420
          name: fdfs-config
        name: volume-ssmj62
      - configMap:
          defaultMode: 420
          name: fdfs-config
        name: volume-zlknkz
      - configMap:
          defaultMode: 420
          name: fdfs-config
        name: volume-q4ylwb
  updateStrategy:
    rollingUpdate:
      partition: 0
    type: RollingUpdate
  volumeClaimTemplates:
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: fdfs-storage-data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 120Gi
      storageClassName: local
      volumeMode: Filesystem
---
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: fdfs-storaged
  name: fdfs-storaged
  namespace: fdfs
spec:
  clusterIP: None
  clusterIPs:
  - None
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: http-0
    port: 23000
    protocol: TCP
    targetPort: 23000
  - name: http-1
    port: 8888
    protocol: TCP
    targetPort: 8888
  - name: http-2
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: fdfs-storaged
  sessionAffinity: None
  type: ClusterIP
---

五、外部服务访问

        当外部服务需要访问fdfs时,需要将服务的 22122和服务的 80 端口暴露。可以将这两个端口对应的服务创建NodePort类型的服务来实现端口暴露。

  • 4
    点赞
  • 8
    收藏
    觉得还不错? 一键收藏
  • 2
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值