django+gunicorn部署web及性能测试

部署

• pip换国内源

mkdir ~/.pip && touch ~/.pip/pip.conf
#####################################
[global]
timeout = 6000
index-url = http://mirrors.aliyun.com/pypi/simple/
trusted-host = mirrors.aliyun.com

• Dockerfile

FROM python:3.7.7-alpine
MAINTAINER duanyiwen dyiwen@aliyun.com
RUN sed -i 's/dl-cdn.alpinelinux.org/mirror.tuna.tsinghua.edu.cn/g' /etc/apk/repositories && \
    apk add --no-cache tzdata && \
    cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN apk add --no-cache --virtual .build-dependencies build-base jpeg-dev curl-dev g++ libffi-dev python-dev
RUN apk del tzdata && \
    rm -rf /var/cache/apk/* && \
    rm -rf /root/.cache && \
    rm -rf /tmp/*
COPY ./Pipenv /home/
RUN mkdir ~/.pip && touch ~/.pip/pip.conf && \
    echo -e "[global]\ntimeout = 6000\nindex-url = http://mirrors.aliyun.com/pypi/simple/\ntrusted-host = mirrors.aliyun.com" > ~/.pip/pip.conf && \
    pip install pipenv && \
    cd /home && pipenv install --skip-lock
WORKDIR /home

• 使用
  docker run -itd --network host --name=${容器名称} -p ${PORT}:${PORT} -v ${工作空间}:/home python:3.7.7-alpine-django sh -c 'pipenv install --skip-lock && pipenv run gunicorn -w ${进程数} -k gthread --threads ${线程数} -b 0.0.0.0:${PORT} --log-level DEBUG cashier.wsgi:app --preload
• gunicorn并发模式
  模式种类：-k [sync,eventlet,gevent,tornado,gthread,gaiohttp]
  • sync
  • eventlet：需要下载eventlet>=0.9.7
  • gevent：需要下载gevent>=0.13
  • tornado：需要下载tornado>=0.2
  • gthread
  • gaiohttp：需要python 3.4和aiohttp>=0.21.5
    • 解释
      • 1.sync 底層實作是每個請求都由一個 process 處理(进程处理)。
      • 2.gthread 則是每個請求都由一個 thread 處理(线程处理)。
      • eventlet、gevent、tarnado 底層則是利用非同步 IO 讓一個 process 在等待 IO 回應時繼續處理下個請求(协程处理)。

性能测试

参考: https://www.jianshu.com/p/e7962469722a

runserver 方式压测结果

Transactions:               24041 hits
Availability:               99.93 %
Elapsed time:               99.60 secs
Data transferred:            0.32 MB
Response time:                1.03 secs
Transaction rate:          241.38 trans/sec  # 并发量只有241
Throughput:                0.00 MB/sec
Concurrency:              248.94
Successful transactions:       24041
Failed transactions:              16
Longest transaction:           32.55
Shortest transaction:            0.05

gunicorn + gevent (4个worker)

Transactions:               23056 hits
Availability:              100.00 %
Elapsed time:               99.49 secs
Data transferred:            0.31 MB
Response time:                1.09 secs
Transaction rate:          231.74 trans/sec # 并发量只有231
Throughput:                0.00 MB/sec
Concurrency:              252.95
Successful transactions:       23056
Failed transactions:               0
Longest transaction:            8.21
Shortest transaction:            0.01

gunicorn + gthread (4个worker, --threads=50)

Transactions:               28231 hits
Availability:               95.67 %
Elapsed time:               30.71 secs
Data transferred:            0.41 MB
Response time:                0.27 secs
Transaction rate:          919.28 trans/sec  # 提高了不少吧，能不能在提高？
Throughput:                0.01 MB/sec
Concurrency:              251.06
Successful transactions:       28231
Failed transactions:            1278        # 但是失败的有些多
Longest transaction:            8.06
Shortest transaction:            0.01

gunicorn + gthread + CONN_MAX_AGE(4个worker, --threads=50)

数据库配置conn_max_age：https://docs.djangoproject.com/en/1.11/ref/databases/

Transactions:              110289 hits
Availability:               99.62 %
Elapsed time:               99.65 secs
Data transferred:            1.47 MB
Response time:                0.23 secs
Transaction rate:         1106.76 trans/sec  # 这次又提升了不少啊
Throughput:                0.01 MB/sec
Concurrency:              253.84
Successful transactions:      110289
Failed transactions:             422
Longest transaction:            3.85
Shortest transaction:            0.01

gunicorn+gevent+CONN_MAX_AGE(4个worker)

会导致数据库连接数飙升，需要优化django数据连接部分的代码，改写使用数据库连接池
django数据库连接复用