docker部署python项目、爬虫注意事项【持续更新】

python项目部署在docker，记录一下出现的问题

pip下载安装包慢、超时、失败的问题

解决办法

时区不正确的问题

异常提示
Timezone offset does not match system offset: 0 != 28800. Please, check your config files.
解决办法

1. 修改代码中BlockingScheduler、BackgroundScheduler指定时区为timezone=“Asia/Shanghai”

 	# 示例
    block_scheduler = BlockingScheduler(timezone="Asia/Shanghai")

2. 程序启动时映射/etc/localtime:/etc/localtime

# 示例
docker run -d -v /etc/localtime:/etc/localtime -p 8080:8080 crawl:1.0

izihawa-commons安装失败

ERROR: Could not find a version that satisfies the requirement izihawa-commons>=0.0.10 (from clickhouse) (from versions: none)

ERROR: No matching distribution found for izihawa-commons>=0.0.10

解决方案：

• 使用豆瓣镜像源
  修改Dockerfile在pip -r 前加上如下指令

RUN pip install -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com izihawa-commons

使用flask启动时报错

ValueError: Timezone offset does not match system offset: 0 != 28800. Please, check your config files

• 原因分析：在启动时挂载了/etc/localtime
  使用date -R查看确实是+0800时区，代码层面虽然指定了timezone，但是没有生效
• 修改前代码

app = Flask(__name__)
scheduler = APScheduler(BackgroundScheduler(timezone="Asia/Shanghai"))

• 查看源码：

		timezone = self.app.config.get('SCHEDULER_TIMEZONE')
        if timezone:
            options['timezone'] = timezone

        self._scheduler.configure(**options)

• 源码位置截图
  
• 修改代码：在配置中加入SCHEDULER_TIMEZONE = "Asia/Shanghai"

class Config:
    JOBS = [
        {
            "id": "nk_job_detail_1",
            "func": "service:run",
            "args": None,
            "trigger": "cron",
            "day": "*/1",
            "hour": "1"
        }
    ]

    SCHEDULER_API_ENABLED = True
    # 加入配置
    SCHEDULER_TIMEZONE = "Asia/Shanghai"

项目中有python包，docker启动报错

ModuleNotFoundError: No module named xxx
例如下目录结构

启动py文件中已经加入了系统环境变量，而且python -m site打印可以看出当前项目确实已经加入到path路径下

• 解决办法：
  修改dockerfile 文件

FROM python:3.9

RUN mkdir /app
# 重点加add指令
ADD . /app
WORKDIR /app
RUN ["python","-m","pip","install","--upgrade","pip"]

RUN ["python","-m","site"]
COPY requirements.txt requirements.txt
RUN pip install -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com --default-timeout=20 --no-cache-dir -r requirements.txt
#RUN pip install -r requirements.txt

COPY ./* /app/

VOLUME ["/data/mo"]

CMD [ "python", "/app/app.py" ]

修改点ADD . /app