写在前面
该问题在django3.2,python3.7.8的开发环境中顺利解决。如是其他django版本可能无法解决。可以做参考。
问题背景
用django写了一个系统平台,开发、调测过程中都没有遇到数据库连接问题。
但是项目上线之后,运行一段时间,有一天突然报错,看报错像是去数据库没查到数据。但是数据库实际是有数据的。因为之前有遇到过项目运行时间一长,出现mysql连接异常的情况(那时候还没用上django的orm)
但是现在用了orm也还是出现这种情况,于是就怀疑是mysql服务主动关闭了连接,造成的出错。
最后根据报错
pymysql.err.InterfaceError: (0, '')
查资料。
有可能发生两种情况
1-查询的数据量过大
2-连接被关闭
结合自己之前遇到的情况,就初步认定是第二个原因导致的。
问题报错
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 73, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python3.7/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/usr/local/lib/python3.7/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/usr/local/lib/python3.7/site-packages/pymysql/connections.py", line 547, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/usr/local/lib/python3.7/site-packages/pymysql/connections.py", line 793, in _execute_command
raise err.InterfaceError(0, "")
pymysql.err.InterfaceError: (0, '')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/apscheduler/executors/base.py", line 125, in run_job
retval = job.func(*job.args, **job.kwargs)
File "/data/ailphatestsystem2/ailpha/apitest/funset.py", line 186, in excuteCaseTask
taskObj = list(Task.objects.filter(**{"task_id":taskid}).values())[0]
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 280, in __iter__
self._fetch_all()
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1324, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 109, in __iter__
for row in compiler.results_iter(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size):
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1130, in results_iter
results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch, chunk_size=chunk_size)
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 98, in execute
return super().execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 73, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python3.7/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/usr/local/lib/python3.7/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/usr/local/lib/python3.7/site-packages/pymysql/connections.py", line 547, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/usr/local/lib/python3.7/site-packages/pymysql/connections.py", line 793, in _execute_command
raise err.InterfaceError(0, "")
django.db.utils.InterfaceError: (0, '')
问题解决
先安装依赖
pip install django-db-connection-pool
然后在项目的settings.py中,设置databases的参数:
DATABASES = {
'default': {
'ENGINE': 'dj_db_conn_pool.backends.mysql', # 需要将驱动改成dj_db_conn_pool.backends.mysql
'NAME': 'test', # 新建数据库名
'USER': 'root', # 数据库登录名
'PASSWORD': '123456', # 数据库登录密码
'HOST': '127.0.0.1', # 数据库所在服务器ip地址
'POOL_OPTIONS': { # dj_db_conn_pool的参数
'POOL_SIZE': 10, # 连接池数量
'MAX_OVERFLOW': 10, # 最大溢出数量
'RECYCLE': 60, # 重新连接的时间
}
}
}
有些资料中说还需要在app配置中注册dj_db_conn_pool。但我没有进行这一步,项目运行也成功了。
解决后的测试
根据SHOW FULL PROCESSLIST;查询结果
发现id字段一直在变。如果系统调用orm的地方,都会刷新某个连接的time字段。