celery使用redis作为broker的话,apply_async的异步任务如果执行时间是延迟超过1小时的话,celery会重复发布任务,会导致任务重复执行。
如:update_DATenc.apply_async((1, 2), countdown=8400),update_DATenc任务在8400秒后执行。
这将导致在8400秒内,每隔一小时触发一次update_DATenc任务
这个在官方文档中有说明celery文档
if a task isn’t acknowledged within the Visibility Timeout the task will be redelivered to another worker and executed.This causes problems with ETA/countdown/retry tasks where the time to execute exceeds the visibility timeout; in fact if that happens it will be executed again, and again in a loop.So you have to increase the visibility timeout to match the time of the longest ETA you’re planning to use.Note that Celery will redeliver messages at worker shutdown, so having a long visibility timeout will only delay the redelivery of ‘lost’ tasks in the event of a power failure or forcefully terminated workers.Periodic tasks won’t be affected by the visibility timeout, as this is a concept separate from ETA/countdown.You can increase this timeout by configuring a transport option with the same name:
app.conf.broker_transport_options = {'visibility_timeout': 43200}*
The value must be an int describing the number of seconds.
这个时候,我们就需要在配置文件里面增加一项,来增大visibility_timeout的时间BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 43200}
或者app.conf.broker_transport_options = {'visibility_timeout': 43200}