gearman failover

转载 2013年12月05日 22:11:20

From gearman's main page, they mention running with multiple job servers so if a job server dies, the clients can pick up a new job server. Given the statement and diagram below, it seems that the job servers do not communicate with each other.

Our question is what happens to those jobs that are queued in the job server that died? What is the best practice to have high-availability for these servers to make sure jobs aren't interrupted in a failure?

You are able to run multiple job servers and have the clients and workers connect to the first available job server they are configured with. This way if one job server dies, clients and workers automatically fail over to another job server. You probably don't want to run too many job servers, but having two or three is a good idea for redundancy.

enter image description here


share|improve this question
add comment

As far as I know there is no proper way to handle this at the moment, but as long as you run both job servers with permanent queues (using MySQL or another datastore - just don't use the same actual queue for both servers), you can simply restart the job server and it'll load its queue from the database. This will allow all the queued tasks to be submitted to available workers, even after the server has died.

There is however no automagical way of doing this when a job server goes down, so if both the job server and the datastore goes down (a server running both locally goes down) will leave the tasks in limbo until it gets back online.

The permanent queue is only read on startup (and inserted / deleted from as tasks are submitted and completed).

I'm not sure about the complexity required to add such functionality to gearmand and whether it's actually wanted, but simple "task added, task handed out, task completed"-notifications between servers shouldn't been too complicated to handle.


由于最近工作需要,要涉及Gearman的Client和worker的开发工作,所以研究了一下环境搭建和Client、Worker的开发。 环境搭建工作就不多说了,网上资料很多,不管是部署在linux上...
  • huozhouhftze
  • huozhouhftze
  • 2015年07月12日 23:43
  • 1882


利用Gearman实现异步任务处理 0774 一、问题分析 问题:在性能测试过程中,发现用户管理平台在进行图片上传时,性能不佳。 分析:经过代码分析得知,主要的...
  • luyizhizaio
  • luyizhizaio
  • 2013年12月08日 14:40
  • 1441


Gearman工作流程细解 一次正常的Gearman任务执行流程如上图所示: worker向Gearman Server注册自身可以执行的功能 worker尝试获取一个任务 server通告...
  • xtjsxtj
  • xtjsxtj
  • 2013年11月15日 13:12
  • 6285

Gearman使用Redis 做数据持久化

2014年的时候,老大说数据持久化,当时听的一愣一愣的,不知道啥是持久化;讲了一堆Gearman使用 Redis 和MySQL的东西,表示听不懂;然后他自己在那搞;左后搞出来了,还在launchpad...
  • u014289186
  • u014289186
  • 2016年08月16日 15:04
  • 1045


好久没用Gearman了,服务器上的Gearman server不知道什么时候被停掉了,所以当执行程序时,报错: File "python/", line 41, in ...
  • hello0370
  • hello0370
  • 2015年01月06日 17:08
  • 2074


前面有一篇文章,是Nginx+Lua来实现推送的方式,这里我们换一种作法,用Nginx+Gearman作队列来实现。 此方案的主要思路是这样的: 客户端不会频繁的轮询服务端,而是对服务端发起一个长...
  • xtjsxtj
  • xtjsxtj
  • 2013年12月03日 16:31
  • 3533


一、Gearman架构中的三个角色 client:    请求的发起者,工作任务的需求方(可以是C、PHP、Java、Perl、Mysql udf等等) Job Server:请求的调度者,负责将...
  • fireroll
  • fireroll
  • 2016年06月16日 10:45
  • 1103


CentOS 上安装Gearman及其PHP扩展 发表于 2013 年 5 月 22 日 》》目标 安装 Gearman 服务端(C语言)安装PHP扩展 安装Gearman服务端 # yum i...
  • sunny5211
  • sunny5211
  • 2013年07月26日 13:31
  • 4109


  • adparking
  • adparking
  • 2014年09月30日 11:23
  • 1753


  • qq43599939
  • qq43599939
  • 2017年01月07日 19:51
  • 3045
您举报文章:gearman failover