gearman安装&使用

为什么80%的码农都做不了架构师？>>>   

Gearman是一个分发任务的程序框架，可以用在各种场合，与Hadoop相比，Gearman更偏向于任务分发功能。它的 任务分布非常 简单，简单得可以只需要用脚本即可完成。Gearman最初用于LiveJournal的图片resize功能，由于图片resize需要消耗大量计算资 源，因此需要调度到后端多台服务器执行，完成任务之后返回前端再呈现到界面。

Gearman 分布式任务实现原理上只用到2个字段，function name和data。function name即任务名称，由client传给job server, job server根据function name选择合适的worker节点来执行。data通常为执行任务所需的自定义的内容，比如简单的做法可以把需要执行的脚本当成data即可(当然要注 意其中的安全防范)。如果有多个worker可以处理同一个function name, 则job server会自动分配一个。当用于远程监控场景时，我们可以让每个worker注册成不同的业务名称，以达到方便控制每台worker节点的目的。

gearmand安装

安装依赖

yum install -y gcc-c++ boost-devel gperf libevent-devel libuuid-devel

boost也可以编译安装(从网上找的，未测试)

//安装ICU4C 
wget http://downloads.sourceforge.net/project/icu/ICU4C/4.0/icu4c-4_0-src.tgz?use_mirror=cdnetworks-kr-2
tar zxvf icu4c-4_0-src.tgz
cd icu/source
./configure –prefix=/usr
make
make install
ldconfig

//安装Boost 
wget http://sourceforge.net/projects/boost/files/boost/1.43.0/boost_1_43_0.tar.gz/download
tar zxvf boost_1_43_0.tar.gz
cd boost_1_43_0
rm -rf /usr/include/boost/
rm -rf /usr/lib/libboost*
./bootstrap.sh -prefix=/usr/local/boost
./b2
编译大概半小时，完成后:
./b2 install

此时编译安装gearman需要制定boost目录
./configure --with-boost=/usr/local/boost --with-boost-libdir=/usr/local/boost/lib

 

编译安装gearman

下载地址https://github.com/gearman/gearmand/releases|https://launchpad.net/gearmand/

# tar xf gearmand-1.1.15.tar.gz
# cd gearmand-1.1.15
# ./configure   --prefix=/app/gearman
# make && make install

运行

/app/gearman/sbin/gearmand -d

 

php扩展安装

下载地址http://pecl.php.net/package/gearman

图简单，直接yum安装php环境
# yum install httpd php php-devel php-mysql php-mhash php-cli php-gd php-common php-mbstring php-ldap php-pdo  mariadb-server   -y

# tar xf gearman-1.1.2.tgz
# cd gearman-1.1.2
# ./configure   --with-php-config=/usr/bin/php-config --with-gearman=/app/gearman
# phpize
# make
# make install
# vim /etc/php.d/gearman.ini
extension=/usr/lib64/php/modules/gearman.so

重启PHP服务

查看状态：/www/wdlinux/php/bin/php --info |grep gearman

PHP测试

Work端

<?php
$worker= new GearmanWorker();
$worker->addServer();
$worker->addFunction("reverse", "my_reverse_function");
while ($worker->work());

function my_reverse_function($job)
{
  return strrev($job->workload());
}
?>

运行

php worker.php

客户端

<?php
$client= new GearmanClient();
$client->addServer();
print $client->do("reverse", "Hello World!");
?>

运行

[root@localhost app]# php client.php
!dlroW olleH

可以看到已经可以使用。

 

JAVA测试

代码引入gearman包

<dependency>
		    <groupId>org.gearman</groupId>
		    <artifactId>gearman-java</artifactId>
		    <version>0.6</version>
</dependency>

编写Work代码

具体执行work的工作类

import org.gearman.client.GearmanJobResult;
import org.gearman.client.GearmanJobResultImpl;
import org.gearman.util.ByteUtils;
import org.gearman.worker.AbstractGearmanFunction;

public class ReverseFunction extends AbstractGearmanFunction {
    public GearmanJobResult executeFunction() {
        StringBuffer sb = new StringBuffer(ByteUtils.fromUTF8Bytes((byte[]) this.data));
        // 封装结果
        GearmanJobResult gjr = new GearmanJobResultImpl(
        		this.jobHandle, true, sb.reverse().toString().getBytes(), new byte[0], new byte[0], 0, 0);
        return gjr;
    }
}

Work端

import java.util.ArrayList;
import java.util.List;
import org.gearman.common.Constants;
import org.gearman.common.GearmanNIOJobServerConnection;
import org.gearman.worker.GearmanFunction;
import org.gearman.worker.GearmanWorker;
import org.gearman.worker.GearmanWorkerImpl;

public class WorkerRunner {

    GearmanNIOJobServerConnection conn;
    List<Class<GearmanFunction>> functions;

    public WorkerRunner(String host, int port, List<Class<GearmanFunction>> funs) {
        conn = new GearmanNIOJobServerConnection(host, port);
        functions = new ArrayList<Class<GearmanFunction>>();
        functions.addAll(funs);
    }

    public void start() {
        GearmanWorker worker = new GearmanWorkerImpl();
        worker.addServer(conn);
        for (Class<GearmanFunction> fun : functions) {
            worker.registerFunction(fun);
        }
        worker.work();
    }

    @SuppressWarnings(value = { "unchecked", "rawtypes" })
    public static void main(String[] args) throws Exception {
        String host = Constants.GEARMAN_DEFAULT_TCP_HOST;
        int port = Constants.GEARMAN_DEFAULT_TCP_PORT;
        Class c = Class.forName(ReverseFunction.class.getCanonicalName());
        if (!GearmanFunction.class.isAssignableFrom(ReverseFunction.class)) {
            System.out.println("Class is not an instance of " + GearmanFunction.class.getCanonicalName());
            return;
        }
        List<Class<GearmanFunction>> functions = new ArrayList<Class<GearmanFunction>>();
        functions.add(c);
        new WorkerRunner(host, port,functions).start();
    }
}

客户端

import org.gearman.client.GearmanClient;
import org.gearman.client.GearmanClientImpl;
import org.gearman.client.GearmanJob;
import org.gearman.client.GearmanJobImpl;
import org.gearman.client.GearmanJobResult;
import org.gearman.common.Constants;
import org.gearman.common.GearmanJobServerConnection;
import org.gearman.common.GearmanNIOJobServerConnection;
import org.gearman.util.ByteUtils;

public class ReverseClient {

    private GearmanClient client;
    
    private String function = ReverseFunction.class.getCanonicalName();

    public ReverseClient(String host, int port) {
    	GearmanJobServerConnection conn = new GearmanNIOJobServerConnection(host, port);
        client = new GearmanClientImpl();
        client.addJobServer(conn);
    }

    public String reverse(String input) {
        String uniqueId = null;
        byte[] data = ByteUtils.toUTF8Bytes(input);
        GearmanJobResult res = null;
        GearmanJob job = GearmanJobImpl.createJob(function, data, uniqueId);
        String value = "";
        client.submit(job);
        try {
            res = job.get();
            value = ByteUtils.fromUTF8Bytes(res.getResults());
        } catch (Exception e) {
            e.printStackTrace();
        }
        return value;
    }

    public void shutdown() throws IllegalStateException {
        if (client == null) {
            throw new IllegalStateException("No client to shutdown");
        }
        client.shutdown();
    }

    public static void main(String[] args) {
        String host = Constants.GEARMAN_DEFAULT_TCP_HOST;
        int port = Constants.GEARMAN_DEFAULT_TCP_PORT;
        String payload = "Hello World";
        ReverseClient rc = new ReverseClient(host, port);
        System.out.println(rc.reverse(payload));
        rc.shutdown();
    }
}

先运行Work端，在运行客户端即可

JAVA和PHP互通测试

1）PHP作为Work端，JAVA作为客户端

Work.php代码不变，ReverseClient.java的function改为reverse即可(Work.php注册的函数名为reverse)，即可做测试。

 

2）JAVA作为Work端，PHP作为客户端

WorkerRunner.java代码不变，client.php的do("reverse", "Hello World!")修改为do("org.inull.util.gearman.ReverseFunction", "Hello World!")，即可做测试。

 

测试第三方的gearman服务器

https://github.com/mywiki/gearman-java，测试，不兼容官方协议吗，不过提供了消息的持久化和web界面功能，也可以试试。

转载于:https://my.oschina.net/mywiki/blog/876912