通过ICE接口对hbase进行文件操作

最近开发了几个对集群做文件操作的模块，都是基于python（2.7）。下面记录一下对通过ICE接口，写HBASE的模块，贴出部分代码。

1. 关于ICE

ICE是一个中间件，我们可以通过ICE，不考虑server和client分别是什么，进行跨集群的操作。ice已经有很多资料可供学习，以后也会自己总结一个关于ice的文档

2. 依赖包

ICE

3. 源码及解释

模块实现的基本内容就是，连接ice服务，按照一定的json格式，将要操作的文件信息及路径传给ice,定的接口json如下：

zip源：
{
    "taskId":"asdfsdafdsf2o3r",
    "files": ["aaa/bbb/a.txt", "aaa/ccc/b.txt"],
    "parent": "/usr/data/input/1/aaa.zip",
    "parentType": "zip"
}

目录源：
{
    "taskId":"asdfsdafdsf2o3r",
    "files": ["aaa/bbb/a.txt", "aaa/ccc/b.txt"],
    "parent": "/usr/data/input/1/",
    "parentType": "nfs"
}

入库结果：
{
    "taskId":"asdfsdafdsf2o3r",
    "parent": "/usr/data/input/1/aaa.zip",
    "result": [
        {"file":"aaa/bbb/a.txt","isSuccess":false,"reason":"putData to hbase failed","length":1234, "costTime":500},
        {"file":"aaa/ccc/b.txt","isSuccess":true,"reason":"","length":1234, "costTime":500}]
}

实现代码段：

def write_hbase(self):
        cfgs = self.configs.configs.app.hbasewriter
        for category in self.contents:
            try:
                self.log_info('begin to write_hbase resource {}', category)
                content_info = self.contents[category]

                msg = channel.Message(data=channel.MessageData())
                uuid_str = uuid.uuid4().hex
                content = Info(taskId=uuid_str, files=content_info.files, parent=content_info.parent,parentType=content_info.parentType)
                msg.content = content.encode()
                msg.data.text = ''
                msg.flag = 0

                result = channel.sync_call(cfgs.proxy, msg)
                result_content = Info().decode(result.content)

                if result_content.result:
                    for res in result_content.result:
                        print('result is {}'.format(res))
                        if trans_bool(res.isSuccess):
                            self.log_info('write_hbase sucess, file: {}, costTime: {}, length: {}'
                                          .format(res.file, res.costTime, res.length))
                        else:
                            self.log_error('write_hbase fail, file: {}, reason: {}'
                                          .format(res.file, res.reason))
                else:
                    self.log_error('get basic data from interface error:{}', result_content.error)
            except:
                self.log_exception('error happen in write_hbase')