mysql 数据 迁移 hbase,从MySQL连续数据迁移到Hbase

I have installed hadoop and hbase for real time analytics purpose. The proble I face is to migrate data on line from mysql to Hbase.

The sqoop tool is useful to do bulk data migrations, is there any way that the data from mysql can be transfered to HBase on line (then and there when an insert/update/delete happens). So that real time analytics can be achieved. Not near-real-time.

Please help me on this regards.

解决方案

I think you faced task of setting up replication between different DBMS. It is a case when native replication mechanism is not useful.

Simplest solution would be create set of triggers over tables you want to replicate - and write the data to be replicated into some additional table. Then you can set up monitoring this table and apply changes to the HBase.

More robust, but more complicated solution would be to analyze MySQL log used for its native replication mechanism and apply the changes to the HBase.

In the same time it is not clear for me hoe HBase will give you real-time analytics. I wrote abit about this issue here:

Group by In HBase