编译hbase2.2.4+hadoop3.1.2

环境:

ubuntu19.10

hadoop3.1.2

hbase2.2.4

完整报错如下:

java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1092)
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:424)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:586)
	at org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1523)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:938)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2124)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
	at java.lang.Thread.run(Thread.java:748)
2020-05-16 14:51:47,983 ERROR [master/Desktop:16000:becomeActiveMaster] master.HMaster: ***** ABORTING master desktop,16000,1589611905045: Unhandled exception. Starting shutdown. *****
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1092)
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:424)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:586)
	at org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1523)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:938)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2124)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
	at java.lang.Thread.run(Thread.java:748)

网上的一个办法之一是直接设定hbase.unsafe.stream.capability.enforce=false

根据官网的说法，hbase.unsafe.stream.capability.enforce=false存在丢失数据的风险，根本无法用于生产环境。

又想用最新版本又想兼容，只能自己编译了。

＃－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－

准备工作:

https://mirror.bit.edu.cn/apache/hbase/2.2.4/hbase-2.2.4-src.tar.gz

注意配置好maven的阿里云镜像

编译命令

mvn clean package -DskipTests assembly:single -Dhadoop.profile=3.0 -Dhadoop-three.version=3.1.2

 

编译时间Total time: 30:23 min

 

编译结束后得到:

/home/appleyuchi/bigdata/hbase-2.2.4-源码/hbase-assembly/target/hbase-2.2.4-bin.tar.gz

 

然后可以看到得到hbase2.2.4都是依赖于hadoop3.2.1了。

 

Reference:

http://www.meilongkui.com/archives/1544