There is a question here connect from java to Hive but mine is different
My hive running on machine1 and I need to pass some queries using Java server running at machine2. As I understand Hive has a JDBC interface for the purpose of receiving remote queries. I took the code from here - HiveServer2 Clients
I installed the dependencies written in the article:
hive-jdbc*.jar
hive-service*.jar
libfb303-0.9.0.jar
libthrift-0.9.0.jar
log4j-1.2.16.jar
slf4j-api-1.6.1.jar
slf4j-log4j12-1.6.1.jar
commons-logging-1.0.4.jar
However I got java.lang.NoClassDefFoundError error at compile time
Full Error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:393)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:187)
at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at com.bidstalk.tools.RawLogsQuerySystem.HiveJdbcClient.main(HiveJdbcClient.java:25)
Another question at StackOverflow recommended to add Hadoop API dependencies in Maven - Hive Error
I don't understand why do I need hadoop API for a client to connect with Hive. Shouldn't JDBC driver be agnostic of the underlying query system? I just need to pass some SQL query?
Edit:
I am using Cloudera(5.3.1), I think I need to add CDH dependencies. Cloudera instance is running hadoop 2.5.0 and HiveServer2
But the servers are at machine 1. On machine the code should at least compile and I should have issues at runtime only!
解决方案
Answering my own question!
With some hit and trial, I have added following dependencies on my pom file and since then I am able to run code on both CHD 5.3.1 and 5.2.1 cluster.
org.apache.hive
hive-jdbc
0.13.1-cdh5.3.1
org.apache.thrift
libthrift
0.9.0
org.apache.thrift
libfb303
0.9.0
org.apache.hadoop
hadoop-core
2.5.0-mr1-cdh5.3.1
org.apache.hadoop
hadoop-common
2.5.0-cdh5.3.1
org.apache.hive
hive-exec
0.13.1-cdh5.3.1
org.apache.hadoop
hadoop-hdfs
2.5.0-cdh5.3.1
Please note that some of these dependencies might not be required