网上关于java集成sqoop的资料很少,这里我写一个java集成sqoop的代码。
sqoop我自然不用介绍了,大家都知道。
直奔主题!
Sqoop Java集成sqoop。数据从Mysql到hdfs
1.windows下载sqoop
网址http://mirror.bit.edu.cn/apache/sqoop/1.4.7/
选择sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz 这个文件夹,点击下载
2.pom文件
mysql mysql-connector-java 5.1.42 org.apache.sqoop sqoop 1.4.7 由于maven中没有sqoop这个包,所以把刚下载的sqoop包中的sqoop-1.4.7.jar 放到你的本地仓库中 ![在这里插入图片描述](https://img-blog.csdnimg.cn/2019112115031214.png) 本地仓库: ![在这里插入图片描述](https://img-blog.csdnimg.cn/20191121150327872.png) ## 3.编写代码import org.apache.hadoop.conf.Configuration;
import org.apache.sqoop.Sqoop;
import org.apache.sqoop.tool.SqoopTool;
import org.apache.sqoop.util.OptionsFileUtil;
public class SqoopTest {
private static int importDataFromMysql() throws Exception {
String[] args = new String[] {
"--connect","jdbc:mysql://master.big:3306/test",
"--driver","com.mysql.jdbc.Driver",
"-username","root",
"-password","root",
"--table","score",
"-m","1",
"--target-dir","/sqoopJavaTest"
};
String[] expandArguments = OptionsFileUtil.expandArguments(args);
SqoopTool tool = SqoopTool.getTool("import");
Configuration conf = new Configuration();
conf.set("fs.default.name", "hdfs://slave1:8020");//设置HDFS服务地址
Configuration loadPlugins = SqoopTool.loadPlugins(conf);
Sqoop sqoop = new Sqoop((com.cloudera.sqoop.tool.SqoopTool) tool, loadPlugins);
return Sqoop.runSqoop(sqoop, expandArguments);
}
public static void main(String[] args) throws Exception {
importDataFromMysql();
}
}
4.查看hdfs,看是否成功
成功。