- 博客(5)
- 资源 (7)
- 收藏
- 关注
RSS订阅转载 spark读写示例
/* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional informati...
2019-04-09 13:58:45
211
原创 spark-sql与HIve
hive的安装 下载hive并解压添加环境变量(增加如下变量) 编辑hive-env.sh 加入 HADOOP_HOME=/opt/apps/software/hadoop-2.7.3 编辑hive-site.xml 加入 <property> <name>javax.jdo.option.ConnectionDriverName</name>...
2019-04-08 16:20:16
127
原创 spark高可用安装
下载scala与spark并解压,添加至环境变量 编辑spark-env.sh 添加如下变量 export JAVA_HOME=/opt/apps/software/jdk1.8.0_201 export SCALA_HOME=/opt/apps/software/scala-2.11.8 export HADOOP_HOME=/opt/apps/software/hadoop-2.7.3 e...
2019-04-08 16:11:06
244
原创 idea测试
导入测试依赖 <!-- test start --> <dependency> <groupId>org.test4j</groupId> <artifactId>test4j.testng</artifactId> <version>2.0.5</version> <...
2019-04-08 10:52:29
332
原创 spark操作
创建sparksession 支持hive val spark = SparkSession.builder().appName(" PlaceCapacity").config("spark.some.config.option", "some-value") .config("spark.sql.hive.filesourcePartitionFileCacheSize", 500 *...
2019-04-04 11:49:39
239
空空如也
TA创建的收藏夹 TA关注的收藏夹
TA关注的人