Szcollar的博客

Path: /D is a directory, which is not supported by the record reader when mapreduce.input.fileinputformat.input.dir.recursive is false.windows本地输入不能是文件夹啊

spark windows下idea运行各环境准备：本地必须有spark,hadoop,scala,并配置完好，设置好路径idea要下载scala的相关插件，并设置SDK，maven相关scala的依赖要写好。scala的sdk不可以导入多个scala和spark版本要对应hadoop的windows补丁要存在如果出现内存错误，在配置conf时，加上如下代码：SparkConf conf = new SparkConf().setAppName("test").setMaster("loca

spark安装踩坑1.下载文件应当下载前三个，最后一个很小估计只有代码资源，该有的包都没有。于是直接运行bin/spark-shell 时会报错：Failed to find Spark jars directory (/opt/module/spark-3.1.1/assembly/target2.JNI error报错：A JNI error has occurred, please check your installation and try again这种情况是由于*/opt/modu