intellij导入spark maven project并打包运行

一、将maven project导入intellij

1.file->open->选择项目所在的目录->OK

2.等待生成项目后，引入依赖jar包：file->Project Structures->Libraries-> + ->Java ->选择依赖jar包 - OK

二、生成jar包并运行

1.file->Project Structures-> Artfacts -> Jar -> From moudles with dependencies...-> moudle一栏选择项目名称-> OK

2.在界面右侧配置OutPut directory,右击项目名称->Put into Output Root ->apply -OK

三、提交spark任务

上传jar包到spark客户端，执行命令 spark-submit --class com.huawei.bigdata.spark.examples.FemaleInfoCollection --master yarn-client /opt/client/spark/xxxxxxxxxx.jar  /tmp/spark/input