sparkStreaming提交任务时指定参数

最新推荐文章于 2021-04-17 11:14:08 发布

辉哥之吞并BAT

最新推荐文章于 2021-04-17 11:14:08 发布

阅读量1.2k

点赞数

分类专栏：大数据文章标签：大数据

本文链接：https://blog.csdn.net/qq_34009542/article/details/72770498

版权

本文介绍了Spark Streaming任务提交时的关键参数选项，包括`--master`, `--deploy-mode`, `--class`, `--jars`, `--packages`, `--executor-memory`等，这些参数用于定义集群连接、应用部署模式、主类、依赖库和执行器内存等，帮助用户定制化Spark Streaming作业的运行环境和性能设置。" 83628388,7672261,深入理解UML：面向对象分析的关键,"['面向对象', 'UML', '软考']

摘要由CSDN通过智能技术生成

1. ******:bin duyang$ ./spark-submit --help
2. Usage: spark-submit [options] <app jar | python file> [app arguments]
3. Usage: spark-submit --kill [submission ID] --master [spark://...]
4. Usage: spark-submit --status [submission ID] --master [spark://...]
5. Usage: spark-submit run-example [options] example-class [example args]
8. Options:
9. --master MASTER_URL spark://host:port, mesos://host:port, yarn, or local.
10. --deploy-mode DEPLOY_MODE Whether to launch the driver program locally ("client") or
11. on one of the worker machines inside the cluster ("cluster")
12. (Default: client).
13. --class CLASS_NAME Your application's main class (for Java / Scala apps).
14. --name NAME A name of your application.
15. --jars JARS Comma-separated lis