docker下，极速搭建spark集群(含hdfs集群)，面试真题解析

[root@hedy ~]# docker exec -it master spark-shell --executor-memory 512M --total-executor-cores 2

2019-02-09 17:13:44 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

Setting default log level to “WARN”.

To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).

Spark context Web UI available at http://localhost:4040

Spark context available as ‘sc’ (master = spark://master:7077, app id = app-20190209171354-0000).

Spark session available as ‘spark’.

Welcome to

---

/ / ___ ___/ /

\ / _ / _ `/ __/ '/

// ./_,// //_\ version 2.3.0

/_/

Using Scala version 2.11.8 (Java HotSpot™ 64-Bit Server VM, Java 1.8.0_131)

Type in expressions to have them evaluated.

Type :help for