环境搭建以及一些命令的记录

1、在CRT 上传文件只需在shell终端仿真器中输入命令“rz”，即可从弹出的对话框中选择本地磁盘上的文件，利用Zmodem上传到服务器当前路径下。
下载文件只需在shell终端仿真器中输入命令“sz 文件名”，即可利用Zmodem将文件下载到本地某目录下。

2、解压：tar -zxvf jdk-7u79-linux-x64.tar.gz -C ~/app

3、启动hdfs，启动yarn，启动hive

show tables;可以看到表

4、启动spark

在software目录先先放置了mysql驱动

/spark-shell --master local[2] --jars ~/software/mysql-connector-java-5.1.27-bin.jar

结果：

[hadoop@hadoop000 bin]$ ./spark-shell --master local[2] --jars ~/software/mysql-connector-java-5.1.27-bin.jar
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
18/01/24 23:57:52 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/01/24 23:58:12 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
Spark context Web UI available at http://192.168.99.141:4040
Spark context available as 'sc' (master = local[2], app id = local-1516809477452).
Spark session available as 'spark'.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.2.0
      /_/


Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_144)
Type in expressions to have them evaluated.
Type :help for more information.