一、启动方法/data/spark-1.4.0-bin-cdh4/bin/spark-sql --master spark://master:7077 --total-executor-cores 10 --executor-memory 1g --executor-cores 2注:/data/spark-1.4.0-bin-cdh4/为spark的安装路径/data/spark-1.4.0-bin-cdh4/bin/spark-sql –help 查看启动选项--master MASTER_URL 指定master url--executor-memory MEM 每个executor的内存,默认为1G--total-executor-cores NUM 所有executor的总核数-e
二、Spark sql对hive支持的功能1、查询语句:SELECT GROUP BY ORDER BY CLUSTER BY SORT BY2、hive操作运算:1) 关系运算:= ==, <>, , >=, <=2) 算术运算:+, -, *, /, %3) 逻辑运算:AND, &&, OR, ||4) 复杂的数据结构5) 数学函数:(sign, ln, cos, etc)6) 字符串函数:3、 UDF4、 UDAF