sparkSQL官网简单解析

###sparksql
1.  sql:访问方式：python;command-line;JDBC/ODBC;
2. Datasets and DataFrames:datasets只支持java，scala(2.2.0);Dataframe:structured data files, tables in Hive, external databases, or existing RDDs. 
3. SparkSession:可以轻松的执行hiveSQL，hiveUDFs,不需要启动hive;在spark2.0出现；


    ```


    from pyspark.sql import SparkSession


    spark = SparkSession \
        .builder \
        .appName("Python Spark SQL basic example") \
        .config("spark.some.config.option", "some-value") \
        .getOrCreate()
    
    ```
4.createDataFrame:create DataFrames from an existing RDD, from a Hive table, or from Spark data sources.
# spark is an existing SparkSession
df = spark.read.json("examples/src/main/resources/people.json")
# Displays the content of the DataFrame to stdout
df.show()
5.Untyped Dataset Operations (aka DataFrame Operations)
df.printSchema()
# root
# |-- age: long (nullable = true)
# |-- name: string (nullable = true)


# Select only the "name" column
df.select("name").show()
# +-------+
# |   name|
# +-------+
# |Michael|
# |   Andy|
# | Justin|
# +-------+


# Select everybody, but increment the age by 1
df.select(df['name'], df['age'] + 1).show()
# +-------+---------+
# |   name|(age + 1)|
# +-------+---------+
# |Michael|     null|
# |   Andy|       31|
# | Justin|       20|
# +-------+---------+


# Select people older than 21
df.filter(df['age'] > 21).show()
# +---+----+
# |age|name|
# +---+----+
# | 30|Andy|
# +---+----+


# Count people by age
df.groupBy("age").count().show()
更多操作：
https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrame
6. Running SQL Queries Programmatically
df.createOrReplaceTempView("people")


sqlDF = spark.sql("SELECT * FROM people")
sqlDF.show()
#Global Temporary View
# Register the DataFrame as a global temporary view
df.createGlobalTempView("people")


# Global temporary view is tied to a system preserved database `global_temp`
spark.sql("SELECT * FROM global_temp.people").show()
# +----+-------+
# | age|   name|
# +----+-------+
# |null|Michael|
# |  30|   Andy|