python调用spark和调用hive_[Spark][Hive][Python][SQL]Spark 读取Hive表的小例子

[Spark][Hive][Python][SQL]Spark 读取Hive表的小例子

$ cat customers.txt

1Alius

2Bsbca

3Carlsmx

$ hive

hive>

> CREATE TABLE IF NOT EXISTS customers(

> cust_id string,

> name string,

> country string

> )

> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';

hive> load data local inpath '/home/training/customers.txt' into table customers;

hive>exit

$pyspark

sqlContext =HiveContext(sc)

filterDF=sqlContext.sql(""" SELECT * FROM customers WHERE name LIKE "A%" """)

filterDF.limit(3).show()

+-------+----+-------+

|cust_id|name|country|

+-------+----+-------+

| 001| Ani| us|

+-------+----+-------+