spark-sql-hive
spark-sql --database src --hiveconf hive.metastore.uris=thrift://hdp02:2083 -e "select * from src_db2" --driver-java-options '-Djava.net.preferIPv4Stack=true'
spark-sql-jdbc
1.启动spark-sql
## --driver-class-path:指定Driver端所需jar
## --jars :指定executor所需要jar
spark-sql --driver-class-path /home/dw/hubg/mysql-connector-java-5.1.45-bin.jar \
--jars /home/dw/hubg/mysql-connector-java-5.1.45-bin.jar
2.执行Spark SQL语句:
spark-sql> CREATE TEMPORARY VIEW spark_mysql USING org.apache.spark.sql.jdbc OPTIONS ( url "jdbc:mysql://xxx.xx.xx.x:xxxx/shurongtest", dbtable "dmpt_serv_object",user 'root',password 'root');
Time taken: 1.504 seconds
22/08/01 15:21:48 INFO SparkSQLCLIDriver: Time taken: 1.504 seconds
spark-sql> select * from spark_mysql;
1541964275789103105 第一条 4bb8df021931375dc80f056f433ba2b6 2022-06-30 12:59:59 第一条 2022-06-29 22:58:26 admin 2022-06-30 22:41:32 admin
1542701764438929409 第二条2 57119498e7649c33e2c5e4f56f2a50dc 2022-07-02 12:59:59 第二条 2022-07-01 23:48:57 yanshi_0627 2022-07-19 04:00:44 admin
1543104243022495745 服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1 9d48f9824f9cb02190f331267966150c 2022-07-03 12:59:59 服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1服务对象1 2022-07-03 02:28:16 yhx 2022-07-03 02:28:16 yhx
1544282208364134401 11111111 c334ad6011cf3100a5e61f2832a4ae56 2022-07-06 12:59:59 2022-07-06 08:29:05 admin 2022-07-06 08:29:05 admin
1546804008885166082 111 a2184be4c6f233380e54ce128dd73bbe 2022-07-13 12:59:59 111 2022-07-13 07:29:49 admin 2022-07-13 07:29:49 admin
Time taken: 12.592 seconds, Fetched 5 row(s)
22/08/01 15:22:04 INFO SparkSQLCLIDriver: Time taken: 12.592 seconds, Fetched 5 row(s)