zhengzaifeidelushang的博客

深入浅出理解数据分析系列之：python绘图和可视化一、figure和add_subplot二、subplots和subplots_adjust三、颜色、标记和线型四、刻度和标签五、图例和注解六、将图标保存到文件一、figure和add_subplotimport numpy as npimport matplotlib.pyplot as pltimport pandas as pdfrom numpy.random import randnfig = plt.figure()ax1 =

Presto查询数据，通过pyspark把数据存入mysql数据库Python连接Presto查询数据pyspark读写mysql数据库pyspark读取mysql、sqlserver数据库数据，pymysql往mysql数据库中写入数据from pyhive import prestofrom pyspark.shell import sparkfrom requests import Sessionimport requestsfrom requests.auth import HTTP

Python连接Presto查询数据一、pyhive二、presto-python-client一、pyhivepip3 install pyhivefrom pyhive import prestofrom requests import Sessionimport requestsfrom requests.auth import HTTPBasicAuthrequests.packages.urllib3.disable_warnings()req_kw = { 'auth'

Spark3查询Hive表最新分区方法一：spark sql查询max(分区字段)方法二：spark 代码show partitions表方法一：spark sql查询max(分区字段)%sqlselect max(date_time) from 表名方法二：spark 代码show partitions表%sparkval partition_df = spark.sql( s"show partitions optics.data " )partition_df: org.apache.

Spark3注册udf和使用窗口函数一、准备数据二、pyspark注册udf三、pyspark使用窗口函数一、准备数据%pysparkdata = [(1,"火男",6000),(1,"流浪法师",7000),(2,"盖伦",10000),(2,"皇子",8000)]df = spark.createDataFrame(data,["属性","角色名称","战斗力"])df.show()二、pyspark注册udf%pyspark#注册udfimport pandas as pdfr

Zeppelin上通过Spark读写mysql数据库一、从mysql数据库获取数据%sparkval df = spark.read.format("jdbc") .option("url","jdbc:mysql://192.168.216.130:3306/dw") .option("dbtable","stu_info") .option("password","xxxx") .load() .select("id","name","sex") .withColumn("id",co