【python】多线程获取数据渲染折线图笔记整理

文章整个流程通过神策api获取数据，生成折线趋势图，业务功能和简单，

• matplotlib
• requests
• threading

获取数据

def getData(start, end):
    sql = "select count(distinct(stu_id)) as num, date from events where "+_os+" date between '"+start+"' and '"+end+"' and business='ai'  group by date order by date"
    print(sql)
    data = {'format': 'json', 'q': sql}
    req_url = "https://"+url+"/api/sql/query?token="+token+"&project=production"
    req_header = {
        'User-Agent':'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36',
    }
    response = requests.post(req_url,data = data, headers = req_header)
    rst = response.text.split('\n')
    del rst[-1]
    print('user'+str(len(rst)))

    return rst

这里注意的地方 del rst[-1] 删除最后一项空数据，防止后面解析异常。
另外就是 requests package的学习

渲染图片

def showImage(xs,ys,hs):
    plt.figure(num=1, figsize=(len(xs)//5,10))
    plt.ylabel('count', color='red')
    plt.xticks(rotation=45, fontsize=10) 
    plt.xlabel("date", fontsize=10)
    #设置区间
    plt.xlim(0,len(ys))
    plt.ylim(0,max(xs)+100)
    #设置显示间隔 
    y_major_locator=MultipleLocator(200) # y轴 0 200 400 600间隔都是200
    x_major_locator=MultipleLocator(len(xs)//30) 
    #设置边距
    plt.subplots_adjust(left=0.05, right=0.95, top=0.9, bottom=0.1)
    plt.title(gstart+"~"+gend, fontsize=20)
    # 这里是核心渲染的数据
    plt.plot(ys,xs, color='green', linewidth=3, label='total')
    plt.plot(ys,hs, color='red', linestyle='--', linewidth=2, label='support')
    # --- end -- 
    plt.legend() #显示图例
    plt.savefig("人数-"+gstart+"~"+gend+".png", bbox_inches='tight')
    # plt.show();

折线图学习 参考地址 https://www.runoob.com/w3cnote/matplotlib-tutorial.html

照虎画猫 线程

import threading
import time

class TaskThread(threading.Thread):
    def __init__(self, func, args=()):
        super(TaskThread, self).__init__()
        self.func = func
        self.args = args

    def run(self):
        self.result = self.func(self.args)

    def getresult(self):
        try:
            return self.result
        except Exception as ex:
            print(ex)
            return "ERROR"

看图上有两条线也就是说有两组对比的数据，如果说单独请求，整体需要耗费20s左右，进一步学习吧， 看了下线程部分，就照虎画猫，搬下来，用了多线程之后差不多 7~9秒左右；

def main():
    usertask = taskthread.TaskThread(getUserTask, args=[gstart, gend])
    helptask = taskthread.TaskThread(getHelpTask, args=[gstart, gend])
    tasklist = [usertask,helptask]
    usertask.start()
    helptask.start()

    resultlist = []
    for task in tasklist:
        task.join()
        resultlist.append(task.getresult())

    showResult(resultlist[0], resultlist[1])


if __name__ == "__main__":
    _sts = int(time.time()*1000)
    # user = getData(gstart, gend)
    # helplist = getHelpData(gstart, gend)
    # showResult(user, helplist)
    main()
    print('dur',int(time.time()*1000) - _sts)

• time.time() 获取时间戳 float类型 参考地址 https://ibeeger.com/blog/2018/11/13/python-time-md/

源码已经提交git https://github.com/ibeeger/python-learn