python爬取丁香园的疫情数据绘制Echarts地图

最新推荐文章于 2023-03-21 09:31:30 发布

请叫我算术嘉

最新推荐文章于 2023-03-21 09:31:30 发布

阅读量1.8w

点赞数 10

分类专栏： Django python 爬虫文章标签： python 爬虫 django echarts

本文链接：https://blog.csdn.net/ssjdoudou/article/details/104079974

版权

python 同时被 3 个专栏收录

119 篇文章 2 订阅

订阅专栏

Django

59 篇文章 3 订阅

订阅专栏

爬虫

3 篇文章 0 订阅

订阅专栏

2020.2.4更新

完整的项目工程地址https://github.com/Arithmeticjia/wuhan

2020.1.26更新

几乎每天页面都在变，放一下最新的爬虫代码，由于页面需要滚动，加入了selenium

def china_wuhan(request):
    import requests
    from bs4 import BeautifulSoup
    from selenium import webdriver

    try:
        target = 'https://3g.dxy.cn/newh5/view/pneumonia?scene=2&clicktime=1579579384&enterid=1579579384&from=groupmessage&isappinstalled=0'
        # req = requests.get(url=target)
        # req.encoding = 'urf-8'
        # html = req.text
        option = webdriver.ChromeOptions()
        option.add_argument('headless')  # 设置option,后台运行
        driver = webdriver.Chrome(chrome_options=option)
        driver.get(target)
        js = "var q=document.documentElement.scrollTop=1500"
        driver.execute_script(js)
        selenium_page = driver.page_source
        driver.quit()
        soup = BeautifulSoup(selenium_page, 'html.parser')
        cities = soup.find('div', {'class': 'areaBox___3jZkr'})
        # 每个省
        protocols = cities.find_all('div')
        data = {}

        for i in protocols:
            try:
                first = i.find('div', {'class': 'areaBlock1___3V3UU'})
                content = first.find_all('p')
                name = content[0].get_text()
                num = content[1].get_text()
                if num == "":
                    num = 0
                data['{}'.format(name)] = num
            except AttributeError as e:
                continue
    except:
        data = {}


    context = {
        'data': data
    }
    return render(request, 'china-wuhan.html', context=context)

1、软件准备

1、Django2.1
2、request
3、bs4

import requests
from bs4 import BeautifulSoup

target = 'https://3g.dxy.cn/newh5/view/pneumonia?scene=2&clicktime=1579579384&enterid=1579579384&from=groupmessage&isappinstalled=0'
req = requests.get(url=target)
req.encoding = 'urf-8'
html = req.text
soup = BeautifulSoup(html, 'html.parser')
cities = soup.find('div', {'class': 'areaBox___3jZkr'})
# 每个省
protocols = cities.find_all('div')
data = {}

for i in protocols:
    try:
        first = i.find('div', {'class': 'areaBlock1___3V3UU'})
        content = first.find_all('p')
        name = content[0].get_text()
        num = content[1].get_text()
        print(num)
        if num == "":
            num = 0
        data['{}'.format(name)] = num
        print('疫情：', name, '确诊', num, '例')
    except AttributeError as e:
        continue
print(data)

拿到的数据大概是这样

{'湖北': '549', '广东': '53', '浙江': '43', '北京': '26', '上海': '20', '湖南': '9', '安徽': '15', '重庆': '27', '四川': '15', '山东': '9', '广西': '13', '福建': '5', '江苏': '9', '河南': '9', '海南': '8', '天津': '5', '江西': '7', '陕西': '3', '贵州': '3', '辽宁': '3', '香港': '2', '黑龙江': '4', '澳门': '2', '新疆': '2', '甘肃': '2', '云南': '2', '台湾': '1', '山西': '1', '吉林': '3', '河北': '2', '宁夏': '1', '内蒙古': 0}

这是一个字典

然后交给Django返回

def china_wuhan(request):
    import requests
    from bs4 import BeautifulSoup

    target = 'https://3g.dxy.cn/newh5/view/pneumonia?scene=2&clicktime=1579579384&enterid=1579579384&from=groupmessage&isappinstalled=0'
    req = requests.get(url=target)
    req.encoding = 'urf-8'
    html = req.text
    soup = BeautifulSoup(html, 'html.parser')
    cities = soup.find('div', {'class': 'areaBox___3jZkr'})
    # 每个省
    protocols = cities.find_all('div')
    data = {}

    for i in protocols:
        try:
            first = i.find('div', {'class': 'areaBlock1___3V3UU'})
            content = first.find_all('p')
            name = content[0].get_text()
            num = content[1].get_text()
            print(num)
            if num == "":
                num = 0
            data['{}'.format(name)] = num
        except AttributeError as e:
            continue
    context = {
       'data': data
    }
    return render(request, 'china-wuhan.html', context=context)

然后就来到了html，你可以理解为django返回了一个字典格式

对于echarts来说，它需要的数据格式是

var datavirus=[
            {name:"南海诸岛",value: 0},
            {name: '北京', value: 14},
            {name: '天津', value: 4},
            {name: '上海', value: 16},
            {name: '重庆', value: 9},
            {name: '河北', value: 1},
            ...
            ...
]

那么从django返回数据就是这样用

data: [{% for key,value in data.items%}
          {value:{{ value }}, name:'{{ key }}'},
       {% endfor %}
      ]

展示链接https://www.guanacossj.com/china-wuhan/

下面放一下完整的html代码，自行修改

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <link rel="shortcut icon" href="/static/images/写博客.png">
    <title>武汉加油</title>
    <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=0">
    <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"/>
    <script src="/static/china/dist/echarts.js"></script>
    <script src="/static/china/map/js/china.js"></script>
    <style>
        *{margin:0;padding:0}
        html,body{
            width:100%;
            height:100%;
        }
        #main{
              width:850px;
              height:637.5px;
              margin: 150px auto;
              border:1px solid #ddd;
          }
        /*默认长宽比0.75*/
    </style>
</head>
<body>
    <div id="main">
    </div>
    <script type="text/javascript">
        var dataList=[
            {name:"南海诸岛",value: virus(0)},
            {name: '北京', value: virus(14)},
            {name: '天津', value: virus(4)},
            {name: '上海', value: virus(16)},
            {name: '重庆', value: virus(9)},
            {name: '河北', value: virus(1)},
            {name: '河南', value: virus(5)},
            {name: '云南', value: virus(1)},
            {name: '辽宁', value: virus(3)},
            {name: '黑龙江', value: virus(1)},
            {name: '湖南', value: virus(9)},
            {name: '安徽', value: virus(9)},
            {name: '山东', value: virus(6)},
            {name: '新疆', value: virus(0)},
            {name: '江苏', value: virus(1)},
            {name: '浙江', value: virus(10)},
            {name: '江西', value: virus(3)},
            {name: '湖北', value: virus(444)},
            {name: '广西', value: virus(5)},
            {name: '甘肃', value: virus(0)},
            {name: '山西', value: virus(1)},
            {name: '内蒙古', value: virus(-1)},
            {name: '陕西', value: virus(4)},
            {name: '吉林', value: virus(1)},
            {name: '福建', value: virus(4)},
            {name: '贵州', value: virus(3)},
            {name: '广东', value: virus(32)},
            {name: '青海', value: virus(0)},
            {name: '西藏', value: virus(0)},
            {name: '四川', value: virus(8)},
            {name: '宁夏', value: virus(1)},
            {name: '海南', value: virus(4)},
            {name: '台湾', value: virus(1)},
            {name: '香港', value: virus(1)},
            {name: '澳门', value: virus(2)}
        ];
        var datavirus=[
            {name:"南海诸岛",value: 0},
            {name: '北京', value: 14},
            {name: '天津', value: 4},
            {name: '上海', value: 16},
            {name: '重庆', value: 9},
            {name: '河北', value: 1},
            {name: '河南', value: 5},
            {name: '云南', value: 1},
            {name: '辽宁', value: 3},
            {name: '黑龙江', value: 1},
            {name: '湖南', value: 9},
            {name: '安徽', value: 9},
            {name: '山东', value: 6},
            {name: '新疆', value: 0},
            {name: '江苏', value: 1},
            {name: '浙江', value: 10},
            {name: '江西', value: 3},
            {name: '湖北', value: 444},
            {name: '广西', value: 5},
            {name: '甘肃', value: 0},
            {name: '山西', value: 1},
            {name: '内蒙古', value: 0},
            {name: '陕西', value: 4},
            {name: '吉林', value: 1},
            {name: '福建', value: 4},
            {name: '贵州', value: 3},
            {name: '广东', value: 32},
            {name: '青海', value: 0},
            {name: '西藏', value: 0},
            {name: '四川', value: 8},
            {name: '宁夏', value: 1},
            {name: '海南', value: 5},
            {name: '台湾', value: 1},
            {name: '香港', value: 1},
            {name: '澳门', value: 2}
        ];
        var myChart = echarts.init(document.getElementById('main'));
        function randomValue() {
            return Math.round(Math.random()*1000);
        }
        function virus(num){
            if (num === 0){
                return 0;
            }else if(num >= 1 && num <= 9){
                return 2;
            }else if(num >=10 && num <= 100){
                return 3;
            }else if(num > 100){
                return 4;
            }else
                return 1;
        }
        option = {
            backgroundColor: '#404a59',
            title:  {
                text: '武汉',
                x: 'center',
                textStyle: {//主标题文本样式{"fontSize": 18,"fontWeight": "bolder","color": "#333"}
                    fontSize:28,
                    color: '#fff'
                },
            },
            tooltip: {
                formatter:function(params,ticket, callback){
                    return params.seriesName+'<br />'+params.name+'：'+params.value
                }//数据格式化
            },
            visualMap: {
                min: 0,
                max: 50,
                align: 'auto',
                {#type: 'piecewise',    //是否连续#}
                left: 'left',
                top: 'bottom',
                text: ['高','低'],    //取值范围的文字
                {#categories: [#}
                {#    '0','1','2','3','4','5','6','7','8','9','10','14','16','32','444',#}
                {#],#}
                inRange: {
                    color: ['#fdfff8','#f0a23f','#dd5340','#dd292f','#DD2020'],
                    {#color: {#}
                    {#    '0':'#fcffde',#}
                    {#    '1':'#ffaa2a',#}
                    {#    '2':'#ffaa2a',#}
                    {#    '3':'#ffaa2a',#}
                    {#    '4':'#ffaa2a',#}
                    {#    '5':'#ffaa2a',#}
                    {#    '6':'#ffaa2a',#}
                    {#    '7':'#ffaa2a',#}
                    {#    '8':'#ffaa2a',#}
                    {#    '9':'#ffaa2a',#}
                    {#    '10':'#f06a5c',#}
                    {#    '14':'#f06a5c',#}
                    {#    '16':'#f06a5c',#}
                    {#    '32':'#f06a5c',#}
                    {#    '444':'#dd292f',#}
                    //取值范围的颜色
                },
                show:false//图注
            },
            geo: {
                map: 'china',
                roam: false,//不开启缩放和平移
                zoom:1.23,//视角缩放比例
                label: {
                    normal: {
                        show: true,
                        fontSize:'10',
                        color: 'rgba(0,0,0,0.7)'
                    }
                },
                itemStyle: {
                    normal:{
                        borderColor: 'rgba(0, 0, 0, 0.2)'
                    },
                    emphasis:{
                        areaColor: '#53adf3',//鼠标选择区域颜色
                        shadowOffsetX: 0,
                        shadowOffsetY: 0,
                        shadowBlur: 20,
                        borderWidth: 0,
                        shadowColor: 'rgba(0, 0, 0, 0.5)'
                    }
                }
            },
            series : [
                {
                    name: '确诊人数',
                    type: 'map',
                    geoIndex: 0,
                    data: [{% for key,value in data.items%}
                        {value:{{ value }}, name:'{{ key }}'},
                    {% endfor %}]
                }
            ]
        };
        myChart.setOption(option);
        myChart.on('click', function (params) {
            alert(params.name+"确诊人数"+params.data['value']);
        });

      /*  setTimeout(function () {
            myChart.setOption({
                series : [
                    {
                        name: '信息量',
                        type: 'map',
                        geoIndex: 0,
                        data:dataList
                    }
                ]
            });
        },1000)*/
    </script>
</body>
</html>