python入门 爬虫笔记

先将要用到的包导入

import requests
import time
import random
import pandas as pd
import re
import lxml.html

​​​​​​行情中心：国内快捷全面的股票、基金、期货、美股、港股、外汇、黄金、债券行情系统_东方财富网d​​​​​​

到目标网站找到url获取单页的数据并构造headers

def gethtml(cmd,page):
      
    headers = {
        'Accept':'*/*',
        'Accept-Encoding':'gzip, deflate',
        'Accept-Language':'zh-CN,zh;q=0.9',
        'Connection':'keep-alive',
        'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36'
    }
    url="http://quote.eastmoney.com/center/gridlist.html#hs_a_board"
    r &