回家很久了,实在熬不住,想起来爬点数据玩一玩,之前自己笔记本是win7加ubuntu16.04双系统,本打算在ubuntu里写代码的,可是回到家ubuntu打开一直是紫屏,百度知乎方法用了也没解决,厉害的兄弟可以教下我哦,过年有红包哦!!然后就还是在win7下开始写代码了(电脑太卡,一直不想装Python),今天爬的是豆瓣音乐top250,比较简单,主要是练练手。
代码import requestsimport refrom bs4 import BeautifulSoupimport timeimport pymongo
client = pymongo.MongoClient('localhost', 27017)
douban = client['douban']
musictop = douban['musictop']
headers = { 'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'}
urls = ['https://music.douban.com/top250?start={}'.format(str(i)) for i in range(0,250,25)]def get_url_music(url):
wb_data = requests.get(url,headers=headers)
soup = BeautifulSoup(wb_data.text,'lxm