在学习神经网络相关知识时,在搜索时可能获取的一些学习知识网页链接较少,简单利用爬虫爬取一些网页链接,正好也学习一下python爬虫啦,看下有没有新鲜的学习素材,代码如下:
import requests
from bs4 import BeautifulSoup
url = 'https://www.google.com/search?q=neural+network+learning'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
links = []
for link in soup.find_all('a'):
if 'https://' in link.get('href'):
links.append(link.get('href'))
with open('neural_network_learning_links.txt', 'w') as f:
for link in links:
f.write(link + '\n')
爬取到的一些链接会保存在tet文本中,在VSCode中打开可以以ctrl+单机直接访问链接: