Python-简单的获取 超链 文字信息与URL 信息

#!/usr/bin/python
# Filename: testspider.py
# coding: UTF-8

import requests
from bs4 import BeautifulSoup
from lxml import etree

link = "http://www.santostang.com/"
headers = {'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT6.1;en-US;rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6'}

r = requests.get(link, headers = headers)
soup = BeautifulSoup(r.text,"lxml")
title_list = soup.find_all("h1",class_="post-title")
for str in title_list:
    url = str.a['href']
    title = str.a.text.strip()
    print ("文章标题：%s 文章URL：%s" %(title,url))
 

结果：

文章标题：第四章 – 4.3 通过selenium 模拟浏览器抓取 文章URL：http://www.santostang.com/2018/07/15/4-3-%e9%80%9a%e8%bf%87selenium-%e6%a8%a1%e6%8b%9f%e6%b5%8f%e8%a7%88%e5%99%a8%e6%8a%93%e5%8f%96/
文章标题：第四章 – 4.2 解析真实地址抓取 文章URL：http://www.santostang.com/2018/07/14/4-2-%e8%a7%a3%e6%9e%90%e7%9c%9f%e5%ae%9e%e5%9c%b0%e5%9d%80%e6%8a%93%e5%8f%96/