爬虫，第八次实战xpath之（下载网站的视频播放链接）

import requests
from lxml import etree

headers = {'User-Agen':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
video_list = []
for i in range(4,19):
    url = 'http://www.d9dyy.com/html/dl/list_%s.html'%i
    page_text = requests.get(url=url,headers=headers).text
    tree = etree.HTML(page_text)
    page_li = tree.xpath('/html/body/div[2]/div[2]/ul/li')
    for li in page_li:
            li_url = li.xpath('./a/@href')[0]
            # print(li_url)
            real_url = 'www.d9dyy.com'+li_url
            video_list.append(real_url)
            for video_li in video_list:
                video_li = 'http://'+video_li
                video_text = requests.get(url=video_li,headers=headers).text
                video_tree = etree.HTML(video_text)
                video_player_url = video_tree.xpath('/html/body/div[2]/div[1]/div[1]/iframe/@src')[0]
                # print(video_player_url)
                fp = open('网站影片播放链接.txt','w',encoding='UTF-8')
                fp.write(video_player_url)
                print('辛辛苦苦')

中间有点小不足，太懒了，没把名字给写上去。爬虫慢慢娴熟了，后面要学习更精湛的技术了，就是反反反。。。爬虫技术。哈哈哈

转载于:https://www.cnblogs.com/sucanji/p/10858046.html