【无标题】

最新推荐文章于 2024-01-18 23:25:36 发布

神游py

最新推荐文章于 2024-01-18 23:25:36 发布

阅读量90

点赞数

分类专栏：笔记文章标签： python 爬虫 css html flask

本文链接：https://blog.csdn.net/qq_43997786/article/details/132009047

版权

笔记专栏收录该内容

4 篇文章 0 订阅

订阅专栏

该代码实现了一个Python爬虫，利用selenium库和XPath解析从CUB网站抓取并下载鸟类的音频文件。它首先搜索指定的鸟类名称，然后遍历所有页面，获取每个音频的链接并使用wget下载。项目已开源在GitHub上。

摘要由CSDN通过智能技术生成

爬取CUB鸟类音频爬虫

环境

Python==3.8.0
elenium
lxml
requests
tqdm
wget

开源在github

https://github.com/Jacky-Android/spider-cub-voice

from selenium import webdriver
from selenium.webdriver.common.by import By
from lxml import etree
import time
import requests
from tqdm import tqdm
import os 
import wget

def pachong(name):
    driver = webdriver.Edge()
    driver.get('https://xeno-canto.org/explore?query='+name )
    try:
        nums = driver.find_element(By.XPATH,'/html/body/div[1]/nav[1]')
        num_ = nums.find_elements(By.XPATH,'ul/li')
        numss = int(num_[-2].text)
        for j in range(1,numss+1):
            num = driver.find_element(By.XPATH,'/html/body/div[1]/table/tbody').find_elements(By.XPATH,'tr')
            for i in num:
                url = (i.find_element(By.XPATH,'td[12]/a[1]').get_attribute('href'))
                
                try:
                    wget.download(url,name+'/')
                except:
                    continue
            
            driver.find_element(By.XPATH,'/html/body/div[1]/nav[1]').find_elements(By.XPATH,'ul/li')[-1].click()
            #time.sleep(10)
            
    except:
        num = driver.find_element(By.XPATH,'/html/body/div[1]/table/tbody').find_elements(By.XPATH,'tr')
        for i in num:
            url = (i.find_element(By.XPATH,'td[12]/a[1]').get_attribute('href'))
            try:
                wget.download(url,name+'/')
            except:
                continue
    driver.close()  

f=open('class.txt')

data = f.readlines()  
f.close()  # 关
for i in tqdm(range(len(data))):
    os.makedirs(data[i].replace('\n','').split(' ')[-1].split('.')[-1]) 
    
    pachong(data[i].replace('\n','').split(' ')[-1].split('.')[-1])
    time.sleep(10+i)