基于Python的Arxiv文章批量下载

最新推荐文章于 2025-02-28 12:30:13 发布

ghostInSh3ll

最新推荐文章于 2025-02-28 12:30:13 发布

阅读量2.4k

点赞数 2

分类专栏：论文下载文章标签： python arxiv

本文链接：https://blog.csdn.net/AAAAshin/article/details/114117509

版权

将你需要下载的arxiv文章链接或者arxiv number，分行写入arxiv_paper_list.txt
形如
https://arxiv.org/pdf/2006.09238
https://arxiv.org/pdf/2009.14178
https://arxiv.org/pdf/2101.01373
运行arxivPaperDownload.py,代码基于项目(https://github.com/Tachyu/Arxiv-download/tree/c9eb7cdcff80a86c7df4db062572a7439cf5383c)

import time 
from selenium import webdriver
import requests
import threading
import os 

def Handler(start, end, url, filename): 
    # specify the starting and ending of the file 
    headers = {
   'Range': 'bytes=%d-%d' % (start, end)} 
    # request the specified part and get into variable     
    r = requests.get(url, headers=headers, stream=True) 
    # open the file and write the content of the html page into file. 
    with open(filename, "r+b") as fp: 
        fp.seek(start) 
        var = fp.tell() 
        fp.write(r.content)

def download_file(url_of_file,name,number_of_threads): 
    r = requests.head(url_of_file) 
    if name: 
        file_name = name 
    else: 
        file_name = url_of_file