异步爬取动漫电影fate

A Tarnish

已于 2022-04-02 20:25:44 修改

阅读量187

点赞数

文章标签： python

于 2022-04-02 20:19:19 首次发布

本文链接：https://blog.csdn.net/weixin_51273963/article/details/123928178

版权

本文介绍了一种利用Python异步协程下载m3u8格式视频的方法，通过将大视频拆分为多个ts文件，分别进行下载，最后再进行合并，实现高效且快速的动漫电影fate下载策略。

摘要由CSDN通过智能技术生成

# coding=utf-8
import os
import aiohttp
import aiofiles
import asyncio
def get_ts_url():
    maxurl=[]
    name=1
    with open("../January0121/fate_m3u8.txt","r") as f,open("fate.txt","w") as nf:
        for line in f:
            if line.startswith("#"):
                continue
            else:
                line=line.strip()
                if line.startswith("https://puap"):
                    nf.write(line+".ts$"+str(name)+"\n")
                    maxurl.append(line + ".ts$" + str(name))
                else:
                    maxurl.append(line+"$"+str(name))
                    nf.write(line+"$"+str(name)+"\n")
                name+=1
    return maxurl
async def predownload():
    tasks=[]
    timeout = aiohttp.ClientTimeout(total=600)
    connector = aiohttp.TCPConnector(limit=50)
    async with aiohttp.ClientSession(connector=connector,timeout=timeout) as session:
        async with aiofiles.open("fate.txt","r") as f:
            async for line in f:
                line=line.strip()
                url=line.split("$")[0]
                name=line.split("$")[1]+".ts"
                task=asyncio.create_task(download(url,name,session))
                tasks.append(task)
            await asyncio.wait(tasks)
async def download(url,name,session):
    async with session.get(url) as resp:
        async with aiofiles.open(f"fate/{name}","wb") as f:
            await f.write(await resp.content.read())
        print(f"{name}下载完毕！！！")
def ts_join():
    a=[]
    for i in range(1,1120):
        a.append(r"D:\pythonProject\FootStone\January0122\fate"+str(i)+".ts")
        allstr="+".join(a)
    # os.system(f"copy /b {allstr} movie.mp4")
def changeerror():
    now=[]
    errror_list=os.listdir(r"D:\pythonProject\FootStone\January0122\fate")
    for err in errror_list:
        now.append(int(err.strip(".ts")))
    now.sort()
    m=1
    for n in now:
        if n==m:
            m+=1
        else:
            print(n-1)
            m+=2

if __name__ == '__main__':
    #maxurl=get_ts_url()
    #asyncio.get_event_loop().run_until_complete(predownload())
    #ts_join()
    changeerror()

原理是通过下载并读取m3u8文件将一个大视频分成多个ts文件然后用异步协程下载下来，最后可通过多种途径拼接到一起（压缩包法，cmd法等等）