bilibili视频批量下载

看到B站一个机器学习的系列视频，为节约时间，想将视频下载后加速观看，视频太多，一个个下载太费事，研究了一下通过程序得到所有的下载连接，直接放迅雷中批量下载。

1. 下载起始页面

   https://space.bilibili.com/97678687/#/video?tid=0&page=1&keyword=&order=pubdate（我需下载的内容）

   

   在URL中可以得到用户ID，这个ID在下面会用到。

2. 视频列表链接

   分析第1步中的页面请求，发现视频列表由以下请求得到，参数者是字面意思，不解释

   https://space.bilibili.com/ajax/member/getSubmitVideos?mid=97678687&pagesize=30&tid=0&page=1&keyword=&order=pubdate

   请求的json数据，给合视频地址信息，aid为视频ID

   
   
3. 得到某一视频的下载连接
   https://www.bilibili.com/video/av15705305

   
   

   https://www.bilibili.com/video/av15705305

   问度娘，可知下载链接是在bilibili前面加kan

   

   点击 超清 批量下载 得到下载地址：

   分析请求 可知下载连接由

   https://www.kanbilibili.com/api/video/15705305/download?cid=25572499&quality=80&page=1

   cid 可从 https://www.kanbilibili.com/video/av15705305 中得到。

   到这里，介绍完了得到一个完整的下载链接过程了。
4. 最重要code

   import json
   import requests
   import re
   from scrapy.selector import Selector
   
   def make_unicode(value, prefer_encodings=None):
       if prefer_encodings is None:
           prefer_encodings = ['utf8', 'gbk', 'gbk?']
       if isinstance(value, unicode) or value is None:
           return value
       if not isinstance(value, str):
           return value
       for enc in prefer_encodings:
           try:
               if enc.endswith('!'):
                   return value.decode(enc[:-1], 'ignore')
               elif enc.endswith('?'):
                   return value.decode(enc[:-1], 'replace')
               elif enc.endswith('&'):
                   return value.decode(enc[:-1], 'xmlcharrefreplace')
               elif enc.endswith('\\'):
                   return value.decode(enc[:-1], 'backslashreplace')
               else:
                   return value.decode(enc)
           except UnicodeError:
               pass
       else:
           raise
   
   def get_down_urls(url):
       response = requests.get(url, verify=False)
       response = Selector(text=make_unicode(response.text))
       alist = response.xpath('.//script')
       for href in alist:
           scripts_content = href.get()
           scripts_content = re.sub(r'[\n\t\r]', '', scripts_content)
           # print scripts_content
           if re.search('window.__init__', scripts_content):
               start_pos = scripts_content.index("{")
               end_pos = scripts_content.rindex("}")
               #print scripts_content[start_pos:end_pos + 1]
               json_str = json.loads(scripts_content[start_pos:end_pos + 1])
               cid = json_str['list'][0]['cid']
               title = json_str['title']
               print title
               aid = json_str['aid']
               print aid
   
               donwlaod_url = "https://www.kanbilibili.com/api/video/%s/download?cid=%s&quality=112&page=1" % (aid, cid)
               get_download_urls = requests.get(donwlaod_url)
               get_download_urls = json.loads(get_download_urls.text)
               durls = get_download_urls['data']['durl']
               for durl in durls:
                   down_url = durl['url']
                   down_order = durl['order']
                   print "down load url : %s " % down_url
   
   
   def Get_spalce_AVList(url):
       re_mid = re.search(r'\/([\d]+)\/',url)
       if re_mid:
           mid = re_mid.group(1)
       else:
           print "url error can find mid in url"
           return
       aidList = []
       page = 1
       while True:
           url_vlist = "https://space.bilibili.com/ajax/member/getSubmitVideos?mid=%s&pagesize=100&tid=0&page=%s&keyword=&order=pubdate" % (mid, page)
           #print url_vlist
           response = requests.get(url_vlist, verify=False)
           vlist = json.loads(response.text)
           for info in vlist['data']['vlist']:
               aidList.append(info['aid'])
               #print info['aid']
           if vlist['data']['pages'] > page:
               page = page + 1
           else:
               break
       for aid in aidList:
           url_downLoad = "https://www.kanbilibili.com/video/av%s" % aid
           print url_downLoad
           get_down_urls(url_downLoad)
   
   if __name__ == "__main__":
       space_url = "https://space.bilibili.com/97678687/#/video?tid=0&page=1&keyword=&order=pubdate"
       Get_spalce_AVList(space_url)

5. 结果
   将下载链接复制到迅雷中进行批量下载就可以了。