视频 http下载代码 dn.go(注意:代码很ugly,没怎么花时间)
总体感觉特别简单,网上看了下 net/http ,io这2个库的使用, 几分钟就写完了,感觉cpp 在做工具这块 开发效率的确差太多(没有轮子的话)。
再放一个 go example 网站,https://gobyexample.com/
package main import ( "fmt" "io" "net/http" "os" "strconv" ) /* var ( //url = "http://flv5.bn.netease.com/live163/store/208588/serverpush_18783_1521024752481_208588_0-1.ts" url string = "http://flv5.bn.netease.com/live163/store/208588/serverpush_18783_1521024752481_208588_0-" url_suffix string = ".ts" ) */ func main() { fmt.Println("download begin.....") url := "http://flv5.bn.netease.com/live163/store/208588/serverpush_18783_1521024752481_208588_0-" url_suffix := ".ts" for i := 180; i <= 180; i++ { str_index := strconv.Itoa(i) // 通过Itoa方法转换 //str2 := fmt.Sprintf("%d", i) // 通过Sprintf方法转换 //fmt.Println(str_index) // 打印str1 strFinal := url+str_index+ url_suffix res, err := http.Get(strFinal) if err != nil { panic(err) } f, err := os.Create(str_index+url_suffix) if err != nil { panic(err) } fileSize,writeErr := io.Copy(f, res.Body) fmt.Println(strFinal + " download done,", "file size(byte)=", fileSize) if writeErr != nil { fmt.Println(strFinal + " download failed ", "errorInfo=", writeErr.Error()) panic(err) } } fmt.Println("download finish.") }
-------------------
因为有些m3u8不是有规则的,
所以又写了新版本 ,按行 读取m3u8,逐行读取 ts文件(ffmpe concat 合并文件,所以还要写 merge.txt)
ffmpeg文件合并
merge.txt 格式
```txt
file 1.ts
file 2.ts
...
file 100.ts
```
> .\ffmpeg.exe -y -f concat -safe -1 -i .\record.m3u8-merge.txt -c copy -bsf:a aac_adtstoasc dst.mp4
代码
package main import ( "fmt" "io" "net/http" "os" //"strconv" "bufio" "strings" "os/exec" "log" ) func MergeVideo(mergeFile, videoName string) int{ var nRetsult int nRetsult=0 fmt.Printf("Merging video parts into %s\n", videoName) cmd := exec.Command("ffmpeg", "-y", "-f", "concat", "-safe", "-1", "-i", mergeFile, "-c", "copy", "-bsf:a", "aac_adtstoasc", videoName, ) err := cmd.Run() if err != nil { log.Fatal(err) } nRetsult=1 return nRetsult } func downloadByM3u8File(strM3u8FilePath string) int{ var nRetsult int nRetsult=0 var strSlices []string var strFilename string M3u8file, err := os.Open(strM3u8FilePath)//打开文件 //check m3u8 Is Exist //create ffmpeg merge file mergeFile := strM3u8FilePath + "-merge.txt" file, _ := os.Create(mergeFile) //if exist ,truncate defer M3u8file.Close() //打开文件出错处理 if nil == err { buff := bufio.NewReader(M3u8file) //读入缓存 for { //line, isPrefix,err := buff.ReadLine() httpAddress, err := buff.ReadString('\n') //以'\n'为结束符读入一行 if err != nil || io.EOF == err { break } //if(strings.Contains(httpAddress, ".ts")){ if(strings.Contains(httpAddress, "http")){ httpAddress = strings.TrimSpace(httpAddress) httpAddress = strings.TrimRight(httpAddress,"\n") strSlices = strings.Split(httpAddress,"/") //分离成多个 slices strFilename = strSlices[len(strSlices)-1] //取最后 的文件名 //write ffmpeg merge file file.Write([]byte(fmt.Sprintf("file %s\n", strFilename))) res, err := http.Get(httpAddress) //获取失败? if err != nil { panic(err) } TsFile, err := os.Create(strFilename) if err != nil { panic(err) } fileSize,writeErr := io.Copy(TsFile, res.Body) TsFile.Close() fmt.Println(strFilename + " download done,", "file size(byte)=", fileSize) if writeErr != nil { fmt.Println(strFilename + " download failed ", "errorInfo=", writeErr.Error()) panic(err) } //此行有ts文件,应该下载 nRetsult++ fmt.Print(httpAddress) //可以对一行进行处理 } //fmt.Print(line) //可以对一行进行处理 } }else{ fmt.Println(strM3u8FilePath , " file do not exist") } file.Close() return nRetsult } /* // M3u8URLs get all urls from m3u8 url func M3u8URLs(uri string) []string { html := request.Get(uri) lines := strings.Split(html, "\n") var urls []string for _, line := range lines { line = strings.TrimSpace(line) if line != "" && !strings.HasPrefix(line, "#") { if strings.HasPrefix(line, "http") { urls = append(urls, line) } else { base, _ := url.Parse(uri) u, _ := url.Parse(line) urls = append(urls, fmt.Sprintf("%s", base.ResolveReference(u))) } } } return urls } */ func main() { //要设置launch.json的 argv 才能debug 用参数 if( len(os.Args)!= 2 ){ fmt.Println("参数不正确,请确认输入m3u8文件。 格式如: 1.exe a.mau8") return } /* argsWithProg := os.Args argsWithoutProg := os.Args[1:] */ arg := os.Args[1] //第二个参数 fmt.Println(arg) //还需要 判断输入的参数个数, fmt.Println("download begin.....") nRet := downloadByM3u8File(arg) fmt.Println("已下载共 %d 个ts文件",nRet) //MergeVideo("1.mp4","record.m3u8-merge.txt") fmt.Println("download finish.") }
1.代码可以用格式处理下,
2.代码比较基础,没时间写异常, 简单能用就行。
附: 一些网站如何获取m3u8 文件
1> 学堂在线( 用flash 播放器)
1.打开网站http://www.xuetangx.com/event/cvpr2017 F12查看html代码 <div class="video_container"> <div class="wrap"> <div class="title" id="videoTitle">微软亚洲研究院创研论坛 CVPR 2017 论文分享会</div> <div class="video_wrap cf" id="video_container"><iframe src="http://cloud.quklive.com/cloud/a/embed/1497351554750965" allowfullscreen="" width="720" height="460" frameborder="0"></iframe></div> <div class="video_function cf"> <ul class="video_function_btn fr cf"> <li><a href="javascript:" id="infoShare"><span class="icon_share"></span><span>分享到...</span></a></li> </ul> </div> </div> </div> 注意: "video_container" div 中有 <iframe src="http://cloud.quklive.com/cloud/a/embed/1497351554750965" http://cloud.quklive.com/cloud/a/embed/1497351554750965就是码流地址,但是 是js加密的, 2. 打开 http://cloud.quklive.com/cloud/a/embed/1497351554750965 F12查看js 代码 在 function onJSBridge(playerId, event, data) { switch(event) { case "onJavaScriptBridgeCreated": case "ready": DM.watching("1497351554750965","http://recordcdn.quklive.com/broadcast/activity/1497351554750965/record.m3u8","record"); break; default:; break; } } 这个就是我们的码流 文件了, 用http下载把上面TS文件下载 下来,再用ffmpeg 合并就可以了。
2>上直播(HTML5)
http://shangzhibo.tv/watch/5833145
网站html 直接查找m3u8(当然用video, player src,等关键字 搜索)
https://shangzhibo-img.b0.upaiyun.com/client/user/100994/1526289188264/1526289188242_Session1GANandSynthesis-processed.m3u8
3>网易云音乐(HTML5)
网易云直播
http://live.163.com/room/173789.html
视频解析,可以得到 flv5.bn.netease.com/live163/store/208588/serverpush_18783_1521024752481_208588_208588_20180314185233_20180314202528_0.m3u8
4> 抖音 (HTML5)
5> bilibili
再附上一个github 项目,里面有一些其他网站 视频下载方式
https://github.com/iawia002/annie
Supported Sites
Site | URL | ? Videos | ? Images | ? Playlist | ? VIP adaptation |
---|---|---|---|---|---|
抖音 | https://www.douyin.com | ✓ | |||
哔哩哔哩 | https://www.bilibili.com | ✓ | ✓ | ✓ | |
半次元 | https://bcy.net | ✓ | |||
pixivision | https://www.pixivision.net | ✓ | |||
优酷 | https://www.youku.com | ✓ | ✓ | ||
YouTube | https://www.youtube.com | ✓ | ✓ | ||
爱奇艺 | https://www.iqiyi.com | ✓ | |||
芒果TV | https://www.mgtv.com | ✓ | |||
Tumblr | https://www.tumblr.com | ✓ | ✓ | ||
Vimeo | https://vimeo.com | ✓ | |||
https://facebook.com | ✓ | ||||
斗鱼视频 | https://v.douyu.com | ✓ | |||
秒拍 | https://www.miaopai.com | ✓ | |||
微博 | https://weibo.com | ✓ | |||
https://www.instagram.com | ✓ | ✓ | |||
https://twitter.com | ✓ |