一、安装
环境:Ubuntu18.04
1)安装docker
2)下拉splash镜像
sudo docker pull scrapinghub/splash
3) 启动
sudo docker run -p 8050:8050 scrapinghub/splash
4)访问http://服务器ip:8050/验证
二、基本使用
import requests
def splash_render(url):
splash_url = "http://服务器ip:8050/render.html"
args = {
"url": url,
"timeout": 10,
"image": 0,
}
response = requests.get(splash_url, params=args)
return response.text
if __name__ == '__main__':
url = "http://quotes.toscrape.com/js/"
html = splash_render(url)
with open("test.html", "w", encoding="utf-8") as w:
w.write(html)