通常情况下用selenium开发爬虫时,会使用以下方式添加UA
chrome_options.add_argument('user-agent=' + self.ua)
但是每次更换需要重启浏览器,这会造成资源浪费。好在selenium支持CDP,于是翻了文档终于找到了解决方法。解决方法如下:
driver.execute_cdp_cmd("Emulation.setUserAgentOverride", {
"userAgent": "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20100101 Firefox/6.0"
})
执行后打开控制台,输入navigator.userAgent查看,会发现userAgent已经改变,访问任何页面都会是目前设置的这个userAgent
同时官方文档中还有以下参数可以使用:
{
"name": "setUserAgentOverride",
"description": "Allows overriding user agent with the given string.",
"redirect": "Emulation",
"parameters": [
{
"name": "userAgent",
"description": "User agent to use.",
"type": "string"
},
{
"name": "acceptLanguage",
"description": "Browser langugage to emulate.",
"optional": true,
"type": "string"
},
{
"name": "platform",
"description": "The platform navigator.platform should return.",
"optional": true,
"type": "string"
},
{
"name": "userAgentMetadata",
"description": "To be sent in Sec-CH-UA-* headers and returned in navigator.userAgentData",
"experimental": true,
"optional": true,
"$ref": "Emulation.UserAgentMetadata"
}
]
}