Puppeteer入门

最新推荐文章于 2024-05-08 11:00:29 发布

-夏末微凉-

最新推荐文章于 2024-05-08 11:00:29 发布

阅读量329

点赞数

分类专栏： headless 文章标签： chrome nodejs js proxy

本文链接：https://blog.csdn.net/qq_38000422/article/details/116674925

版权

headless 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

简介

Puppeteer是 GoogleChrome 团队官方的无界面(Headless)Chrome工具
它是一个NodeJs库
提供了高级的 API 来控制基于Chrome Debug Protocols(简称 CDP)协议上的Headless Chrome
可以用于配置控制正常的Chrome
本文主要添加一些使用示例

安装

安装chrome

不再详细描述，下载安装即可。启动headless，参考链接

chrome \
  --headless \                   # Runs Chrome in headless mode.
  --disable-gpu \                # Temporarily needed if running on Windows.
  --remote-debugging-port=9222 \
  https://www.chromestatus.com   # URL to open. Defaults to about:blank.

安装Puppeteer

npm安装，详情参考链接

$ npm i --save puppeteer

示例代码

网页源码

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto('https://www.toutiao.com', { waitUntil: 'networkidle2' });
    const pageContent = await page.content()
    console.log(pageContent)
    await page.close()
    await page.deleteCookie()
    await browser.close();
})();

网页截图

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    page.setViewport({ width: 1920, height: 2160})
    await page.goto('https://www.toutiao.com', { waitUntil: 'networkidle2' });
    await page.screenshot({
        path:'toutiao.png'
    })
    await page.close()
    await page.deleteCookie()
    await browser.close();
})();

代理及认证

参考文章

const puppeteer = require('puppeteer');

(async () => {
    const proxyServer = 'http://proxy_ip:proxy_port'
    const browser = await puppeteer.launch({
        headless:false,
        args: [`--proxy-server=${proxyServer}`]
    });

    const page = await browser.newPage();
    await page.authenticate({ username: 'username', password: 'password' })
    await page.goto('https://www.toutiao.com/', { waitUntil: 'networkidle2' });
    const pageContent = await page.content()
    console.log(pageContent)
    await page.close()
    await page.deleteCookie()
    await browser.close();
})();

设置代理

正常代理

const puppeteer = require('puppeteer');

(async () => {
    const proxyServer = 'http://proxy_ip:proxy_port'
    const browser = await puppeteer.launch({
        headless: false,
        args: [`--proxy-server=${proxyServer}`]
    });
})();

含有账密的代理

参考[示例代码 - 代理及认证]

PAC代理

const puppeteer = require('puppeteer');

(async () => {
    const pacProxyFile = 'file location'
    const browser = await puppeteer.launch({
        headless: false,
        args: [`google-chrome --proxy-pac-url=${pacProxyFile}`]
    });
    const page = await browser.newPage();
    await page.authenticate({ username: 'username', password: 'password' })
})();