mitmproxy python 使用案例

远方的飞猪

已于 2023-07-30 14:26:36 修改

阅读量1.2k

点赞数

分类专栏：爬虫 Python 文章标签： python

于 2020-09-20 11:04:56 首次发布

本文链接：https://blog.csdn.net/tanjunchen/article/details/108675890

版权

爬虫同时被 2 个专栏收录

22 篇文章 4 订阅

订阅专栏

Python

21 篇文章 1 订阅

订阅专栏

python  mitmproxy 使用案例

完整代码下载： https://github.com/tanjunchen/SpiderProject/tree/master/mitmproxyspider

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from mitmproxy import ctx, flow


def job(url):
    injected_javascript = '''
    // overwrite the `languages` property to use a custom getter
    Object.defineProperty(navigator, "languages", {
      get: function() {
        return ["zh-CN","zh","zh-TW","en-US","en"];
      }
    });
    // Overwrite the `plugins` property to use a custom getter.
    Object.defineProperty(navigator, 'plugins', {
      get: () => [1, 2, 3, 4, 5],
    });
    // Pass the Webdriver test
    Object.defineProperty(navigator, 'webdriver', {
      get: () => false,
    });
    // Pass the Chrome Test.
    // We can mock this in as much depth as we need for the test.
    window.navigator.chrome = {
      runtime: {},
      // etc.
    };
    // Pass the Permissions Test.
    const originalQuery = window.navigator.permissions.query;
    window.navigator.permissions.query = (parameters) => (
      parameters.name === 'notifications' ?
        Promise.resolve({ state: Notification.permission }) :
        originalQuery(parameters)
    );
    '''
    # Only process 200 responses of HTML content.
    if not flow.response.status_code == 200:
        return
    # Inject a script tag containing the JavaScript.
    html = flow.response.text
    html = html.replace('<head>', '<head><script>%s</script>' % injected_javascript)
    flow.response.text = str(html)
    ctx.log.info('>>>> js代码插入成功 <<<<')

    # 只要url链接以target开头，则将网页内容替换为目前网址
    if flow.url.startswith(url):
        flow.response.text = flow.url
        print(flow.url)
    print(url)


if __name__ == '__main__':
    target = 'https://passport.zcool.com.cn/regPhone.do?appId=1006&cback=https://my.zcool.com.cn/focus/activity'
    job(target)