Scrapy 发送 post 请求

又逢乱世

于 2024-08-17 19:32:58 发布

阅读量39

点赞数 1

分类专栏：爬虫文章标签： scrapy

本文链接：https://blog.csdn.net/a1053765496/article/details/141285372

版权

爬虫专栏收录该内容

10 篇文章 0 订阅

订阅专栏

发送 post 请求，使用 scrapy.FormRequest 方法

# 发送post请求
yield scrapy.FormRequest(
    # url
    url=post_url,
    # post接口参数
    formdata=post_data,
    # 请求头
    headers=self.headers,
    # cookie
    cookies=self.cookies,
    # 回调方法
    callback=self.after_reply
)

示例：回复贴吧信息

import scrapy


class TeiBaSpider(scrapy.Spider):
    name = "teiba"
    allowed_domains = ["tieba.baidu.com"]
    start_urls = ["https://tieba.baidu.com/p/91xxxxxxx"]

    cookies_str = 'XFI=720862e0-5c86-11ef-90cf-edada47cd573; XFCS=A534DA9021023FD1141B9DA6E929053811C874F; XFT=nZmiPbxW66wPyRLQUCjslM1POx8BU3zu/EpEbunMvbA=; BIDUPSID=CC4827F65EB646623A40EA9'

    cookies = {data.split('=')[0]: data.split('=')[-1] for data in cookies_str.split("; ")}

    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36',
        'Accept-Language': 'zh-CN,zh;q=0.9',
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
        'Cache-Control': 'no-cache',
        'Connection': 'keep-alive',
        'Host': 'tieba.baidu.com'
    }

    def start_requests(self):
        for url in self.start_urls:
            yield scrapy.Request(url, headers=self.headers, callback=self.parse, cookies=self.cookies)

    def parse(self, response):
        # 添加评论的post url
        post_url = "https://tieba.baidu.com/f/commit/post/add"

        # post 请求参数
        post_data = {
            "content": "我是回复内容123456",
            "ie": "utf-8",
            "kw": "沙井",
            "fid": "335358",
            "floor_num": "38",
            "rich_text": "1",
            "basilisk": "1",
            "mouse_pwd_isclick": "1",
            "ev": "comment",
            "__type__": "reply",
            "geetest_success": "0"
        }

        # 发送post请求
        yield scrapy.FormRequest(
            # url
            url=post_url,
            # post接口参数
            formdata=post_data,
            # 请求头
            headers=self.headers,
            # cookie
            cookies=self.cookies,
            # 回调方法
            callback=self.after_reply
        )

    def after_reply(self, response):
        print(response.status)