Request Payload参数(字典，json，列表，字符串，以及DWR框架的参数)---持续更新

郑德帅

已于 2022-02-16 17:01:52 修改

阅读量2.3w

点赞数 2

分类专栏： Python 文章标签：爬虫

于 2019-03-21 17:08:43 首次发布

本文链接：https://blog.csdn.net/qq_36853469/article/details/88720102

版权

Python 专栏收录该内容

43 篇文章 9 订阅

订阅专栏

1.正常的Json数据

常规提交POST数据
当然了，这样写和下面的提交的原理是一样的：

2.Request Payload是正常json数据

什么是Request Payload ？
也是Json数据的一种。但是也稍有不同，因为Request Payload 也可以是列表。具体的不涉及前后端的话知道这点就够了。

json数据
那么发请求的话
在这里插入图片描述
如果发送失败的话，记得添加method=“POST”

这样就OK了

3.Request Payload 是一个列表

情形一  注意：这种方式的话是一定需要设置请求头的。

在这里插入图片描述
这种情况我也是第一次见，且使用Scrapy框架时利用自带的FormReuqest 是无法完成要求的

转而利用requests模块，直接将内容转换为字符串提交，如图

在这里插入图片描述
这样就请求过来的。但是是直接调用的requests模块。

情形二：列表中有中文字
在这里插入图片描述
需要先将中文字编译成UTF-8格式

print("昆明市".encode('utf-8'))

#注意请求头的第一个参数一定要有
headers = {
        "ajax-method": "AjaxMethodFactory",
        "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36"
}

data = '["TrueLore.Web.WebUI.WebAjaxService","GetPageZBGGByCCGC",[0,15,"\xe6\x98\x86\xe6\x98\x8e\xe5\xb8\x82","11,12,13,14,15,16,17,18,19,20","1","0","JSGC",1,"","","BDMCGGBT",""],null,null]1574150259770'

proxies = random.choice([
    {"HTTP": "http://代理"},
    ])
res = requests.post("https://www.kmggzy.com/TrueLoreAjax/TrueLore.Web.WebUI.AjaxHelper,TrueLore.Web.WebUI.ashx",data = data,headers=headers,verify=False,proxies=proxies)

到此成功请求

4.Java DWR异步框架参数提交

DWR框架的参数列表
提交方式:
在这里插入图片描述
将参数提交的POST请求即可将数据获取过来，注意不要讲提交参数的格式改变了

5.参数及请求头

{"token":"","pn":20,"rn":10,"sdt":"","edt":"","wd":"","inc_wd":"","exc_wd":"","fields":"title","cnum":"001;002;003;004;005;006;007;008;009;010","sort":"{\"showdate\":\"0\"}","ssort":"title","cl":200,"terminal":"","condition":[{"fieldName":"categorynum","isLike":true,"likeType":2,"equal":"001001"}],"time":null,"highlights":"title","statistics":null,"unionCondition":null,"accuracy":"100","noParticiple":"0","searchRange":null,"isBusiness":1}:

Content-Type: application/x-www-form-urlencoded; charset=UTF-8

解决方法:
将参数构成dict

form_data = {
                 "token": "", "pn": str(page*10), "rn": "10", "sdt": "", "edt": "", "wd": "", "inc_wd": "", "exc_wd": "",
                 "fields": "title", "cnum": "001;002;003;004;005;006;007;008;009;010", "sort": "{\"showdate\":\"0\"}",
                 "ssort": "title", "cl": "200", "terminal": "",
                 "condition": '[{"fieldName": "categorynum", "isLike": "true", "likeType": "2", "equal": "001001"}]',
                 "time": 'null', "highlights": "title", "statistics": 'null', "unionCondition": 'null',
                 "accuracy": "100",
                 "noParticiple": "0", "searchRange": 'null', "isBusiness": "1"
                 }

json.dumps提交，成功获取数据
2022-02-16,最新版本：
将null替换成None,true替换成True,false替换成False

{"token": "", "pn": 0, "rn": 10, "sdt": "", "edt": "", "wd": "", "inc_wd": "", "exc_wd": "",
        "fields": "titlenew", "cnum": "005", "sort": "{\"webdate\":0}", "ssort": "title", "cl": 200, "terminal": "",
        "condition": [{"fieldName": "categorynum", "equal": "002001003", "notEqual": None, "equalList": None,
                       "notEqualList": None, "isLike": True, "likeType": 2}], "time": None, "highlights": "",
        "statistics": None, "unionCondition": None, "accuracy": "", "noParticiple": "0", "searchRange": None,
        "isBusiness": "1"}

在这里插入图片描述

6.参数及请求头

在这里插入图片描述

{"noticeTitle":"","effectTime":"","failureTime":"","noticeType":"null","purchaseNoticeType":"2","resultsNoticeType":"","level":"","provinceInput":"","cityInput":"","pageIndex":1,"pageSize":10,"sortField":"","sortOrder":"","page":{"begin":10,"length":10}}

提交方式

data1 = '''{"noticeTitle":"","effectTime":"","failureTime":"","noticeType":"null","purchaseNoticeType":"2","resultsNoticeType":"","level":"","provinceInput":"","cityInput":"","pageIndex":1,"pageSize":10,"sortField":"","sortOrder":"","page":{"begin":10,"length":10}}'''

requests提交

res = requests.post("http://www.tower.com.cn/default/main/index/cn.chinatowercom.obp.main.index.obphomepage.queryNoticeDetails.biz.ext",data = json.dumps(data),headers=headers,verify=False,proxies=proxies)

scrapy提交

 form_data = '''{"noticeTitle":"","effectTime":"","failureTime":"","noticeType":"null","purchaseNoticeType":"2","resultsNoticeType":"","level":"","provinceInput":"","cityInput":"","pageIndex":1,"pageSize":10,"sortField":"","sortOrder":"","page":{"begin":10,"length":10}}'''

full_url = "http://www.tower.com.cn/default/main/index/cn.chinatowercom.obp.main.index.obphomepage.queryNoticeDetails.biz.ext"

yield scrapy.Request(full_url, callback=self.parse,body=json.dumps(form_data),method="POST",dont_filter=True)