python websocket爬虫_python爬虫----爬取阿里数据银行websocket接口

业务需求:爬取阿里品牌数据银行的自定义模块==>>>人群透视==>>>查看报告==>>数据

最终获取页面:

1921160-20200227172958954-211008069.png

页面获取情况如下:

1921160-20200227172008990-2137945302.png

绿色的就是我们需要模拟的请求,红色朝下的就是请求对应的数据,通过rid参数来找寻对应请求和数据,这个rid和时间戳很类似,没错这就是一个13位的时间戳,和随机数组合而成的:

randomID = str(int(time.time()*1000))+str(self.count).zfill(3)

观察发送的请求:

message = {

"method": "/iSheetCrowdService/report",

"headers": {"rid": random_id, "type": "PULL"},

"body": {

"args": {

"id": "286", "condition":"{\"compute\":\"INTERSECT\",\"ruleList\":[{\"filterList\":[{\"key\":\"brand\",\"type\":\"SINGLE_VALUE\",\"value\":\"372718624\"},{\"key\":\"ds\",\"type\":\"SINGLE_VALUE\",\"value\":\"20200224\"},{\"key\":\"stage\",\"type\":\"MULTI_VALUE\",\"value\":[\"1010\",\"1020\",\"1030\",\"1040\"]}],\"name\":\"stage\"},{\"type\":\"crowd\",\"value\":[66466812]}]}", "tags": ["mobile_brand_name_prefer"], "bizParam": {   "databankCrowdId": "50104182", "bizType": "CUSTOM_ANALYSIS", "tag_identifier": "all", "captcha":'%7B%22a%22%3A%22TSF2%22%2C%22c%22%3A%221582700342424%3A0.23973179491554664%22%2C%22d%22%3A%22nvc_register%22%2C%22h%22%3A%7B%22key1%22%3A%22code0%22%2C%22nvcCode%22%3A400%2C%22umidToken%22%3A%22T08C67EE3AD81E11A23D01F0EE4CA1134D6022447F84D66B6623678D5FE%22%7D%2C%22j%22%3A%7B%22test%22%3A1%7D%2C%22b%22%3A%22122%23i5kRCD9eEE%2BqAJpZy4pjEJponDJE7SNEEP7ZpJRBuDPpJFQLpCGwoHZDpJEL7SwBEyGZpJLlu4Ep%2BFQLpoGUEELWn4yE7SNEEP7ZpERBuDPE%2BBQPpC76EJponDJLKMQEspPA04nTtBOmKBvALOESIAOsJhoR8HJNZIbVrA4tlm9BE5R8XAJh3Ue7sqj97bm98oL6JXWpO%2F%2BDqMfEl8WxXWplul5EELXZ8CL6JNbERF3mqM3okBTlpM%2B1ul9rDLVZ8CRfJ4bEyF3mqW32E5pangLlul5EELXZ8oL6JNEEyB3DqMfbDEpxnSp1tP0EDLf%2F8om6J4EqRoBHqevfAIAKkS7gz4MzTChsGRkP%2BeqxYlg3ypGmtolPBVsN6ovcGQYuNVCPWaTtjBbU5X0QWvFMSFzE%2FaVEWBKYt4%2FavytijGp1%2FQCeg%2F2SfxjZw8fIlfio42QXEpWhdEspy4fBSEZgHrMRhwhnBIFLuxd8hTc7Y99x%2BfF2UuU5oOXLG0an0CHvFQqCYMqZXb8myhlZHuHOXI0GZ%2BPvFxGzVjTf4AmL63HeSjbSP8L6CSCuq47zX7tPDBWS%2FhdLqwO8dtOkEXes248plPRkVf5gVlcqsHSdYxDkjT%2FpfSszn8vXlIUg3GRKm9eMWi09PxekLM2tUu0nivXzonmbHdGHLsqhNe%2FbdjoKHYG6ygnnx3aZ9DU9ugUWmZgB9Ztbv1BYo%2FhPlYLykr4j14BCvVlwMUtgZbK%2BpgNu7vKERSRkRosaHoNotkt%2B%2BToClNeIRM%2Fk7vQm1x0YbZT3hzfU9k5kadIafvosIReZwiQhd4%2B0sYXOjxvChWtv2%2FSMKb9fIeSKsALCP%2FNshczJBF5y1TMo4YIPh7%2BaMuMfcUqmMC%2BWrr1Xm%2FUAjtyHnttlwQlkGDRPsSS4DMlM0OdWYLD9vL0ekEm7iz566ESLHP2aykbJ%2F3id3DwjcdgDENA6%2F8oojlvm6WP0JBDUviVTeDPK9V5RFekM3drtuDFit2UwRc%2B09xUvKcueMc%2FPrKKRhC%2FuRLXdx0WzgP4%2F2RJJeZhuKEQhYDTsFiDZnArDQMQyiMN0hRuwdfalhZFe27jXUG4Y%2BwPnHvpj1OXxEJ1VOgEtBzOO1AgwUr5SYa6UKhrbynm2X1J1HRyEn%2FVUqkvgc9Rx8ZYG2GqRr4L7eQ1N4sa7S6oqKEND6fX3eQhxFaqnimVCIg%2B6TwTp9Ant4P15WlGdVueq6HWATk8zB1CglVmsVH08lUwArNKjBpArD7v%2BIxx5VP1hJLChZOgfHqa6MVh7fP0lg6HXLeyMLSOXCz3oAK7iSOFkfu6RAx%2BGrBAj50ha%22%2C%22e%22%3A%22HELtCcggjijySBif5QK5Flm60fyrLzjVSvY2GZ7kF9k2ufvery5t6e1OxIGoUHpc7a0IbkE_0FA-F5WgiEpV7aeWlyQyxr1LL83v6PCoc3YbWdFNpRGiSow97HJFmhSolqL2iP8Yg3b6GvpNCl1IVN3_kiy7mdt7qA7PsE2Fu9J1ZID-lo1BWsvQpV6riLNbYizM9JlKkpiqJYbEB2zQGA%22%7D'}, "insightType": 0, "interaction": 'false', "rateParam": {}, "appId": "208"}}}

其中关注几个变化的参数:databankCrowdId参数表示人群ID,condition参数含义未知可能与标签有关,可从https://databank.tmall.com/api/ecapi返回的数据得到;captcha则是先需要URL解码得到是字典格式的数据

{"a":"TSF2",

"c":"1582700342424:0.23973179491554664",

"d":"nvc_register",

"h":{"key1":"code0","nvcCode":400,"umidToken":"T08C67EE3AD81E11A23D01F0EE4CA1134D6022447F84D66B6623678D5FE"}, "j":{"test":1},

"b":"122#i5kRCD9eEE+qAJpZy4pjEJponDJE7SNEEP7ZpJRBuDPpJFQLpCGwoH

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值