淘宝评论爬取（2020-08-21）update 2020-09-18

爬取淘宝产品评论及分析

需求

爬取淘宝天猫店，某个单品的评论，根据评论内容生成词云，以及对评论做情感分析

分析

1.登录淘宝。找到某个产品，点击评论。
2.进入评论页面后，按F12调出开发者工具，在Network 找list_det。打开后点击Preview。可以找到评论数据，数据类型是json。一页评论20条（怎么判断是ajax异步加载？个人判断是点击第二页的时，ie中的url没有改变从而推测是ajax异步加载。）

3. 构建ajax异步加载分析：
   ajax请问url 就是红色框的Request URL ；用的是get 请求。可以用request库中request.get()。
   分析url内容：Request URL:
   https://rate.tmall.com/list_detail_rate.htm?itemId=576746298719&spuId=1042574054&sellerId=2328901391&order=3&currentPage=1&append=0&content=1&tagId=&posi=&picture=&groupId=&ua=098%23E1hvAQvEvbQvUvCkvvvvvjiPnLSytj1bPFLUsjnEPmPwgjECRsMyQjtWPsMZQjEV3QhvCvvhvvvEvpCWvPiIvvwdD7zwaXTAVA1ldExrVut%2BCNLy0bvqrADn9W2%2BFfmtEpch1RFE%2BFIlBqevrf9gKFnfIWFvgEkXw6cv6R29DCOrvGjxs4hZebyCvm9vvvvvphvvvvvv9X1vpvFjvvmmZhCv2CUvvUEpphvWOpvv9DCvpv13kphvC99vvOCgoTyCvv9vvUvTZhOHSQhCvvOvCvvvphvPvpvhvv2MMsyCvvpvvvvviQhvCvvv9UU%3D&needFold=0&_ksTS=1597997731033_540&callback=jsonp541

   url构成：
   itemId: 576746298719
   spuId: 1042574054
   sellerId: 2328901391
   order: 3
   currentPage: 1
   append: 0
   content: 1
   tagId:
   posi:
   picture:
   groupId:
   ua: 098#E1hvAQvEvbQvUvCkvvvvvjiPnLSytj1bPFLUsjnEPmPwgjECRsMyQjtWPsMZQjEV3QhvCvvhvvvEvpCWvPiIvvwdD7zwaXTAVA1ldExrVut+CNLy0bvqrADn9W2+FfmtEpch1RFE+FIlBqevrf9gKFnfIWFvgEkXw6cv6R29DCOrvGjxs4hZebyCvm9vvvvvphvvvvvv9X1vpvFjvvmmZhCv2CUvvUEpphvWOpvv9DCvpv13kphvC99vvOCgoTyCvv9vvUvTZhOHSQhCvvOvCvvvphvPvpvhvv2MMsyCvvpvvvvviQhvCvvv9UU=
   needFold: 0
   _ksTS: 1597997731033_540
   callback: jsonp541
   
   解释：itemId:商品ID，spuId：产品id，sellerId：卖家ID，currentPage:评论页数，ua: 用户名（这个是淘宝有它自己的算法，动态的），_ksTS:时间戳，callback：不知道什么鬼。。。

继续分析url构成及请求：

第一页url

Request URL:
https://rate.tmall.com/list_detail_rate.htm?itemId=576746298719&spuId=1042574054&sellerId=2328901391&order=3&currentPage=1&append=0&content=1&tagId=&posi=&picture=&groupId=&ua=098%23E1hvAQvEvbQvUvCkvvvvvjiPnLSytj1bPFLUsjnEPmPwgjECRsMyQjtWPsMZQjEV3QhvCvvhvvvEvpCWvPiIvvwdD7zwaXTAVA1ldExrVut%2BCNLy0bvqrADn9W2%2BFfmtEpch1RFE%2BFIlBqevrf9gKFnfIWFvgEkXw6cv6R29DCOrvGjxs4hZebyCvm9vvvvvphvvvvvv9X1vpvFjvvmmZhCv2CUvvUEpphvWOpvv9DCvpv1