京东生鲜评论数据获取

于 2024-01-26 17:36:21 发布
阅读量309 收藏 1
点赞数 2
分类专栏： 各类网站爬取 爬虫记录 文章标签： python 爬虫 json
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_52561726/article/details/135871511
版权

牛肉评论

# https://club.jd.com/comment/productPageComments.action?callback=fetchJSON_comment98&productId=100025128936&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0&fold=1
import re

import requests
import csv
import random
import time
import re

headers={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36',
    'Cookie':'unpl=JF8EAK1nNSttWRgGARMGSBIXGFhcW1pcGERWZ2FWXQ8ITgMBHAQcEkN7XlVdXhRKHx9ubhRVXVNKXQ4aCysSEXteXVdZDEsWC2tXVgQFDQ8VXURJQlZAFDNVCV9dSRZRZjJWBFtdT1xWSAYYRRMfDlAKDlhCR1FpMjVkXlh7VAQrAh0TEU5cU1tbOHsQM19XAlFaXEtVNRoyGiJSHwFdXVUKTBROb2EEVVhZTFEDKwMrEQ; __jdu=170040647; areaId=13; PCSYCityID=CN_370000_370900_0; shshshfpa=d13cd1dc-5642-0f9b-e976-ec229feb579d-1662570491; shshshfpb=qzRhxau6Qqa2HQrqI5HSfpQ; __jdc=122270672; __jdv=122270672|baidu|-|organic|notset|1662570520701; shshshfp=2aff9b962254e92011938f034976e9b4; ip_cityCode=1112; ipLoc-djd=13-1112-46665-46698; jwotest_product=99; mt_xid=V2_52007VwMUU1xdU10aTylbB24BQlcKUE5ZGk9NQAAzBxJODllUCQNKSglQNVNCBQpaVVMvShhfBnsCGk5cUUNbH0IZWg5lASJQbVhiUxlOHVUHYwQbUl1ZUVkdQRxZB1cDFFZZ; jsavif=1; __jda=122270672.170040647.1662570487.1662570521.1662614449.3; JSESSIONID=E4183D4825A3F25B79066C6FAB065E13.s1; token=631272b9b7166cb267e9a2f9907da026,2,923675; __tk=f37514b34c6ce7e350f7e33722bc6f58,2,923675; shshshsID=daa07f7ca60bb8ada946f160dad78a29_3_1662615826441; __jdb=122270672.3.170040647|3.1662614449; 3AB9D23F7A4B3C9B=3G7NRO6TRKPOSZN7QIQA75I36DUPPJUCM735CSVTFQHNTIPGFBRM42YWWAJU53IBGTMCBX5TUVJDGS5CWSS3RXD2YQ'
}
f = open('牛肉评论.csv',mode='a',encoding='gbk',newline='')
csv_writer = csv.writer(f)
csv_writer.writerow(['用户名称','用户头像','评分','评论内容','评论时间','点赞数','评论回复人数','商品属性','页面标题','评论照片链接','评论视频链接','sku','好评度小数表示','好评度百分比表示','全部评价数','视频晒单数','追评数','好评数','中评数','差评数','评价关键词'])

zs=['100009161505','100009235785','100016273168','100009235831','100018175162','100009307855','100010205682','100011848502','100011848488','100015445637','100015445663','2154645','3926460','100031452532','100031452478']
for i in zs:
    print(f"--------------------正在爬取{i}网址-----------------------")
    for page in range(0,100,1):
        print(f"-------------------正在爬取第{page}页----------------------")
        url=f'https://club.jd.com/comment/productPageComments.action?productId={i}&score=0&sortType=5&page={page}&pageSize=10&isShadowSku=0&fold=1'

        response = requests.get(url=url,headers=headers)

        # print(response.text)
        json_data = response.json()

        # print(json_data['comments'])

        comment_list = json_data['comments']
        for comment in comment_list:
            nickname = comment['nickname']  #用户名称
            userImageUrl = 'https://' + str(comment['userImageUrl']) #用户头像
            score = comment['score']  # 评分
            content = comment['content'] #评论内容
            content = re.sub(r'[\t\n]','',content)
            creationTime = comment['creationTime'] #评论时间
            usefulVoteCount = comment['usefulVoteCount']  #点赞数
            replyCount = comment['replyCount']  # 评论回复人数
            productColor = comment['productColor'] #商品属性
            productSize = comment['referenceName'] #页面标题
            try:
                images = comment['images']
                for zp in images:
                    imgUrl = 'https:' + str(zp['imgUrl']) #评论照片链接
                videos = comment['videos']
                for lj in videos:
                    # mainUrl = lj['mainUrl'] #评论照片链接
                    remark = lj['remark'] #评论视频链接
            except:
                pass
            productCommentSummary = json_data['productCommentSummary']
            productId = productCommentSummary['productId']  #sku
            goodRate = productCommentSummary['goodRate']  # 好评度小数表示
            goodRateShow = str(productCommentSummary['goodRateShow']) + '%'  # 好评度百分比表示
            commentCountStr = productCommentSummary['commentCountStr']  # 全部评价数
            videoCountStr = productCommentSummary['videoCountStr']  # 视频晒单数
            afterCountStr = productCommentSummary['afterCountStr']  # 追评数
            goodCountStr = productCommentSummary['goodCountStr']  # 好评数
            generalCountStr = productCommentSummary['generalCountStr']  # 中评数
            poorCountStr = productCommentSummary['poorCountStr']  # 差评数
            # print(nickname,userImageUrl,score,content,creationTime,usefulVoteCount,replyCount,productColor,productSize,imgUrl,remark)
            hotCommentTagStatistics_list = json_data['hotCommentTagStatistics']
            gjc=[]
            for hotCommentTagStatistics in hotCommentTagStatistics_list:
                # gjc=[]
                name = hotCommentTagStatistics['name'] #评价关键词
                count = hotCommentTagStatistics['count'] #评价关键词数
                guanjianci = str(name) + '(' + str(count) + ')' #评价关键词汇总
                gjc.append(guanjianci)
            # print(gjc)
            csv_writer.writerow(
                [nickname, userImageUrl, score, content, creationTime, usefulVoteCount, replyCount, productColor, productSize,
                 imgUrl, remark, productId, goodRate, goodRateShow, commentCountStr, videoCountStr, afterCountStr, goodCountStr,
                 generalCountStr, poorCountStr,str(gjc).replace('[','').replace(']','').replace("'",'').replace(',','')]
            )
        # print(productId,goodRate,goodRateShow,commentCountStr,videoCountStr,afterCountStr,goodCountStr,generalCountStr,poorCountStr)

        time.sleep(float(format(random.uniform(0, 2), '.2f')))

猪肉评论

# https://club.jd.com/comment/productPageComments.action?callback=fetchJSON_comment98&productId=100025128936&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0&fold=1
import re

import requests
import csv
import random
import time
import re

headers={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36',
    'Cookie':'unpl=JF8EAK1nNSttWRgGARMGSBIXGFhcW1pcGERWZ2FWXQ8ITgMBHAQcEkN7XlVdXhRKHx9ubhRVXVNKXQ4aCysSEXteXVdZDEsWC2tXVgQFDQ8VXURJQlZAFDNVCV9dSRZRZjJWBFtdT1xWSAYYRRMfDlAKDlhCR1FpMjVkXlh7VAQrAh0TEU5cU1tbOHsQM19XAlFaXEtVNRoyGiJSHwFdXVUKTBROb2EEVVhZTFEDKwMrEQ; __jdu=170040647; areaId=13; PCSYCityID=CN_370000_370900_0; shshshfpa=d13cd1dc-5642-0f9b-e976-ec229feb579d-1662570491; shshshfpb=qzRhxau6Qqa2HQrqI5HSfpQ; __jdc=122270672; __jdv=122270672|baidu|-|organic|notset|1662570520701; shshshfp=2aff9b962254e92011938f034976e9b4; ip_cityCode=1112; ipLoc-djd=13-1112-46665-46698; jwotest_product=99; mt_xid=V2_52007VwMUU1xdU10aTylbB24BQlcKUE5ZGk9NQAAzBxJODllUCQNKSglQNVNCBQpaVVMvShhfBnsCGk5cUUNbH0IZWg5lASJQbVhiUxlOHVUHYwQbUl1ZUVkdQRxZB1cDFFZZ; jsavif=1; __jda=122270672.170040647.1662570487.1662570521.1662614449.3; JSESSIONID=E4183D4825A3F25B79066C6FAB065E13.s1; token=631272b9b7166cb267e9a2f9907da026,2,923675; __tk=f37514b34c6ce7e350f7e33722bc6f58,2,923675; shshshsID=daa07f7ca60bb8ada946f160dad78a29_3_1662615826441; __jdb=122270672.3.170040647|3.1662614449; 3AB9D23F7A4B3C9B=3G7NRO6TRKPOSZN7QIQA75I36DUPPJUCM735CSVTFQHNTIPGFBRM42YWWAJU53IBGTMCBX5TUVJDGS5CWSS3RXD2YQ'
}
f = open('猪肉评论新.csv',mode='a',encoding='gbk',newline='')
csv_writer = csv.writer(f)
csv_writer.writerow(['用户名称','用户头像','评分','评论内容','评论时间','点赞数','评论回复人数','商品属性','页面标题','评论照片链接','评论视频链接','sku','好评度小数表示','好评度百分比表示','全部评价数','视频晒单数','追评数','好评数','中评数','差评数','评价关键词'])

zs=['100024004047','100024004049','100024004053','100024004055','68204751824','11597710970','100008118481','100008118487','100008118503','100008118501','100008118493','100008118511','100014591902','100014591908']
for i in zs:
    print(f"--------------------正在爬取{i}网址-----------------------")
    for page in range(0,100,1):
        print(f"-------------------正在爬取第{page}页----------------------")
        url=f'https://club.jd.com/comment/productPageComments.action?productId={i}&score=0&sortType=5&page={page}&pageSize=10&isShadowSku=0&fold=1'

        response = requests.get(url=url,headers=headers)

        # print(response.text)
        json_data = response.json()

        # print(json_data['comments'])

        comment_list = json_data['comments']
        for comment in comment_list:
            nickname = comment['nickname']  #用户名称
            userImageUrl = 'https://' + str(comment['userImageUrl']) #用户头像
            score = comment['score']  # 评分
            content = comment['content'] #评论内容
            content = re.sub(r'[\t\n]','',content)
            creationTime = comment['creationTime'] #评论时间
            usefulVoteCount = comment['usefulVoteCount']  #点赞数
            replyCount = comment['replyCount']  # 评论回复人数
            productColor = comment['productColor'] #商品属性
            productSize = comment['referenceName'] #页面标题
            try:
                images = comment['images']
                for zp in images:
                    imgUrl = 'https:' + str(zp['imgUrl']) #评论照片链接
                videos = comment['videos']
                for lj in videos:
                    # mainUrl = lj['mainUrl'] #评论照片链接
                    remark = lj['remark'] #评论视频链接
            except:
                pass
            productCommentSummary = json_data['productCommentSummary']
            productId = productCommentSummary['productId']  #sku
            goodRate = productCommentSummary['goodRate']  # 好评度小数表示
            goodRateShow = str(productCommentSummary['goodRateShow']) + '%'  # 好评度百分比表示
            commentCountStr = productCommentSummary['commentCountStr']  # 全部评价数
            videoCountStr = productCommentSummary['videoCountStr']  # 视频晒单数
            afterCountStr = productCommentSummary['afterCountStr']  # 追评数
            goodCountStr = productCommentSummary['goodCountStr']  # 好评数
            generalCountStr = productCommentSummary['generalCountStr']  # 中评数
            poorCountStr = productCommentSummary['poorCountStr']  # 差评数
            # print(nickname,userImageUrl,score,content,creationTime,usefulVoteCount,replyCount,productColor,productSize,imgUrl,remark)
            hotCommentTagStatistics_list = json_data['hotCommentTagStatistics']
            gjc=[]
            for hotCommentTagStatistics in hotCommentTagStatistics_list:
                # gjc=[]
                name = hotCommentTagStatistics['name'] #评价关键词
                count = hotCommentTagStatistics['count'] #评价关键词数
                guanjianci = str(name) + '(' + str(count) + ')' #评价关键词汇总
                gjc.append(guanjianci)
            # print(gjc)
            csv_writer.writerow(
                [nickname, userImageUrl, score, content, creationTime, usefulVoteCount, replyCount, productColor, productSize,
                 imgUrl, remark, productId, goodRate, goodRateShow, commentCountStr, videoCountStr, afterCountStr, goodCountStr,
                 generalCountStr, poorCountStr,str(gjc).replace('[','').replace(']','').replace("'",'').replace(',','')]
            )
        # print(productId,goodRate,goodRateShow,commentCountStr,videoCountStr,afterCountStr,goodCountStr,generalCountStr,poorCountStr)

        time.sleep(float(format(random.uniform(0, 2), '.2f')))

羊肉评论

# https://club.jd.com/comment/productPageComments.action?callback=fetchJSON_comment98&productId=100025128936&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0&fold=1
import re

import requests
import csv
import random
import time
import re

headers={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36',
    'Cookie':'unpl=JF8EAK1nNSttWRgGARMGSBIXGFhcW1pcGERWZ2FWXQ8ITgMBHAQcEkN7XlVdXhRKHx9ubhRVXVNKXQ4aCysSEXteXVdZDEsWC2tXVgQFDQ8VXURJQlZAFDNVCV9dSRZRZjJWBFtdT1xWSAYYRRMfDlAKDlhCR1FpMjVkXlh7VAQrAh0TEU5cU1tbOHsQM19XAlFaXEtVNRoyGiJSHwFdXVUKTBROb2EEVVhZTFEDKwMrEQ; __jdu=170040647; areaId=13; PCSYCityID=CN_370000_370900_0; shshshfpa=d13cd1dc-5642-0f9b-e976-ec229feb579d-1662570491; shshshfpb=qzRhxau6Qqa2HQrqI5HSfpQ; __jdc=122270672; __jdv=122270672|baidu|-|organic|notset|1662570520701; shshshfp=2aff9b962254e92011938f034976e9b4; ip_cityCode=1112; ipLoc-djd=13-1112-46665-46698; mt_xid=V2_52007VwMUU1xdU10aTykJBGEDE1VeX04ISkwYQAA0UBdOVV4FCANOEVkNN1AbBV9YAA4vShhfBnsCGk5cUUNZHkIcXw5jByJQbVhiXRlJGl8CbgITUm1YVF4b; __jda=122270672.170040647.1662570487.1662638740.1662649143.6; mba_muid=170040647; jsavif=1; wlfstk_smdl=3xrhic2zyiy1m4haz2z82lt1ce4gptlt; TrackID=12sC0yMovK7SLxiG7KVpywHWXbK7ycxX-zPQ4W1ayO5In1o6MR5gZIFZUafxeWT_18tnyj6qVykt69fouAAdFZ4f8cCAO0bHwy-7jv89ucrc; thor=6C6F8D919DE139ECE11D28FFF3E5EC84273A857577959D0B03CF0F44FD7D58194A99992F4F5ABEE3C2C43FFBC769D73B6155388EE2BF8FBF8D7C0889F8693448BE7F09429E93F4007D9ECB6ED6E3BBEF9F756043062DF85FA13A3C9F4E957630FF6F10BEA3CE9758A3890C3FFD74959D539A518F5A158853FD57853BA794F21AC2074AEC08B657B3FEC70C83D752F8A0193D3EC703629B1AB38335E42AEE0DA6; pinId=fs_MVnVFxUehygdImyF_b7V9-x-f3wj7; pin=jd_46cd846c5dded; unick=jd_46cd846c5dded; ceshi3.com=000; _tp=uy5RB0M89VjLDNQ7XsGqh5lPpymDUPR0iKlt%2FBf8bEI%3D; _pst=jd_46cd846c5dded; token=2780e6694f589c8dafa7dd0e349b400a,3,923695; __tk=qux31D2YKzxFrDM32cxnrASA2wbt1zM5rYsDqzbtrUTsKwbY1DaDrn,3,923695; 3AB9D23F7A4B3C9B=3G7NRO6TRKPOSZN7QIQA75I36DUPPJUCM735CSVTFQHNTIPGFBRM42YWWAJU53IBGTMCBX5TUVJDGS5CWSS3RXD2YQ; shshshsID=50bb9d1cc99e76bac5444016311ec585_14_1662652174060; __jdb=122270672.23.170040647|6.1662649143'
}
f = open('羊肉评论新.csv',mode='a',encoding='gbk',newline='')
csv_writer = csv.writer(f)
csv_writer.writerow(['用户名称','用户头像','评分','评论内容','评论时间','点赞数','评论回复人数','商品属性','页面标题','评论照片链接','评论视频链接','sku','好评度小数表示','好评度百分比表示','全部评价数','视频晒单数','追评数','好评数','中评数','差评数','评价关键词'])

zs=['100016753390','100016753392','100017741418','100034360825','100023134332','100021494638','100000597004','100022399502','57218917697','65891661893','72295690963','100022910853','20676871318','100000679588','100031630216']
for i in zs:
    print(f"--------------------正在爬取{i}网址-----------------------")
    for page in range(0,100,1):
        print(f"-------------------正在爬取第{page}页----------------------")
        url=f'https://club.jd.com/comment/productPageComments.action?productId={i}&score=0&sortType=5&page={page}&pageSize=10&isShadowSku=0&fold=1'

        response = requests.get(url=url,headers=headers)

        # print(response.text)
        json_data = response.json()

        # print(json_data['comments'])

        comment_list = json_data['comments']
        for comment in comment_list:
            nickname = comment['nickname']  #用户名称
            userImageUrl = 'https://' + str(comment['userImageUrl']) #用户头像
            score = comment['score']  # 评分
            content = comment['content'] #评论内容
            content = re.sub(r'[\t\n]','',content)
            creationTime = comment['creationTime'] #评论时间
            usefulVoteCount = comment['usefulVoteCount']  #点赞数
            replyCount = comment['replyCount']  # 评论回复人数
            productColor = comment['productColor'] #商品属性
            productSize = comment['referenceName'] #页面标题
            try:
                images = comment['images']
                for zp in images:
                    imgUrl = 'https:' + str(zp['imgUrl']) #评论照片链接
                videos = comment['videos']
                for lj in videos:
                    # mainUrl = lj['mainUrl'] #评论照片链接
                    remark = lj['remark'] #评论视频链接
            except:
                pass
            productCommentSummary = json_data['productCommentSummary']
            productId = productCommentSummary['productId']  #sku
            goodRate = productCommentSummary['goodRate']  # 好评度小数表示
            goodRateShow = str(productCommentSummary['goodRateShow']) + '%'  # 好评度百分比表示
            commentCountStr = productCommentSummary['commentCountStr']  # 全部评价数
            videoCountStr = productCommentSummary['videoCountStr']  # 视频晒单数
            afterCountStr = productCommentSummary['afterCountStr']  # 追评数
            goodCountStr = productCommentSummary['goodCountStr']  # 好评数
            generalCountStr = productCommentSummary['generalCountStr']  # 中评数
            poorCountStr = productCommentSummary['poorCountStr']  # 差评数
            # print(nickname,userImageUrl,score,content,creationTime,usefulVoteCount,replyCount,productColor,productSize,imgUrl,remark)
            hotCommentTagStatistics_list = json_data['hotCommentTagStatistics']
            gjc=[]
            for hotCommentTagStatistics in hotCommentTagStatistics_list:
                # gjc=[]
                name = hotCommentTagStatistics['name'] #评价关键词
                count = hotCommentTagStatistics['count'] #评价关键词数
                guanjianci = str(name) + '(' + str(count) + ')' #评价关键词汇总
                gjc.append(guanjianci)
            # print(gjc)
            csv_writer.writerow(
                [nickname, userImageUrl, score, content, creationTime, usefulVoteCount, replyCount, productColor, productSize,
                 imgUrl, remark, productId, goodRate, goodRateShow, commentCountStr, videoCountStr, afterCountStr, goodCountStr,
                 generalCountStr, poorCountStr,str(gjc).replace('[','').replace(']','').replace("'",'').replace(',','')]
            )
        # print(productId,goodRate,goodRateShow,commentCountStr,videoCountStr,afterCountStr,goodCountStr,generalCountStr,poorCountStr)

        time.sleep(float(format(random.uniform(0, 2), '.2f')))

鸡肉评论

# https://club.jd.com/comment/productPageComments.action?callback=fetchJSON_comment98&productId=100025128936&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0&fold=1
import re

import requests
import csv
import random
import time
import re

headers={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36',
    'Cookie':'unpl=JF8EAK1nNSttWRgGARMGSBIXGFhcW1pcGERWZ2FWXQ8ITgMBHAQcEkN7XlVdXhRKHx9ubhRVXVNKXQ4aCysSEXteXVdZDEsWC2tXVgQFDQ8VXURJQlZAFDNVCV9dSRZRZjJWBFtdT1xWSAYYRRMfDlAKDlhCR1FpMjVkXlh7VAQrAh0TEU5cU1tbOHsQM19XAlFaXEtVNRoyGiJSHwFdXVUKTBROb2EEVVhZTFEDKwMrEQ; __jdu=170040647; areaId=13; PCSYCityID=CN_370000_370900_0; shshshfpa=d13cd1dc-5642-0f9b-e976-ec229feb579d-1662570491; shshshfpb=qzRhxau6Qqa2HQrqI5HSfpQ; __jdc=122270672; __jdv=122270672|baidu|-|organic|notset|1662570520701; shshshfp=2aff9b962254e92011938f034976e9b4; ip_cityCode=1112; ipLoc-djd=13-1112-46665-46698; mt_xid=V2_52007VwMUU1xdU10aTykJBGEDE1VeX04ISkwYQAA0UBdOVV4FCANOEVkNN1AbBV9YAA4vShhfBnsCGk5cUUNZHkIcXw5jByJQbVhiXRlJGl8CbgITUm1YVF4b; __jda=122270672.170040647.1662570487.1662638740.1662649143.6; mba_muid=170040647; jsavif=1; wlfstk_smdl=3xrhic2zyiy1m4haz2z82lt1ce4gptlt; TrackID=12sC0yMovK7SLxiG7KVpywHWXbK7ycxX-zPQ4W1ayO5In1o6MR5gZIFZUafxeWT_18tnyj6qVykt69fouAAdFZ4f8cCAO0bHwy-7jv89ucrc; thor=6C6F8D919DE139ECE11D28FFF3E5EC84273A857577959D0B03CF0F44FD7D58194A99992F4F5ABEE3C2C43FFBC769D73B6155388EE2BF8FBF8D7C0889F8693448BE7F09429E93F4007D9ECB6ED6E3BBEF9F756043062DF85FA13A3C9F4E957630FF6F10BEA3CE9758A3890C3FFD74959D539A518F5A158853FD57853BA794F21AC2074AEC08B657B3FEC70C83D752F8A0193D3EC703629B1AB38335E42AEE0DA6; pinId=fs_MVnVFxUehygdImyF_b7V9-x-f3wj7; pin=jd_46cd846c5dded; unick=jd_46cd846c5dded; ceshi3.com=000; _tp=uy5RB0M89VjLDNQ7XsGqh5lPpymDUPR0iKlt%2FBf8bEI%3D; _pst=jd_46cd846c5dded; token=2780e6694f589c8dafa7dd0e349b400a,3,923695; __tk=qux31D2YKzxFrDM32cxnrASA2wbt1zM5rYsDqzbtrUTsKwbY1DaDrn,3,923695; 3AB9D23F7A4B3C9B=3G7NRO6TRKPOSZN7QIQA75I36DUPPJUCM735CSVTFQHNTIPGFBRM42YWWAJU53IBGTMCBX5TUVJDGS5CWSS3RXD2YQ; shshshsID=50bb9d1cc99e76bac5444016311ec585_14_1662652174060; __jdb=122270672.23.170040647|6.1662649143'
}
f = open('鸡肉评论.csv',mode='a',encoding='gbk',newline='')
csv_writer = csv.writer(f)
csv_writer.writerow(['用户名称','用户头像','评分','评论内容','评论时间','点赞数','评论回复人数','商品属性','页面标题','评论照片链接','评论视频链接','sku','好评度小数表示','好评度百分比表示','全部评价数','视频晒单数','追评数','好评数','中评数','差评数','评价关键词'])

zs=['1904033','11828234904','1904056','3901551','100007021816','100005012320','3936157','3901547','2136193','100021752502','100036115056','100016729022','100009920113','100012521838','100013951564','100007991207']
for i in zs:
    print(f"--------------------正在爬取{i}网址-----------------------")
    for page in range(0,100,1):
        print(f"-------------------正在爬取第{page}页----------------------")
        url=f'https://club.jd.com/comment/productPageComments.action?productId={i}&score=0&sortType=5&page={page}&pageSize=10&isShadowSku=0&fold=1'

        response = requests.get(url=url,headers=headers)

        # print(response.text)
        json_data = response.json()

        # print(json_data['comments'])

        comment_list = json_data['comments']
        for comment in comment_list:
            nickname = comment['nickname']  #用户名称
            userImageUrl = 'https://' + str(comment['userImageUrl']) #用户头像
            score = comment['score']  # 评分
            content = comment['content'] #评论内容
            content = re.sub(r'[\t\n]','',content)
            creationTime = comment['creationTime'] #评论时间
            usefulVoteCount = comment['usefulVoteCount']  #点赞数
            replyCount = comment['replyCount']  # 评论回复人数
            productColor = comment['productColor'] #商品属性
            productSize = comment['referenceName'] #页面标题
            try:
                images = comment['images']
                for zp in images:
                    imgUrl = 'https:' + str(zp['imgUrl']) #评论照片链接
                videos = comment['videos']
                for lj in videos:
                    # mainUrl = lj['mainUrl'] #评论照片链接
                    remark = lj['remark'] #评论视频链接
            except:
                pass
            productCommentSummary = json_data['productCommentSummary']
            productId = productCommentSummary['productId']  #sku
            goodRate = productCommentSummary['goodRate']  # 好评度小数表示
            goodRateShow = str(productCommentSummary['goodRateShow']) + '%'  # 好评度百分比表示
            commentCountStr = productCommentSummary['commentCountStr']  # 全部评价数
            videoCountStr = productCommentSummary['videoCountStr']  # 视频晒单数
            afterCountStr = productCommentSummary['afterCountStr']  # 追评数
            goodCountStr = productCommentSummary['goodCountStr']  # 好评数
            generalCountStr = productCommentSummary['generalCountStr']  # 中评数
            poorCountStr = productCommentSummary['poorCountStr']  # 差评数
            # print(nickname,userImageUrl,score,content,creationTime,usefulVoteCount,replyCount,productColor,productSize,imgUrl,remark)
            hotCommentTagStatistics_list = json_data['hotCommentTagStatistics']
            gjc=[]
            for hotCommentTagStatistics in hotCommentTagStatistics_list:
                # gjc=[]
                name = hotCommentTagStatistics['name'] #评价关键词
                count = hotCommentTagStatistics['count'] #评价关键词数
                guanjianci = str(name) + '(' + str(count) + ')' #评价关键词汇总
                gjc.append(guanjianci)
            # print(gjc)
            csv_writer.writerow(
                [nickname, userImageUrl, score, content, creationTime, usefulVoteCount, replyCount, productColor, productSize,
                 imgUrl, remark, productId, goodRate, goodRateShow, commentCountStr, videoCountStr, afterCountStr, goodCountStr,
                 generalCountStr, poorCountStr,str(gjc).replace('[','').replace(']','').replace("'",'').replace(',','')]
            )
        # print(productId,goodRate,goodRateShow,commentCountStr,videoCountStr,afterCountStr,goodCountStr,generalCountStr,poorCountStr)

        time.sleep(float(format(random.uniform(0, 2), '.2f')))

鸭肉评论

# https://club.jd.com/comment/productPageComments.action?callback=fetchJSON_comment98&productId=100025128936&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0&fold=1
import re

import requests
import csv
import random
import time
import re
from fake_useragent import UserAgent

headers={
    # 'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36',
    # 'Cookie':'unpl=JF8EAK1nNSttWRgGARMGSBIXGFhcW1pcGERWZ2FWXQ8ITgMBHAQcEkN7XlVdXhRKHx9ubhRVXVNKXQ4aCysSEXteXVdZDEsWC2tXVgQFDQ8VXURJQlZAFDNVCV9dSRZRZjJWBFtdT1xWSAYYRRMfDlAKDlhCR1FpMjVkXlh7VAQrAh0TEU5cU1tbOHsQM19XAlFaXEtVNRoyGiJSHwFdXVUKTBROb2EEVVhZTFEDKwMrEQ; __jdu=170040647; areaId=13; PCSYCityID=CN_370000_370900_0; shshshfpa=d13cd1dc-5642-0f9b-e976-ec229feb579d-1662570491; shshshfpb=qzRhxau6Qqa2HQrqI5HSfpQ; __jdc=122270672; __jdv=122270672|baidu|-|organic|notset|1662570520701; shshshfp=2aff9b962254e92011938f034976e9b4; ip_cityCode=1112; ipLoc-djd=13-1112-46665-46698; jwotest_product=99; mt_xid=V2_52007VwMUU1xdU10aTylbB24BQlcKUE5ZGk9NQAAzBxJODllUCQNKSglQNVNCBQpaVVMvShhfBnsCGk5cUUNbH0IZWg5lASJQbVhiUxlOHVUHYwQbUl1ZUVkdQRxZB1cDFFZZ; jsavif=1; __jda=122270672.170040647.1662570487.1662570521.1662614449.3; JSESSIONID=E4183D4825A3F25B79066C6FAB065E13.s1; token=631272b9b7166cb267e9a2f9907da026,2,923675; __tk=f37514b34c6ce7e350f7e33722bc6f58,2,923675; shshshsID=daa07f7ca60bb8ada946f160dad78a29_3_1662615826441; __jdb=122270672.3.170040647|3.1662614449; 3AB9D23F7A4B3C9B=3G7NRO6TRKPOSZN7QIQA75I36DUPPJUCM735CSVTFQHNTIPGFBRM42YWWAJU53IBGTMCBX5TUVJDGS5CWSS3RXD2YQ',
    'User-Agent': UserAgent().random
}
f = open('鸭肉评论最最终.csv',mode='a',encoding='gbk',newline='')
csv_writer = csv.writer(f)
csv_writer.writerow(['用户名称','用户头像','评分','评论内容','评论时间','点赞数','评论回复人数','商品属性','页面标题','评论照片链接','评论视频链接','sku','好评度小数表示','好评度百分比表示','全部评价数','视频晒单数','追评数','好评数','中评数','差评数','评价关键词'])

zs=['10030440439616','10030440439615','10024510614096','10032791578778','10033508902560','10035680919445','10035680919444','100013702803','100023960924','100023960908','100023960904','100025691332']
for i in zs:
    print(f"--------------------正在爬取{i}网址-----------------------")
    for page in range(0,100,1):
        print(f"-------------------正在爬取第{page}页----------------------")
        url=f'https://club.jd.com/comment/productPageComments.action?productId={i}&score=0&sortType=5&page={page}&pageSize=10&isShadowSku=0&fold=1'

        response = requests.get(url=url,headers=headers)

        # print(response.text)
        json_data = response.json()

        # print(json_data['comments'])

        comment_list = json_data['comments']
        for comment in comment_list:
            nickname = comment['nickname']  #用户名称
            userImageUrl = 'https://' + str(comment['userImageUrl']) #用户头像
            score = comment['score']  # 评分
            content = comment['content'] #评论内容
            content = re.sub(r'[\t\n]','',content)
            creationTime = comment['creationTime'] #评论时间
            usefulVoteCount = comment['usefulVoteCount']  #点赞数
            replyCount = comment['replyCount']  # 评论回复人数
            productColor = comment['productColor'] #商品属性
            productSize = comment['referenceName'] #页面标题
            try:
                images = comment['images']
                for zp in images:
                    imgUrl = 'https:' + str(zp['imgUrl']) #评论照片链接
                videos = comment['videos']
                for lj in videos:
                    # mainUrl = lj['mainUrl'] #评论照片链接
                    remark = lj['remark'] #评论视频链接
            except:
                pass
            productCommentSummary = json_data['productCommentSummary']
            productId = productCommentSummary['productId']  #sku
            goodRate = productCommentSummary['goodRate']  # 好评度小数表示
            goodRateShow = str(productCommentSummary['goodRateShow']) + '%'  # 好评度百分比表示
            commentCountStr = productCommentSummary['commentCountStr']  # 全部评价数
            videoCountStr = productCommentSummary['videoCountStr']  # 视频晒单数
            afterCountStr = productCommentSummary['afterCountStr']  # 追评数
            goodCountStr = productCommentSummary['goodCountStr']  # 好评数
            generalCountStr = productCommentSummary['generalCountStr']  # 中评数
            poorCountStr = productCommentSummary['poorCountStr']  # 差评数
            # print(nickname,userImageUrl,score,content,creationTime,usefulVoteCount,replyCount,productColor,productSize,imgUrl,remark)
            hotCommentTagStatistics_list = json_data['hotCommentTagStatistics']
            gjc=[]
            for hotCommentTagStatistics in hotCommentTagStatistics_list:
                # gjc=[]
                name = hotCommentTagStatistics['name'] #评价关键词
                count = hotCommentTagStatistics['count'] #评价关键词数
                guanjianci = str(name) + '(' + str(count) + ')' #评价关键词汇总
                gjc.append(guanjianci)
            # print(gjc)
            csv_writer.writerow(
                [nickname, userImageUrl, score, content, creationTime, usefulVoteCount, replyCount, productColor, productSize,
                 imgUrl, remark, productId, goodRate, goodRateShow, commentCountStr, videoCountStr, afterCountStr, goodCountStr,
                 generalCountStr, poorCountStr,str(gjc).replace('[','').replace(']','').replace("'",'').replace(',','')]
            )
        # print(productId,goodRate,goodRateShow,commentCountStr,videoCountStr,afterCountStr,goodCountStr,generalCountStr,poorCountStr)

        time.sleep(float(format(random.uniform(0, 2), '.2f')))

好评获取

# https://club.jd.com/comment/productPageComments.action?callback=fetchJSON_comment98&productId=100025128936&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0&fold=1
import re

import requests
import csv
import random
import time
import re
from fake_useragent import UserAgent

headers={
    # 'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36',
    # 'Cookie':'unpl=JF8EAK1nNSttWRgGARMGSBIXGFhcW1pcGERWZ2FWXQ8ITgMBHAQcEkN7XlVdXhRKHx9ubhRVXVNKXQ4aCysSEXteXVdZDEsWC2tXVgQFDQ8VXURJQlZAFDNVCV9dSRZRZjJWBFtdT1xWSAYYRRMfDlAKDlhCR1FpMjVkXlh7VAQrAh0TEU5cU1tbOHsQM19XAlFaXEtVNRoyGiJSHwFdXVUKTBROb2EEVVhZTFEDKwMrEQ; __jdu=170040647; areaId=13; PCSYCityID=CN_370000_370900_0; shshshfpa=d13cd1dc-5642-0f9b-e976-ec229feb579d-1662570491; shshshfpb=qzRhxau6Qqa2HQrqI5HSfpQ; __jdc=122270672; __jdv=122270672|baidu|-|organic|notset|1662570520701; shshshfp=2aff9b962254e92011938f034976e9b4; ip_cityCode=1112; ipLoc-djd=13-1112-46665-46698; jwotest_product=99; mt_xid=V2_52007VwMUU1xdU10aTylbB24BQlcKUE5ZGk9NQAAzBxJODllUCQNKSglQNVNCBQpaVVMvShhfBnsCGk5cUUNbH0IZWg5lASJQbVhiUxlOHVUHYwQbUl1ZUVkdQRxZB1cDFFZZ; jsavif=1; __jda=122270672.170040647.1662570487.1662570521.1662614449.3; JSESSIONID=E4183D4825A3F25B79066C6FAB065E13.s1; token=631272b9b7166cb267e9a2f9907da026,2,923675; __tk=f37514b34c6ce7e350f7e33722bc6f58,2,923675; shshshsID=daa07f7ca60bb8ada946f160dad78a29_3_1662615826441; __jdb=122270672.3.170040647|3.1662614449; 3AB9D23F7A4B3C9B=3G7NRO6TRKPOSZN7QIQA75I36DUPPJUCM735CSVTFQHNTIPGFBRM42YWWAJU53IBGTMCBX5TUVJDGS5CWSS3RXD2YQ',
    'User-Agent': UserAgent().random
}
f = open('好评评论.csv',mode='a',encoding='gbk',newline='')
csv_writer = csv.writer(f)
csv_writer.writerow(['用户名称','用户头像','评分','评论内容','评论时间','点赞数','评论回复人数','商品属性','页面标题','评论照片链接','评论视频链接','sku','好评度小数表示','好评度百分比表示','全部评价数','视频晒单数','追评数','好评数','中评数','差评数','评价关键词'])

zs=['10030440439616','10030440439615','10024510614096','10032791578778','10033508902560','10035680919445','10035680919444','100013702803','100023960924','100023960908','100023960904','100025691332']
for i in zs:
    print(f"--------------------正在爬取{i}网址-----------------------")
    for page in range(0,100,1):
        print(f"-------------------正在爬取第{page}页----------------------")
        url=f'https://club.jd.com/comment/productPageComments.action?productId={i}&score=3&sortType=5&page={page}&pageSize=10&isShadowSku=0&fold=1'

        response = requests.get(url=url,headers=headers)

        # print(response.text)
        json_data = response.json()

        # print(json_data['comments'])

        comment_list = json_data['comments']
        for comment in comment_list:
            nickname = comment['nickname']  #用户名称
            userImageUrl = 'https://' + str(comment['userImageUrl']) #用户头像
            score = comment['score']  # 评分
            content = comment['content'] #评论内容
            content = re.sub(r'[\t\n]','',content)
            creationTime = comment['creationTime'] #评论时间
            usefulVoteCount = comment['usefulVoteCount']  #点赞数
            replyCount = comment['replyCount']  # 评论回复人数
            productColor = comment['productColor'] #商品属性
            productSize = comment['referenceName'] #页面标题
            try:
                images = comment['images']
                for zp in images:
                    imgUrl = 'https:' + str(zp['imgUrl']) #评论照片链接
                videos = comment['videos']
                for lj in videos:
                    # mainUrl = lj['mainUrl'] #评论照片链接
                    remark = lj['remark'] #评论视频链接
            except:
                pass
            productCommentSummary = json_data['productCommentSummary']
            productId = productCommentSummary['productId']  #sku
            goodRate = productCommentSummary['goodRate']  # 好评度小数表示
            goodRateShow = str(productCommentSummary['goodRateShow']) + '%'  # 好评度百分比表示
            commentCountStr = productCommentSummary['commentCountStr']  # 全部评价数
            videoCountStr = productCommentSummary['videoCountStr']  # 视频晒单数
            afterCountStr = productCommentSummary['afterCountStr']  # 追评数
            goodCountStr = productCommentSummary['goodCountStr']  # 好评数
            generalCountStr = productCommentSummary['generalCountStr']  # 中评数
            poorCountStr = productCommentSummary['poorCountStr']  # 差评数
            # print(nickname,userImageUrl,score,content,creationTime,usefulVoteCount,replyCount,productColor,productSize,imgUrl,remark)
            hotCommentTagStatistics_list = json_data['hotCommentTagStatistics']
            gjc=[]
            for hotCommentTagStatistics in hotCommentTagStatistics_list:
                # gjc=[]
                name = hotCommentTagStatistics['name'] #评价关键词
                count = hotCommentTagStatistics['count'] #评价关键词数
                guanjianci = str(name) + '(' + str(count) + ')' #评价关键词汇总
                gjc.append(guanjianci)
            # print(gjc)
            csv_writer.writerow(
                [nickname, userImageUrl, score, content, creationTime, usefulVoteCount, replyCount, productColor, productSize,
                 imgUrl, remark, productId, goodRate, goodRateShow, commentCountStr, videoCountStr, afterCountStr, goodCountStr,
                 generalCountStr, poorCountStr,str(gjc).replace('[','').replace(']','').replace("'",'').replace(',','')]
            )
        # print(productId,goodRate,goodRateShow,commentCountStr,videoCountStr,afterCountStr,goodCountStr,generalCountStr,poorCountStr)

        time.sleep(float(format(random.uniform(0, 5), '.2f')))

中评获取

# https://club.jd.com/comment/productPageComments.action?callback=fetchJSON_comment98&productId=100025128936&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0&fold=1
import re

import requests
import csv
import random
import time
import re
from fake_useragent import UserAgent

headers={
    # 'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36',
    # 'Cookie':'unpl=JF8EAK1nNSttWRgGARMGSBIXGFhcW1pcGERWZ2FWXQ8ITgMBHAQcEkN7XlVdXhRKHx9ubhRVXVNKXQ4aCysSEXteXVdZDEsWC2tXVgQFDQ8VXURJQlZAFDNVCV9dSRZRZjJWBFtdT1xWSAYYRRMfDlAKDlhCR1FpMjVkXlh7VAQrAh0TEU5cU1tbOHsQM19XAlFaXEtVNRoyGiJSHwFdXVUKTBROb2EEVVhZTFEDKwMrEQ; __jdu=170040647; areaId=13; PCSYCityID=CN_370000_370900_0; shshshfpa=d13cd1dc-5642-0f9b-e976-ec229feb579d-1662570491; shshshfpb=qzRhxau6Qqa2HQrqI5HSfpQ; __jdc=122270672; __jdv=122270672|baidu|-|organic|notset|1662570520701; shshshfp=2aff9b962254e92011938f034976e9b4; ip_cityCode=1112; ipLoc-djd=13-1112-46665-46698; jwotest_product=99; mt_xid=V2_52007VwMUU1xdU10aTylbB24BQlcKUE5ZGk9NQAAzBxJODllUCQNKSglQNVNCBQpaVVMvShhfBnsCGk5cUUNbH0IZWg5lASJQbVhiUxlOHVUHYwQbUl1ZUVkdQRxZB1cDFFZZ; jsavif=1; __jda=122270672.170040647.1662570487.1662570521.1662614449.3; JSESSIONID=E4183D4825A3F25B79066C6FAB065E13.s1; token=631272b9b7166cb267e9a2f9907da026,2,923675; __tk=f37514b34c6ce7e350f7e33722bc6f58,2,923675; shshshsID=daa07f7ca60bb8ada946f160dad78a29_3_1662615826441; __jdb=122270672.3.170040647|3.1662614449; 3AB9D23F7A4B3C9B=3G7NRO6TRKPOSZN7QIQA75I36DUPPJUCM735CSVTFQHNTIPGFBRM42YWWAJU53IBGTMCBX5TUVJDGS5CWSS3RXD2YQ',
    'User-Agent': UserAgent().random
}
f = open('中评评论.csv',mode='a',encoding='gbk',newline='')
csv_writer = csv.writer(f)
csv_writer.writerow(['用户名称','用户头像','评分','评论内容','评论时间','点赞数','评论回复人数','商品属性','页面标题','评论照片链接','评论视频链接','sku','好评度小数表示','好评度百分比表示','全部评价数','视频晒单数','追评数','好评数','中评数','差评数','评价关键词'])

zs=['10030440439616','10030440439615','10024510614096','10032791578778','10033508902560','10035680919445','10035680919444','100013702803','100023960924','100023960908','100023960904','100025691332']
for i in zs:
    print(f"--------------------正在爬取{i}网址-----------------------")
    for page in range(0,100,1):
        print(f"-------------------正在爬取第{page}页----------------------")
        url=f'https://club.jd.com/comment/productPageComments.action?productId={i}&score=2&sortType=5&page={page}&pageSize=10&isShadowSku=0&fold=1'

        response = requests.get(url=url,headers=headers)

        # print(response.text)
        json_data = response.json()

        # print(json_data['comments'])

        comment_list = json_data['comments']
        for comment in comment_list:
            nickname = comment['nickname']  #用户名称
            userImageUrl = 'https://' + str(comment['userImageUrl']) #用户头像
            score = comment['score']  # 评分
            content = comment['content'] #评论内容
            content = re.sub(r'[\t\n]','',content)
            creationTime = comment['creationTime'] #评论时间
            usefulVoteCount = comment['usefulVoteCount']  #点赞数
            replyCount = comment['replyCount']  # 评论回复人数
            productColor = comment['productColor'] #商品属性
            productSize = comment['referenceName'] #页面标题
            try:
                images = comment['images']
                for zp in images:
                    imgUrl = 'https:' + str(zp['imgUrl']) #评论照片链接
                videos = comment['videos']
                for lj in videos:
                    # mainUrl = lj['mainUrl'] #评论照片链接
                    remark = lj['remark'] #评论视频链接
            except:
                pass
            productCommentSummary = json_data['productCommentSummary']
            productId = productCommentSummary['productId']  #sku
            goodRate = productCommentSummary['goodRate']  # 好评度小数表示
            goodRateShow = str(productCommentSummary['goodRateShow']) + '%'  # 好评度百分比表示
            commentCountStr = productCommentSummary['commentCountStr']  # 全部评价数
            videoCountStr = productCommentSummary['videoCountStr']  # 视频晒单数
            afterCountStr = productCommentSummary['afterCountStr']  # 追评数
            goodCountStr = productCommentSummary['goodCountStr']  # 好评数
            generalCountStr = productCommentSummary['generalCountStr']  # 中评数
            poorCountStr = productCommentSummary['poorCountStr']  # 差评数
            # print(nickname,userImageUrl,score,content,creationTime,usefulVoteCount,replyCount,productColor,productSize,imgUrl,remark)
            hotCommentTagStatistics_list = json_data['hotCommentTagStatistics']
            gjc=[]
            for hotCommentTagStatistics in hotCommentTagStatistics_list:
                # gjc=[]
                name = hotCommentTagStatistics['name'] #评价关键词
                count = hotCommentTagStatistics['count'] #评价关键词数
                guanjianci = str(name) + '(' + str(count) + ')' #评价关键词汇总
                gjc.append(guanjianci)
            # print(gjc)
            csv_writer.writerow(
                [nickname, userImageUrl, score, content, creationTime, usefulVoteCount, replyCount, productColor, productSize,
                 imgUrl, remark, productId, goodRate, goodRateShow, commentCountStr, videoCountStr, afterCountStr, goodCountStr,
                 generalCountStr, poorCountStr,str(gjc).replace('[','').replace(']','').replace("'",'').replace(',','')]
            )
        # print(productId,goodRate,goodRateShow,commentCountStr,videoCountStr,afterCountStr,goodCountStr,generalCountStr,poorCountStr)

        time.sleep(float(format(random.uniform(0, 5), '.2f')))

差评获取

# https://club.jd.com/comment/productPageComments.action?callback=fetchJSON_comment98&productId=100025128936&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0&fold=1
import re

import requests
import csv
import random
import time
import re
from fake_useragent import UserAgent

headers={
    # 'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36',
    # 'Cookie':'unpl=JF8EAK1nNSttWRgGARMGSBIXGFhcW1pcGERWZ2FWXQ8ITgMBHAQcEkN7XlVdXhRKHx9ubhRVXVNKXQ4aCysSEXteXVdZDEsWC2tXVgQFDQ8VXURJQlZAFDNVCV9dSRZRZjJWBFtdT1xWSAYYRRMfDlAKDlhCR1FpMjVkXlh7VAQrAh0TEU5cU1tbOHsQM19XAlFaXEtVNRoyGiJSHwFdXVUKTBROb2EEVVhZTFEDKwMrEQ; __jdu=170040647; areaId=13; PCSYCityID=CN_370000_370900_0; shshshfpa=d13cd1dc-5642-0f9b-e976-ec229feb579d-1662570491; shshshfpb=qzRhxau6Qqa2HQrqI5HSfpQ; __jdc=122270672; __jdv=122270672|baidu|-|organic|notset|1662570520701; shshshfp=2aff9b962254e92011938f034976e9b4; ip_cityCode=1112; ipLoc-djd=13-1112-46665-46698; jwotest_product=99; mt_xid=V2_52007VwMUU1xdU10aTylbB24BQlcKUE5ZGk9NQAAzBxJODllUCQNKSglQNVNCBQpaVVMvShhfBnsCGk5cUUNbH0IZWg5lASJQbVhiUxlOHVUHYwQbUl1ZUVkdQRxZB1cDFFZZ; jsavif=1; __jda=122270672.170040647.1662570487.1662570521.1662614449.3; JSESSIONID=E4183D4825A3F25B79066C6FAB065E13.s1; token=631272b9b7166cb267e9a2f9907da026,2,923675; __tk=f37514b34c6ce7e350f7e33722bc6f58,2,923675; shshshsID=daa07f7ca60bb8ada946f160dad78a29_3_1662615826441; __jdb=122270672.3.170040647|3.1662614449; 3AB9D23F7A4B3C9B=3G7NRO6TRKPOSZN7QIQA75I36DUPPJUCM735CSVTFQHNTIPGFBRM42YWWAJU53IBGTMCBX5TUVJDGS5CWSS3RXD2YQ',
    'User-Agent': UserAgent().random
}
f = open('差评评论.csv',mode='a',encoding='gbk',newline='')
csv_writer = csv.writer(f)
csv_writer.writerow(['用户名称','用户头像','评分','评论内容','评论时间','点赞数','评论回复人数','商品属性','页面标题','评论照片链接','评论视频链接','sku','好评度小数表示','好评度百分比表示','全部评价数','视频晒单数','追评数','好评数','中评数','差评数','评价关键词'])

zs=['10030440439616','10030440439615','10024510614096','10032791578778','10033508902560','10035680919445','10035680919444','100013702803','100023960924','100023960908','100023960904','100025691332']
for i in zs:
    print(f"--------------------正在爬取{i}网址-----------------------")
    for page in range(0,100,1):
        print(f"-------------------正在爬取第{page}页----------------------")
        url=f'https://club.jd.com/comment/productPageComments.action?productId={i}&score=1&sortType=5&page={page}&pageSize=10&isShadowSku=0&fold=1'

        response = requests.get(url=url,headers=headers)

        # print(response.text)
        json_data = response.json()

        # print(json_data['comments'])

        comment_list = json_data['comments']
        for comment in comment_list:
            nickname = comment['nickname']  #用户名称
            userImageUrl = 'https://' + str(comment['userImageUrl']) #用户头像
            score = comment['score']  # 评分
            content = comment['content'] #评论内容
            content = re.sub(r'[\t\n]','',content)
            creationTime = comment['creationTime'] #评论时间
            usefulVoteCount = comment['usefulVoteCount']  #点赞数
            replyCount = comment['replyCount']  # 评论回复人数
            productColor = comment['productColor'] #商品属性
            productSize = comment['referenceName'] #页面标题
            try:
                images = comment['images']
                for zp in images:
                    imgUrl = 'https:' + str(zp['imgUrl']) #评论照片链接
                videos = comment['videos']
                for lj in videos:
                    # mainUrl = lj['mainUrl'] #评论照片链接
                    remark = lj['remark'] #评论视频链接
            except:
                pass
            productCommentSummary = json_data['productCommentSummary']
            productId = productCommentSummary['productId']  #sku
            goodRate = productCommentSummary['goodRate']  # 好评度小数表示
            goodRateShow = str(productCommentSummary['goodRateShow']) + '%'  # 好评度百分比表示
            commentCountStr = productCommentSummary['commentCountStr']  # 全部评价数
            videoCountStr = productCommentSummary['videoCountStr']  # 视频晒单数
            afterCountStr = productCommentSummary['afterCountStr']  # 追评数
            goodCountStr = productCommentSummary['goodCountStr']  # 好评数
            generalCountStr = productCommentSummary['generalCountStr']  # 中评数
            poorCountStr = productCommentSummary['poorCountStr']  # 差评数
            # print(nickname,userImageUrl,score,content,creationTime,usefulVoteCount,replyCount,productColor,productSize,imgUrl,remark)
            hotCommentTagStatistics_list = json_data['hotCommentTagStatistics']
            gjc=[]
            for hotCommentTagStatistics in hotCommentTagStatistics_list:
                # gjc=[]
                name = hotCommentTagStatistics['name'] #评价关键词
                count = hotCommentTagStatistics['count'] #评价关键词数
                guanjianci = str(name) + '(' + str(count) + ')' #评价关键词汇总
                gjc.append(guanjianci)
            # print(gjc)
            csv_writer.writerow(
                [nickname, userImageUrl, score, content, creationTime, usefulVoteCount, replyCount, productColor, productSize,
                 imgUrl, remark, productId, goodRate, goodRateShow, commentCountStr, videoCountStr, afterCountStr, goodCountStr,
                 generalCountStr, poorCountStr,str(gjc).replace('[','').replace(']','').replace("'",'').replace(',','')]
            )
        # print(productId,goodRate,goodRateShow,commentCountStr,videoCountStr,afterCountStr,goodCountStr,generalCountStr,poorCountStr)

        time.sleep(float(format(random.uniform(0, 5), '.2f')))
py爱好者~
关注
  • 2
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论
专栏目录
看大神如何用python爬虫爬取京东商品评论
python基础学习教程
04-27 6006
@本文来源于公众号：csdn2299，喜欢可以关注公众号 程序员学府 本篇文章是介绍如何抓取京东商城商品评论信息，并对这些评论信息进行分析和可视化。下面是要抓取的商品信息，一款女士文胸。这个商品共有红色，黑色和肤色三种颜色， 70B到90D共18个尺寸，以及超过700条的购买评论。 京东商品评论信息是由JS动态加载的，所以直接抓取商品详情页的URL并不能获得商品评论的信息。因此我们需要先找到存放...
爬虫案例—京东数据爬取、数据处理及数据可视化（效果+代码）
热门推荐
m0_50902656的博客
12-21 2万+
利用Python知识点对京东相关数据进行爬取及数据可视化处理
Python 读写csv文件总结（pandas、csv库）
u011412768的博客
11-15 1392
1、用pandas读写csv文件，参考：https://blog.csdn.net/u011412768/article/details/109522349 2、用Python自带的csv库读写 （1）按行写csv文件 用csv按行写数据时，每一行是按照一个list写入的，所以将每一行的数据做成一个list就可以输出了。 值得注意的是，csv逐行写入数据时，默认的是行之间会空一行，可以通过newline=""来解决这个问题。 import csv ...
Python代码解析数据
zhulewen的博客
09-19 1499
{"jwotestProduct":null,"score":0,"comments":[{"id":13045772365,"guid":"a9476a61-cbab-4e0e-9f62-52ec5fe0af67","content":"质量很好，做工料子都好，颜色也好看。穿着很舒\n服。脚底很有弹性，走路很轻。脚底很好，防滑\n的。穿得也好看，颜色鲜艳。效果不错，很大方大气，黑面白底清新脱俗特别精神，鞋面透气性不错","creationTime":"2019-07-24 09:22:57","isD..
python制作爬虫爬取京东商品评论教程
onemorepoint的博客
07-18 2655
python制作爬虫爬取京东商品评论教程 作者：蓝鲸 类型：转载 本文是继前2篇Python爬虫系列文章的后续篇，给大家介绍的是如何使用Python爬取京东商品评论信息的方法，并根据数据绘制成各种统计图表，非常的细致，有需要的小伙伴可以参考下   本篇文章是python爬虫系列的第三篇，介绍如何抓取京东商城商品评论信息，并对这些评论信息进行分析和可...
网络爬虫 | 京东全站数据采集（类目、店铺、商品、评论）——基于Python中Scrapy框架
也许有一天我们再相逢 睁开眼睛看清楚 我才是英雄！
05-11 2458
123
京东生鲜各肉类评论数据爬取
最新发布
01-26
京东生鲜各肉类评论数据爬取
京东生鲜自营店铺流量来源数据分析.docx
05-18
京东生鲜自营店铺流量来源数据分析.docx
生鲜电商平台型模式分析——以“京东生鲜”为例.pdf
07-08
生鲜电商平台型模式分析——以“京东生鲜”为例.pdf
京东生鲜行业研究报告-京东大数据研究院-201906.pdf
09-10
京东生鲜行业研究报告-京东大数据研究院-201906.pdf
“京东生鲜”电商平台O2O模式探析.pdf
07-08
“京东生鲜”电商平台O2O模式探析.pdf
京东价格监控软件开发技术探讨七：如何获取京东商品评价信息
奥塔在线
03-30 2883
杀京东 做最走心的京东价格监控软件 免费开源的价格监控软件 欢迎加入QQ群415014949一起讨论 在购买商品的时候，这个商品质量怎么样、价格贵不贵、商家服务态度好不好？这些问题往往是购买者急需知道的信息。
京东评论爬虫教程/手册 Python (4步操作 超级简单)
rileyjin的博客
02-12 3587
感谢 @biptcszy 本文由该博主的教程和code稍作更新而来 因为我做的改动稍微有点多 所以开了新post 原文地址:https://blog.csdn.net/weixin_42474261/article/details/88354134 原文地址:https://blog.csdn.net/weixin_42474261/article/details/88354134 原文地址:ht...
re 匹配整数、浮点数
u010741112的博客
06-21 852
如匹配内容 text=' commentCB({ "version": "v2", "errcode": "0", "errmsg": "", "result":{ "productCommentSummary":{ "SkuId": "1097069", "ProductId": "1097069", "DefaultGoodCount": "258614", ...
[爬虫]使用python抓取京东全站数据（商品，店铺，分类，评论）
weixin_52026996的博客
11-09 2860
网上抓取京东数据的文章，现在要么无法抓取数据，要么只能抓取部分数据，本文将介绍如何抓取京东全站数据，包括商品信息、店铺信息，评论信息，分类信息等。 一、环境 OS：win10 python：3.5 scrapy：1.3.2 pymongo：3.2 pycharm 环境搭建，自行百度 二、数据库说明 产品分类 京东大概有1183个分类，这是除去了一些虚拟产品（话费、彩票、车票等）的分类，可以到如下网页查看： https://www.jd.com/allSort.aspx 我们也是从这个网址开始抓取。由于这些
爬取京东淘宝商品销量并可视化处理数据
KK_2018的技术博客_程序员不止是程序员
06-04 9381
目的：获取京东和淘宝的同一种手机型号的销量信息，获取三组，对比销量，并将数据可视化处理。 一、京东销量获取 如下面的例子： 京东销量： data-sku：对应下一个页面的productId defaultGoodCount：代表销量 commentCount评论数==总销量 fetchJSON_comment98({ "productAttr": null, "productCommentSummary": { "skuId": 100011336064, "defaultGoodCou
京东全站数据采集之Python中Scrapy框架！很详细！
爬遍所有网站
07-30 802
1.定义采集数据的存储结构 【存储结构说明】class CategoriesItem(Item)：存储京东类目信息class ProductsItem(Item)：存储京东商品信息class ShopItem(Item)：存储京东店铺信息class CommentSummaryItem(Item)：存储京东每个商品的评论概况信息class CommentItem(Item)：存储京东每个商品的评论基本信息class CommentImageItem(Item)：存储京东每个商品中每条评论的图像信息说明：
pythonallowpos_利用Python抓取并分析京东商品评论数据
weixin_39800062的博客
12-11 727
2.1 内容简介本章主要介绍如何利用Python抓取京东商城商品评论信息，并对这些评论信息进行分析和可视化。下面是要抓取的商品信息，一款女士文胸。这个商品共有红色，黑色和肤色等颜色， 70B到90D共18个尺寸，以及超过500条的购买评论。2.2 获取页面源码信息京东商品评论信息是由JS动态加载的，所以直接抓取商品详情页的URL并不能获得商品评论的信息。因此我们需要先找到存放商品评论信息的文件。这...
【python爬虫 系列】13.实战二 爬取京东信息
AI_LINNGLONG的博客
03-04 4121
实战2 爬取京东 1. 2.设计架构： 每个商品只有100页， 用线程池处理，不需要分区 对于耗费时间的评论获取使用Celery分布式获取 Celery使用redis中间件和存储 结果写入cs 3.写代码 3.0首先我们打开redis服务： 3.1headers与模块包 需要用到的模块 import requests from bs4 import BeautifulSoup import ...
电子商务物流服务质量评价以京东生鲜为例.pptx
10-08
本次演示以京东生鲜为例，对电子商务物流服务质量进行评价，并提出相应的优化措施。 在背景介绍部分，我们了解到电子商务物流服务是指在网络环境下，将商品从卖家所在地运送至买家所在地的一种服务方式。随着互联网...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

py爱好者~

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值