电商api全境,Python网络爬虫与数据采集

1 爬虫基本概述

1.1 爬虫是什么
网络爬虫(Crawler)又称网络蜘蛛,或者网络机器人(Robots). 它是一种按照一定的规则,自动地抓取万维网信息的程序或者脚本。换句话来说,它可以根据网页的链接地址自动获取网页内容。如果把互联网比做一个大蜘蛛网,它里面有许许多多的网页,网络蜘蛛可以获取所有网页的内容。
爬虫是一个模拟人类请求网站行为, 并批量下载网站资源的一种程序或自动化脚本。

1.2 爬虫可以做什么

  • 搜索引擎
  • 采集金融数据
  • 采集商品数据
  • 采集竞争对手的客户数据
  • 采集行业相关数据,进行数据分析
  • 刷流量

如何运用爬虫采集商品数据 

前往测试

item_get-获得淘宝商品详情

请求参数

请求参数:num_iid=520813250866&is_promotion=1

参数说明:num_iid:淘宝商品ID
is_promotion:是否获取取促销价

请求示例

Curl PHP PHPsdk JAVA C# Python

# coding:utf-8
"""
Compatible for python2.x and python3.x
requirement: pip install requests
"""
from __future__ import print_function
import requests
# 请求示例 url 默认请求参数已经做URL编码
url = "https://api-gw.onebound.cn/taobao/item_get/?key=<您自己的apiKey>&secret=<您自己的apiSecret>&num_iid=520813250866&is_promotion=1"
headers = {
    "Accept-Encoding": "gzip",
    "Connection": "close"
}
if __name__ == "__main__":
    r = requests.get(url, headers=headers)
    json_obj = r.json()
    print(json_obj)

相应示例

{
    "item": {
      "num_iid": "520813250866",
      "title": "三刃木折叠刀过安检创意迷你钥匙扣钥匙刀军刀随身多功能小刀包邮",
      "desc_short": "",
      "price": 25.8,
      "total_price": 0,
      "suggestive_price": 0,
      "orginal_price": "25.80",
      "nick": "欢乐购客栈",
      "num": "832",
      "min_num": 0,
      "detail_url": "http://item.taobao.com/item.htm?id=520813250866",
      "pic_url": "//img.alicdn.com/imgextra/i4/2596264565/TB2p30elFXXXXXQXpXXXXXXXXXX_!!2596264565.jpg",
      "brand": "三刃木",
      "brandId": "4036703",
      "rootCatId": "50013886",
      "cid": "50014822",
      "favcount": "4824",
      "fanscount": "1469",
      "crumbs": [],
      "created_time": "",
      "modified_time": "",
      "delist_time": "",
      "desc": "<p style=\"text-align:center;\"><span style=\"color:#ff0000;\"><span style=\"font-size:24.0px;\"><strong>小店所有产品都支持刻字,如需刻字,拍之前联系客服即可。</strong></span></span></p> <p style=\"text-align:center;\"><span style=\"font-size:18.0px;\"><strong>炎炎夏日来临,一把随身携带便携式的折叠小刀,<span style=\"color:#ff0000;\">带开瓶器功能</span>,喝酒不用愁。(钥匙刀不带开瓶器功能)</strong></span></p> <p style=\"text-align:center;\"><strong style=\"color:#000000;line-height:1.5;\"><span style=\"font-size:18.0px;\"><span style=\"line-height:18.0px;\">可以当吊牌项链装饰,也可当钥匙扣挂饰, 水果刀 开瓶器 户外防身.</span></span></strong></p> <p style=\"text-align:center;\"><strong style=\"line-height:1.5;\"><span style=\"font-size:24.0px;\">部分客户跟我们反映链子质量还不够好,因此小店已重新订购一批质量更好的链子赠送,<span style=\"color:#ff0000;\">加量不加价</span>,只为让你买的更舒心,戴的更放心。</span></strong></p> <p style=\"text-align:center;\"><span style=\"font-size:36.0px;\"><strong>购买就送<span style=\"color:#ff0000;\">工具刀卡和链子</span>一条</strong></span></p> <p style=\"text-align:center;\"><span style=\"font-size:36.0px;\"><strong><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2Sci2jXXXXXXFXpXXXXXXXXXX_!!2596264565.png\" size=\"611x442\"> </strong></span></p> <p style=\"text-align:center;\"><span style=\"font-size:18.0px;\"><strong>璀璨钻石套餐包含【SK016D钥匙扣+GJ019C折叠刀+GJ017D工具卡】</strong></span><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2wWohmXXXXXX8XXXXXXXXXXXX_!!2596264565.jpg\" size=\"750x600\" /></p> <p style=\"text-align:center;\"><span style=\"font-size:18.0px;\"><strong>超凡大师套餐包括【304不锈钢钥匙扣+GJ019C折叠刀+GJ017D工具卡】</strong></span><a href=\"//item.taobao.com/item.htm?id=520798805877\" target=\"_blank\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2_uiXnFXXXXXBXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"float:none;margin:0.0px;\" /></a></p> <p style=\"text-align:center;\">&nbsp;</p> <p style=\"text-align:center;\"><span style=\"font-size:18.0px;\"><strong>最强王者套餐包括【钛钢钥匙扣+GJ019C折叠刀+GJ017D工具卡】<a href=\"//item.taobao.com/item.htm?id=520801586805\" target=\"_blank\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2Gm9xnFXXXXbmXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"float:none;margin:0.0px;\" /></a> </strong></span></p> <p style=\"text-align:center;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB28Ox4b77OyuJjSsplXXXqdpXa_!!2596264565.jpg\" size=\"749x553\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i1/2596264565/TB2.mTddVXXXXbeXpXXXXXXXXXX_!!2596264565.jpg\" style=\"line-height:1.5;\" size=\"750x653\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB21Ro.jl0lpuFjSszdXXcdxFXa_!!2596264565.jpg\" size=\"750x653\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2q9CelVXXXXcUXXXXXXXXXXXX_!!2596264565.jpg\" size=\"750x340\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB21EX9lVXXXXaXXpXXXXXXXXXX_!!2596264565.jpg\" size=\"750x416\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2udCylVXXXXXgXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i1/2596264565/TB2EARxjB8lpuFjSspaXXXJKpXa_!!2596264565.jpg\" size=\"750x515\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2ssuwlVXXXXafXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2nAHqgyC9MuFjSZFoXXbUzFXa_!!2596264565.jpg\" size=\"750x606\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2ahCelVXXXXc_XXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2w1JnjwRkpuFjy1zeXXc.6FXa_!!2596264565.jpg\" size=\"750x469\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2C902lVXXXXbnXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i1/2596264565/TB2sGR3lVXXXXblXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i1/2596264565/TB2ZBGxlVXXXXXMXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i1/2596264565/TB2MjWklVXXXXcaXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2UgV3lVXXXXbdXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2ip5XlVXXXXX2XpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2sMTBdVXXXXXlXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2d3HfdVXXXXahXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2AVbBdVXXXXXkXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2nf_wdVXXXXaMXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2dLYddVXXXXbtXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2H1_adVXXXXbWXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2eBzsdVXXXXbuXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2dOTndVXXXXcCXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2fK2tdVXXXXbkXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB29zjedVXXXXaFXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2i7rmdVXXXXcTXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\" /></p>",
      "item_imgs": [
        {
          "url": "//img.alicdn.com/imgextra/i4/2596264565/TB2p30elFXXXXXQXpXXXXXXXXXX_!!2596264565.jpg"
        },
        {
          "url": "//img.alicdn.com/imgextra/i2/2596264565/TB2onxRlVXXXXcDXpXXXXXXXXXX_!!2596264565.jpg"
        },
        {
          "url": "//img.alicdn.com/imgextra/i1/2596264565/TB2a.x.lVXXXXXPXpXXXXXXXXXX_!!2596264565.jpg"
        },
        {
          "url": "//img.alicdn.com/imgextra/i2/2596264565/TB2iWE6bKZkyKJjSszbXXblwFXa_!!2596264565.jpg"
        },
        {
          "url": "//img.alicdn.com/imgextra/i4/2596264565/TB2j2cTXib_F1JjSZFzXXc6KXXa_!!2596264565.jpg"
        }
      ],
      "item_weight": "0",
      "item_size": "",
      "location": "广东深圳",
      "post_fee": "",
      "express_fee": "0.00",
      "ems_fee": "",
      "shipping_to": "广东广州白云区",
      "has_discount": "false",
      "video": [],
      "is_virtual": "",
      "sample_id": "",
      "is_promotion": "false",
      "props_name": "1627207:1347647754:颜色分类:长方形带开瓶器+送工具刀卡+链子;1627207:1347647753:颜色分类:椭圆形带开瓶器+送工具刀卡+链子;1627207:1195392087:颜色分类:GJ018X钥匙刀+送工具刀卡+链子;1627207:1331112595:颜色分类:超凡大师套餐【送工具卡+链子】;1627207:1331112594:颜色分类:最强王者套餐【送工具卡+链子】;1627207:1331264247:颜色分类:璀璨钻石套餐【送工具卡+链子】",
      "prop_imgs": {
        "prop_img": [
          {
            "properties": "1627207:1347647754",
            "url": "//img.alicdn.com/imgextra/i3/2596264565/TB2.XeblVXXXXXkXpXXXXXXXXXX_!!2596264565.jpg"
          },
          {
            "properties": "1627207:1347647753",
            "url": "//img.alicdn.com/imgextra/i4/2596264565/TB2dTrjdVXXXXXBXpXXXXXXXXXX_!!2596264565.jpg"
          },
          {
            "properties": "1627207:1195392087",
            "url": "//img.alicdn.com/imgextra/i2/2596264565/TB2j22kdVXXXXXdXpXXXXXXXXXX_!!2596264565.jpg"
          },
          {
            "properties": "1627207:1331112595",
            "url": "//img.alicdn.com/imgextra/i4/2596264565/TB2_uiXnFXXXXXBXXXXXXXXXXXX_!!2596264565.jpg"
          },
          {
            "properties": "1627207:1331112594",
            "url": "//img.alicdn.com/imgextra/i4/2596264565/TB2Gm9xnFXXXXbmXXXXXXXXXXXX_!!2596264565.jpg"
          },
          {
            "properties": "1627207:1331264247",
            "url": "//img.alicdn.com/imgextra/i3/2596264565/TB2wWohmXXXXXX8XXXXXXXXXXXX_!!2596264565.jpg"
          }
        ]
      },
      "property_alias": "1627207:1347647754:长方形带开瓶器+送工具刀卡+链子;1627207:1347647753:椭圆形带开瓶器+送工具刀卡+链子;1627207:1195392087:GJ018X钥匙刀+送工具刀卡+链子;1627207:1331112595:超凡大师套餐【送工具卡+链子】;1627207:1331112594:最强王者套餐【送工具卡+链子】;1627207:1331264247:璀璨钻石套餐【送工具卡+链子】",
      "props": [
        {
          "name": "品牌",
          "value": "三刃木"
        },
        {
          "name": "产地",
          "value": "中国"
        },
        {
          "name": "颜色分类",
          "value": "长方形带开瓶器+送工具刀卡+链子,椭圆形带开瓶器+送工具刀卡+链子,GJ018X钥匙刀+送工具刀卡+链子,超凡大师套餐【送工具卡+链子】,最强王者套餐【送工具卡+链子】,璀璨钻石套餐【送工具卡+链子】"
        },
        {
          "name": "吊牌价",
          "value": "46"
        },
        {
          "name": "功能数量",
          "value": "5个及以下"
        },
        {
          "name": "货号",
          "value": "GJ019C"
        },
        {
          "name": "附加功能",
          "value": "开瓶器,刀,螺丝刀,钥匙圈,其他"
        }
      ],
      "total_sold": "1",
      "skus": {
        "sku": [
          {
            "price": "39",
            "orginal_price": "39.00",
            "properties": "1627207:1347647754",
            "properties_name": "1627207:1347647754:颜色分类:长方形带开瓶器+送工具刀卡+链子",
            "quantity": "104",
            "sku_id": "3166598625985"
          },
          {
            "price": "39",
            "orginal_price": "39.00",
            "properties": "1627207:1347647753",
            "properties_name": "1627207:1347647753:颜色分类:椭圆形带开瓶器+送工具刀卡+链子",
            "quantity": "370",
            "sku_id": "3166598625984"
          },
          {
            "price": "25.8",
            "orginal_price": "25.80",
            "properties": "1627207:1195392087",
            "properties_name": "1627207:1195392087:颜色分类:GJ018X钥匙刀+送工具刀卡+链子",
            "quantity": "34",
            "sku_id": "3144644292458"
          },
          {
            "price": "73.8",
            "orginal_price": "73.80",
            "properties": "1627207:1331112595",
            "properties_name": "1627207:1331112595:颜色分类:超凡大师套餐【送工具卡+链子】",
            "quantity": "157",
            "sku_id": "3161300228970"
          },
          {
            "price": "91.8",
            "orginal_price": "91.80",
            "properties": "1627207:1331112594",
            "properties_name": "1627207:1331112594:颜色分类:最强王者套餐【送工具卡+链子】",
            "quantity": "0",
            "sku_id": "3161300228969"
          },
          {
            "price": "63.8",
            "orginal_price": "63.80",
            "properties": "1627207:1331264247",
            "properties_name": "1627207:1331264247:颜色分类:璀璨钻石套餐【送工具卡+链子】",
            "quantity": "167",
            "sku_id": "3161107666655"
          }
        ]
      },
      "seller_id": "2596264565",
      "sales": "1",
      "shop_id": "127203758",
      "props_list": {
        "1627207:1347647754": "颜色分类:长方形带开瓶器+送工具刀卡+链子",
        "1627207:1347647753": "颜色分类:椭圆形带开瓶器+送工具刀卡+链子",
        "1627207:1195392087": "颜色分类:GJ018X钥匙刀+送工具刀卡+链子",
        "1627207:1331112595": "颜色分类:超凡大师套餐【送工具卡+链子】",
        "1627207:1331112594": "颜色分类:最强王者套餐【送工具卡+链子】",
        "1627207:1331264247": "颜色分类:璀璨钻石套餐【送工具卡+链子】"
      },
      "seller_info": {
        "title": "欢乐购客栈",
        "shop_name": "欢乐购客栈",
        "sid": "127203758",
        "zhuy": "//shop127203758.taobao.com",
        "level": "12",
        "shop_type": "C",
        "user_num_id": "2596264565",
        "nick": "欢乐购客栈",
        "cid": null,
        "delivery_score": "4.8 ",
        "item_score": "4.8 ",
        "score_p": "4.8 "
      },
      "tmall": "false",
      "error": "",
      "warning": "",
      "url_log": [],
      "stuff_status": "",
      "shopinfo": {
        "shop_name": "欢乐购客栈",
        "shop_id": "127203758"
      },
      "data_from": "mtop",
      "method": "item_mtop:item_get",
      "promo_type": null,
      "props_img": {
        "1627207:1347647754": "//img.alicdn.com/imgextra/i3/2596264565/TB2.XeblVXXXXXkXpXXXXXXXXXX_!!2596264565.jpg",
        "1627207:1347647753": "//img.alicdn.com/imgextra/i4/2596264565/TB2dTrjdVXXXXXBXpXXXXXXXXXX_!!2596264565.jpg",
        "1627207:1195392087": "//img.alicdn.com/imgextra/i2/2596264565/TB2j22kdVXXXXXdXpXXXXXXXXXX_!!2596264565.jpg",
        "1627207:1331112595": "//img.alicdn.com/imgextra/i4/2596264565/TB2_uiXnFXXXXXBXXXXXXXXXXXX_!!2596264565.jpg",
        "1627207:1331112594": "//img.alicdn.com/imgextra/i4/2596264565/TB2Gm9xnFXXXXbmXXXXXXXXXXXX_!!2596264565.jpg",
        "1627207:1331264247": "//img.alicdn.com/imgextra/i3/2596264565/TB2wWohmXXXXXX8XXXXXXXXXXXX_!!2596264565.jpg"
      },
      "rate_grade": "",
      "shop_item": [],
      "relate_items": []
    },
    "secache": "d8bd20b8f0d9df02dd030dd374110240",
    "secache_time": 1591841298,
    "secache_date": "2020-06-11 10:08:18",
    "translate_status": "error",
    "translate_time": 0.001,
    "language": {
      "current_lang": "zh-CN",
      "source_lang": "cn"
    },
    "error": "",
    "reason": "",
    "error_code": "0000",
    "cache": 1,
    "api_info": "today:7033 max:10000",
    "execution_time": 0.211,
    "server_time": "Beijing/2020-06-11 10:12:27",
    "call_args": {
      "num_iid": "520813250866",
      "area_id": "440111"
    },
    "api_type": "taobao",
    "translate_language": "zh-CN",
    "translate_engine": "google_cn",
    "server_memory": "8.26MB",
    "request_id": "15ee1930b4e098"
  }

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值