在爬取今日头条的图片时,由于今日头条用了ajax加载图片,所以,通过re模块来对链接进行提取,但是在提取的过程中,遇到了一点小问题,如图:
['"{\\"count\\":9,\\"sub_images\\":[{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/418185332_tt\\",\\"width\\":1200,\\"url_list\\":[{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/418185332_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb9.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/418185332_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb1.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/418185332_tt\\"}],\\"uri\\":\\"origin\\\\/tuchong.fullscreen\\\\/418185332_tt\\",\\"height\\":1200},{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/529858694_tt\\",\\"width\\":1200,\\"url_list\\":[{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/529858694_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb9.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/529858694_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb1.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/529858694_tt\\"}],\\"uri\\":\\"origin\\\\/tuchong.fullscreen\\\\/529858694_tt\\",\\"height\\":1200},{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/374079621_tt\\",\\"width\\":1200,\\"url_list\\":[{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/374079621_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb9.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/374079621_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb1.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/374079621_tt\\"}],\\"uri\\":\\"origin\\\\/tuchong.fullscreen\\\\/374079621_tt\\",\\"height\\":1200},{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/583008374_tt\\",\\"width\\":1200,\\"url_list\\":[{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/583008374_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb9.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/583008374_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb1.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/583008374_tt\\"}],\\"uri\\":\\"origin\\\\/tuchong.fullscreen\\\\/583008374_tt\\",\\"height\\":1200},{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/458686594_tt\\",\\"width\\":1200,\\"url_list\\":[{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/458686594_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb9.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/458686594_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb1.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/458686594_tt\\"}],\\"uri\\":\\"origin\\\\/tuchong.fullscreen\\\\/458686594_tt\\",\\"height\\":1200},{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/147390595_tt\\",\\"width\\":1200,\\"url_list\\":[{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/147390595_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb9.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/147390595_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb1.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/147390595_tt\\"}],\\"uri\\":\\"origin\\\\/tuchong.fullscreen\\\\/147390595_tt\\",\\"height\\":1200},{\\"url\\":\\"http:\\\\/\\\\/p1.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/22543963_tt\\",\\"width\\":1200,\\"url_list\\":[{\\"url\\":\\"http:\\\\/\\\\/p1.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/22543963_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/22543963_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb9.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/22543963_tt\\"}],\\"uri\\":\\"origin\\\\/tuchong.fullscreen\\\\/22543963_tt\\",\\"height\\":1200},{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/552992907_tt\\",\\"width\\":1200,\\"url_list\\":[{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/552992907_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb9.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/552992907_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb1.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/552992907_tt\\"}],\\"uri\\":\\"origin\\\\/tuchong.fullscreen\\\\/552992907_tt\\",\\"height\\":1200},{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/420610157_tt\\",\\"width\\":1200,\\"url_list\\":[{\\"url\\":\\"http:\\\\/\\\\/p3.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/420610157_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb9.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/420610157_tt\\"},{\\"url\\":\\"http:\\\\/\\\\/pb1.pstatp.com\\\\/origin\\\\/tuchong.fullscreen\\\\/420610157_tt\\"}],\\"uri\\":\\"origin\\\\/tuchong.fullscreen\\\\/420610157_tt\\",\\"height\\":1200}],\\"max_img_width\\":1200,\\"labels\\":[\\"\\\\u6444\\\\u5f71\\"],\\"sub_abstracts\\":[\\" \\\\u6444\\\\u5f71\\\\uff1a\\\\u61d2\\\\u4ebade\\\\u903b\\\\u8f91\\",\\" \\",\\" \\",\\" \\",\\" \\",\\" \\",\\" \\",\\" \\",\\" \\"],\\"sub_titles\\":[\\"\\\\u56fe\\\\u866b\\\\u8857\\\\u62cd\\\\u6444\\\\u5f71\\\\uff1a\\\\u8857\\\\u62cd06\\",\\"\\\\u56fe\\\\u866b\\\\u8857\\\\u62cd\\\\u6444\\\\u5f71\\\\uff1a\\\\u8857\\\\u62cd06\\",\\"\\\\u56fe\\\\u866b\\\\u8857\\\\u62cd\\\\u6444\\\\u5f71\\\\uff1a\\\\u8857\\\\u62cd06\\",\\"\\\\u56fe\\\\u866b\\\\u8857\\\\u62cd\\\\u6444\\\\u5f71\\\\uff1a\\\\u8857\\\\u62cd06\\",\\"\\\\u56fe\\\\u866b\\\\u8857\\\\u62cd\\\\u6444\\\\u5f71\\\\uff1a\\\\u8857\\\\u62cd06\\",\\"\\\\u56fe\\\\u866b\\\\u8857\\\\u62cd\\\\u6444\\\\u5f71\\\\uff1a\\\\u8857\\\\u62cd06\\",\\"\\\\u56fe\\\\u866b\\\\u8857\\\\u62cd\\\\u6444\\\\u5f71\\\\uff1a\\\\u8857\\\\u62cd06\\",\\"\\\\u56fe\\\\u866b\\\\u8857\\\\u62cd\\\\u6444\\\\u5f71\\\\uff1a\\\\u8857\\\\u62cd06\\",\\"\\\\u56fe\\\\u866b\\\\u8857\\\\u62cd\\\\u6444\\\\u5f71\\\\uff1a\\\\u8857\\\\u62cd06\\"]}"']
提取出来的文本全部都转义了的,解决方法也十分的简单,用replace来进行替换:
replace('\\\\','\\')
replace('\\"','"')
然后用json.loads(),将str 转换为dict
这样,就可以获得正常的json数据了