最近我写的爬虫在爬取数据的时候变得不太好用了,两三天前还是可以正常爬取数据的,今天运行的时候返回了空值。我打开源代码,将抓取的网页内容print出来看了一下,发现返回了这样一个网页:
<html style="height:100%">
<head>
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<meta name="format-detection" content="telephone=no">
<meta name="viewport" content="initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
</head>
<body style="margin:0px;height:100%">
<iframe src="/_Incapsula_Resource?SWUDNSAI=9&xinfo=14-73717949-0%202CNN%20RT%281531788981588%200%29%20q%280%20-1%20-1%20-1%29%20r%280%20-1%29%20B12%288%2c881023%2c0%29&incident_id=810000430074365444-466505767058014910&edet=12&cinfo=08000000" frameborder=0 width="