爬虫资料收集

1.互联网网站的反爬虫策略浅析
http://robbinfan.com/blog/11/anti-crawler-strategy#comments
2.当爬虫被拒绝时(Access Denied)
http://www.cnblogs.com/FengYan/archive/2012/07/31/2614335.html
3.curl伪造ip访问网站
http://www.poboke.com/study/curl-forge-ip-to-visit-the-website.html
4.构造HTTP请求Header实现“伪造来源IP”
http://zhangxugg-163-com.iteye.com/blog/1663687
5.社会化海量数据采集爬虫框架搭建
http://www.lanceyan.com/tech/arch/snscrawler.html
6.python 爬虫抓取心得分享----有使用代理IP的代码，要特别留意
http://yxmhero1989.blog.163.com/blog/static/112157956201311821444664/