python爬虫
gongchengshiv
这个作者很懒,什么都没留下…
展开
-
把scrapyd设置为系统后台服务和启动项
一、设置为系统后台服务 1、新建文件/etc/init.d/scrapyd,名称为scrapyd #!/bin/bash PORT=6800 HOME="/var/scrapyd" BIN="/usr/local/bin/scrapyd" pid=`netstat -lnopt | grep :$PORT | awk '/python/{gsub(/\/python/,"",$7);pri原创 2017-08-21 13:56:30 · 2519 阅读 · 3 评论 -
基于scrapy的智联职位爬取
1、项目截图 2、setting.py项目配置文件 # -*- coding: utf-8 -*- # Scrapy settings for zhaopin_zhilian project # # For simplicity, this file contains only settings considered important or # commonly used. You原创 2017-08-29 17:05:41 · 2838 阅读 · 2 评论 -
linux安装scrapy出错'x86_64-Linux-gnu-gcc' failed with exit status 1
出错内容:'x86_64-linux-gnu-gcc' failed with exit status 1 解决办法: sudo apt-get build-dep python-lxml sudo pip install lxml --upgrade sudo apt-get install build-essential libssl-dev libffi-dev python原创 2017-08-17 15:27:39 · 367 阅读 · 0 评论 -
测试ip是否可用(ip代理时)
import telnetlib try: telnetlib.Telnet('192.168.112.95', port='680', timeout=20) except: print('connect failed') else: print('success')原创 2017-08-21 17:58:34 · 3291 阅读 · 1 评论