爬虫 代理问题

1

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\anzhuang\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 682, in runfile
    execfile(filename, namespace)
  File "D:\anzhuang\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 71, in execfile
    exec(compile(scripttext, filename, 'exec'), glob, loc)
  File "D:/python/useProxyAndSurf.py", line 30, in <module>
    doc = url_user_agent(url)
  File "D:/python/useProxyAndSurf.py", line 23, in url_user_agent
    html = urllib2.urlopen(req)
  File "D:\anzhuang\Anaconda\lib\urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "D:\anzhuang\Anaconda\lib\urllib2.py", line 431, in open
    response = self._open(req, data)
  File "D:\anzhuang\Anaconda\lib\urllib2.py", line 449, in _open
    '_open', req)
  File "D:\anzhuang\Anaconda\lib\urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "D:\anzhuang\Anaconda\lib\urllib2.py", line 1227, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "D:\anzhuang\Anaconda\lib\urllib2.py", line 1197, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 10060] >

问题原因：代理出现问题。

类似的简单解决方法
方法1：

import urllib2

proxy_support = urllib2.ProxyHandler({"http":"http://61.233.25.166:80"})
opener = urllib2.build_opener(proxy_support)
urllib2.install_opener(opener)

html = urllib2.urlopen("http://www.google.com").read()
print html

方法2

import urllib2

#proxy = "61.233.25.166:80"
proxy = "YOUR_PROXY_GOES_HERE"

proxies = {"http":"http://%s" % proxy}
url = "http://www.google.com/search?q=test"
headers={'User-agent' : 'Mozilla/5.0'}

proxy_support = urllib2.ProxyHandler(proxies)
opener = urllib2.build_opener(proxy_support, urllib2.HTTPHandler(debuglevel=1))
urllib2.install_opener(opener)

req = urllib2.Request(url, None, headers)
html = urllib2.urlopen(req).read()
print html