python object has no attribute_[求助]python爬虫环境初建 遇到问题AttributeError: 'X509' object has no attribute '...

[求助]python爬虫环境初建 遇到问题AttributeError: 'X509' object has no attribute 'get_extensi

代码很简单  豆瓣250第一页先找序号这个标签

from scrapy import Request

from scrapy.spiders import Spider

class m250Spider(Spider):

name = 'm250'

headers = {

'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36',

}

def start_requests(self):

url = 'http://movie.'

yield Request(url, headers=self.headers)

def parse(self, response):

movie_list= movies.xpath('.//div[@class="pic"]/em/text()').extract()

print movie_list

执行报错

2017-03-29 14:57:09 [scrapy] INFO: Scrapy 1.2.1 started (bot: firstspider)

2017-03-29 14:57:09 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'firstspider.spiders', 'SPIDER_MODULES': ['firstspider.spiders'], 'ROBOTSTXT_OBEY': True, 'BOT_NAME': 'firstspider'}

2017-03-29 14:57:09 [scrapy] INFO: Enabled extensions:

['scrapy.extensions.logstats.LogStats',

'scrapy.extensions.telnet.TelnetConsole',

'scrapy.extensions.corestats.CoreStats']

2017-03-29 14:57:09 [scrapy] INFO: Enabled downloader middlewares:

['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',

'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',

'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',

'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',

'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',

'scrapy.downloadermiddlewares.retry.RetryMiddleware',

'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',

'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',

'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',

'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',

'scrapy.downloadermiddlewares.chunked.ChunkedTransferMiddleware',

'scrapy.downloadermiddlewares.stats.DownloaderStats']

2017-03-29 14:57:09 [scrapy] INFO: Enabled spider middlewares:

['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',

'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',

'scrapy.spidermiddlewares.referer.RefererMiddleware',

'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',

'scrapy.spidermiddlewares.depth.DepthMiddleware']

2017-03-29 14:57:09 [scrapy] INFO: Enabled item pipelines:

[]

2017-03-29 14:57:09 [scrapy] INFO: Spider opened

2017-03-29 14:57:09 [scrapy] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)

2017-03-29 14:57:09 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023

2017-03-29 14:57:09 [scrapy] DEBUG: Crawled (403)

2017-03-29 14:57:09 [scrapy] DEBUG: Redirecting (301) to

Error during info_callback

Traceback (most recent call last):

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 208, in doRead

return self._dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 214, in _dataReceived

rval = self.protocol.dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 415, in dataReceived

self._checkHandshakeStatus()

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 335, in _checkHandshakeStatus

self._tlsConnection.do_handshake()

--- ---

File "D:\Python27\lib\site-packages\twisted\internet\_sslverify.py", line 1148, in infoCallback

return wrapped(connection, where, ret)

File "D:\Python27\lib\site-packages\scrapy\core\downloader\tls.py", line 52, in _identityVerifyingInfoCallback

verifyHostname(connection, self._hostnameASCII)

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 44, in verify_hostname

cert_patterns=extract_ids(connection.get_peer_certificate()),

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 66, in extract_ids

for i in range(cert.get_extension_count()):

exceptions.AttributeError: 'X509' object has no attribute 'get_extension_count'

2017-03-29 14:57:09 [twisted] CRITICAL: Error during info_callback

Traceback (most recent call last):

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 208, in doRead

return self._dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 214, in _dataReceived

rval = self.protocol.dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 415, in dataReceived

self._checkHandshakeStatus()

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 335, in _checkHandshakeStatus

self._tlsConnection.do_handshake()

--- ---

File "D:\Python27\lib\site-packages\twisted\internet\_sslverify.py", line 1148, in infoCallback

return wrapped(connection, where, ret)

File "D:\Python27\lib\site-packages\scrapy\core\downloader\tls.py", line 52, in _identityVerifyingInfoCallback

verifyHostname(connection, self._hostnameASCII)

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 44, in verify_hostname

cert_patterns=extract_ids(connection.get_peer_certificate()),

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 66, in extract_ids

for i in range(cert.get_extension_count()):

exceptions.AttributeError: 'X509' object has no attribute 'get_extension_count'

2017-03-29 14:57:09 [scrapy] DEBUG: Retrying ]

Error during info_callback

Traceback (most recent call last):

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 208, in doRead

return self._dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 214, in _dataReceived

rval = self.protocol.dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 415, in dataReceived

self._checkHandshakeStatus()

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 335, in _checkHandshakeStatus

self._tlsConnection.do_handshake()

--- ---

File "D:\Python27\lib\site-packages\twisted\internet\_sslverify.py", line 1148, in infoCallback

return wrapped(connection, where, ret)

File "D:\Python27\lib\site-packages\scrapy\core\downloader\tls.py", line 52, in _identityVerifyingInfoCallback

verifyHostname(connection, self._hostnameASCII)

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 44, in verify_hostname

cert_patterns=extract_ids(connection.get_peer_certificate()),

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 66, in extract_ids

for i in range(cert.get_extension_count()):

exceptions.AttributeError: 'X509' object has no attribute 'get_extension_count'

2017-03-29 14:57:10 [twisted] CRITICAL: Error during info_callback

Traceback (most recent call last):

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 208, in doRead

return self._dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 214, in _dataReceived

rval = self.protocol.dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 415, in dataReceived

self._checkHandshakeStatus()

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 335, in _checkHandshakeStatus

self._tlsConnection.do_handshake()

--- ---

File "D:\Python27\lib\site-packages\twisted\internet\_sslverify.py", line 1148, in infoCallback

return wrapped(connection, where, ret)

File "D:\Python27\lib\site-packages\scrapy\core\downloader\tls.py", line 52, in _identityVerifyingInfoCallback

verifyHostname(connection, self._hostnameASCII)

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 44, in verify_hostname

cert_patterns=extract_ids(connection.get_peer_certificate()),

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 66, in extract_ids

for i in range(cert.get_extension_count()):

exceptions.AttributeError: 'X509' object has no attribute 'get_extension_count'

2017-03-29 14:57:10 [scrapy] DEBUG: Retrying ]

Error during info_callback

Traceback (most recent call last):

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 208, in doRead

return self._dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 214, in _dataReceived

rval = self.protocol.dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 415, in dataReceived

self._checkHandshakeStatus()

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 335, in _checkHandshakeStatus

self._tlsConnection.do_handshake()

--- ---

File "D:\Python27\lib\site-packages\twisted\internet\_sslverify.py", line 1148, in infoCallback

return wrapped(connection, where, ret)

File "D:\Python27\lib\site-packages\scrapy\core\downloader\tls.py", line 52, in _identityVerifyingInfoCallback

verifyHostname(connection, self._hostnameASCII)

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 44, in verify_hostname

cert_patterns=extract_ids(connection.get_peer_certificate()),

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 66, in extract_ids

for i in range(cert.get_extension_count()):

exceptions.AttributeError: 'X509' object has no attribute 'get_extension_count'

2017-03-29 14:57:10 [twisted] CRITICAL: Error during info_callback

Traceback (most recent call last):

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 208, in doRead

return self._dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\internet\tcp.py", line 214, in _dataReceived

rval = self.protocol.dataReceived(data)

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 415, in dataReceived

self._checkHandshakeStatus()

File "D:\Python27\lib\site-packages\twisted\protocols\tls.py", line 335, in _checkHandshakeStatus

self._tlsConnection.do_handshake()

--- ---

File "D:\Python27\lib\site-packages\twisted\internet\_sslverify.py", line 1148, in infoCallback

return wrapped(connection, where, ret)

File "D:\Python27\lib\site-packages\scrapy\core\downloader\tls.py", line 52, in _identityVerifyingInfoCallback

verifyHostname(connection, self._hostnameASCII)

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 44, in verify_hostname

cert_patterns=extract_ids(connection.get_peer_certificate()),

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 66, in extract_ids

for i in range(cert.get_extension_count()):

exceptions.AttributeError: 'X509' object has no attribute 'get_extension_count'

2017-03-29 14:57:10 [scrapy] DEBUG: Gave up retrying ]

2017-03-29 14:57:10 [scrapy] ERROR: Error downloading ]

2017-03-29 14:57:10 [scrapy] INFO: Closing spider (finished)

2017-03-29 14:57:10 [scrapy] INFO: Dumping Scrapy stats:

{'downloader/exception_count': 3,

'downloader/exception_type_count/twisted.web._newclient.ResponseNeverReceived': 3,

'downloader/request_bytes': 1428,

'downloader/request_count': 5,

'downloader/request_method_count/GET': 5,

'downloader/response_bytes': 611,

'downloader/response_count': 2,

'downloader/response_status_count/301': 1,

'downloader/response_status_count/403': 1,

'finish_reason': 'finished',

'finish_time': datetime.datetime(2017, 3, 29, 6, 57, 10, 515000),

'log_count/CRITICAL': 3,

'log_count/DEBUG': 6,

'log_count/ERROR': 1,

'log_count/INFO': 7,

'response_received_count': 1,

'scheduler/dequeued': 4,

'scheduler/dequeued/memory': 4,

'scheduler/enqueued': 4,

'scheduler/enqueued/memory': 4,

'start_time': datetime.datetime(2017, 3, 29, 6, 57, 9, 375000)}

2017-03-29 14:57:10 [scrapy] INFO: Spider closed (finished)

[color=#000000]感觉主要问题是

File "D:\Python27\lib\site-packages\service_identity\pyopenssl.py", line 66, in extract_ids

for i in range(cert.get_extension_count()):

exceptions.AttributeError: 'X509' object has no attribute 'get_extension_count'

pyopenssl是用pip装的16.2.0版本  不知为什么会报错，求各位大神指点指点，谢啦[/color]

[此贴子已经被作者于2017-3-29 15:11编辑过]