python2 open utf8,python urllib2 utf-8编码

okay, I have: # -*- coding: utf-8 -*- in my python file.

the snippet:

opener = urllib2.build_opener()

opener.addheaders = [('User-agent', 'Mozilla/5.0')]

opener.addheaders = [('Accept-Charset', 'utf-8')]

f =opener.open(url)

doc = f.read().decode('utf-8')

The server response is: (via f.info())

Content-Type: text/html; charset=UTF-8

but i get the error:

UnicodeDecodeError: 'utf8' codec can't decode byte[...]: invalid continuation byte

What's wrong here?

解决方案

Try decoding the data using 'latin-1' to see what it looks like. What you're seeing indicates a UTF-8 decode error (see UnicodeDecodeError, invalid continuation byte ).

It would be helpful if you posted the result of list(f.read())[:100] so we can see the data.

FYI, putting # -*- coding: utf-8 -*- is unrelated to your issue. That encoding refers to the encoding of your python script itself, not the data it is handling :-)