python反序列化的坑

今天遇到python一个坑，调用memcached的get_man报错：

 

memcache2 get :1:photo:latest:forward:5855687 cost 2857.6438427
转发 <#59066487>
unpickle :1:photo:latest:forward:5855687 FAIL __import__() argument 1 must be string without null bytes, not str

 

同样一个key,一个是通过get，一个是通过get_many，结果报错。code如下：

 

# -*- coding: utf-8 -*-

from django.core.management import setup_environ
import settings
setup_environ(settings)

key = 'photo:latest:forward:5855687'
from django.core.cache import cache
print cache.get(key)
print cache.get_many([key])

 跟进代码发现问题出在get_man的反序列化上：

 

def get_multi(self,key):
        result = cacheService.getMulti(key)
        if result is None:
            return []
        else:
            result_dict = {}
            for k,val in result.items():
                try:

                    file = StringIO(val)
                    _up = unpickler(file)
                    val = _up.load()
                    result_dict[k] = val 
                except Exception,e:
                    print 'unpickle %s FAIL  %s'%(k,e)
                    pass
            return result_dict

 尝试把序列化改成：

result_dict[k] = pickle.loads(val)

 报错信息变了：

unpickle :1:photo:latest:forward:5855687 FAIL  must be string, not unicode

 

难道是val的type不是str，是unicode？ 通过print type(val) 打印出来，果然是unicode。