python 使用execjs执行接js解密时报错execjs UnicodeDecodeError: ‘gbk’ codec can’t decode byte 0x86 in position 179: illegal multibyte sequence,stdout = stdout[0] IndexError: list index out of range
问题描述
运行js代码报错如下:
Exception in thread Thread-1:
Traceback (most recent call last):
File "D:\ProgramData\Anaconda3\envs\cs\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "D:\ProgramData\Anaconda3\envs\cs\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "D:\ProgramData\Anaconda3\envs\cs\lib\subprocess.py", line 1268, in _readerthread
buffer.append(fh.read())
UnicodeDecodeError: 'gbk' codec can't decode byte 0x86 in position 179: illegal multibyte sequence
Traceback (most recent call last):
File "E:/pythonProject/spider/企名片/企名片.py", line 37, in <module>
run()
File "E:/pythonProject/spider/企名片/企名片.py", line 33, in run
print(ctx.call("s",get_encrypt_data()))
File "D:\ProgramData\Anaconda3\envs\cs\lib\site-packages\execjs\_abstract_runtime_context.py", line 37, in call
return self._call(name, *args)
File "D:\ProgramData\Anaconda3\envs\cs\lib\site-packages\execjs\_external_runtime.py", line 92, in _call
return self._eval("{identifier}.apply(this, {args})".format(identifier=identifier, args=args))
File "D:\ProgramData\Anaconda3\envs\cs\lib\site-packages\execjs\_external_runtime.py", line 78, in _eval
return self.exec_(code)
File "D:\ProgramData\Anaconda3\envs\cs\lib\site-packages\execjs\_abstract_runtime_context.py", line 18, in exec_
return self._exec_(source)
File "D:\ProgramData\Anaconda3\envs\cs\lib\site-packages\execjs\_external_runtime.py", line 87, in _exec_
output = self._exec_with_pipe(source)
File "D:\ProgramData\Anaconda3\envs\cs\lib\site-packages\execjs\_external_runtime.py", line 103, in _exec_with_pipe
stdoutdata, stderrdata = p.communicate(input=input)
File "D:\ProgramData\Anaconda3\envs\cs\lib\subprocess.py", line 965, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "D:\ProgramData\Anaconda3\envs\cs\lib\subprocess.py", line 1318, in _communicate
stdout = stdout[0]
IndexError: list index out of range
因为window默认编码是gbk,Mac和Linux 默认是uft-8
所有要修改编码格式
解决办法
1修改源码
找到如图的subprocess.py,点进去ctrl+f搜索 encoding= 修改为utf-8
2添加代码
添加下面三行代码
import subprocess
from functools import partial
subprocess.Popen = partial(subprocess.Popen, encoding='utf-8')
#import execjs导入之前执行 修改encoding
import execjs
注意一定要放在import execjs之前
可以打断点看看是否修改成功
gevent 与javascript gbk冲突问题
更改gevent 源码中encoding编码时javascript 会报错:
'NoneType’ object has no attribute ‘queue’
不更改gevent 源码中encoding编码时 execjs包中_external_runtime.py 中p.communicate(input=input)会报错:
‘gbk’ codec can’t encode character
此时需要更改
from subprocess import Popen, PIPE
from functools import partial
Popen2 = partial(Popen, encoding="utf-8")
p = Popen2(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE, cwd=self._cwd, universal_newlines=True)