I want to run a Pythonic project using Python compilation (.pyc or __pycache__). In order to do that in Python2, I haven't any problem.
Here is a simplified example in a Python2 project:
Project tree:
test2
├── main.py
└── subfolder
├── __init__.py
└── sub.py
Compile:
python -m compileall test2
Project tree after the compile:
test2
├── main.py
├── main.pyc
└── subfolder
├── __init__.py
├── __init__.pyc
├── sub.py
└── sub.pyc
As you can see, several .pyc manually generated. Now I can run this project using main.pyc as fine, which has a relation with the sub.py:
python main.pyc
Out:
Hi
Bye
main.py content:
from subfolder import sub
print('Bye')
sub.py content:
print('Hi')
Now I want to retry this behavior in a Python3 project.
Here is a simplified asyncio (available in Python3) project:
Project tree:
test3
├── main.py
└── subfolder
├── __init__.py
└── sub.py
Compile:
python3 -m compileall test3
Project tree after the compile:
test3
├── main.py
├── __pycache__
│ └── main.cpython-36.pyc
└── subfolder
├── __init__.py
├── __pycache__
│ ├── __init__.cpython-36.pyc
│ └── sub.cpython-36.pyc
└── sub.py
As you can see, __pycache__ folders manually generated. But I cannot run this project using main.cpython-36.pyc which has a relation with subfolder:
cd test3/__pycache__
python3 main.cpython-36.pyc
Out (I expected that produced the Hi Bye message):
Traceback (most recent call last):
File "test3/main.py", line 2, in
ModuleNotFoundError: No module named 'subfolder'
main.py content:
import asyncio
from subfolder import sub
async def myCoroutine():
print("Bye")
def main():
loop = asyncio.get_event_loop()
loop.run_until_complete(myCoroutine())
loop.close()
main()
sub.py content:
print('Hi')
Question:
How can I run this project (above Python3 project) using __pycache__ folder?
Or
How can I run a Python3 project with the relation between subfolders using python compilation?
[NOTE]:
I cannot use the python compileall (Python2 compile) in the above Python3 project due to the asyncio method.
My Python(s) version is Python2.7 and Python3.6
解决方案
You can enforce the same layout of pyc-files in the folders as in Python2 by using:
python3 -m compileall -b test3
The option -b triggers the output of pyc-files to their legacy-locations (i.e. the same as in Python2).
After that you can once again use the compiled cache via:
python3 main.pyc
The way the loading of modules works since PEP-3147, it is impossible to use pyc-files from __pycache__ folder in the way you intend: If there is no *.py-file, the content of the __pycache__ is never looked-up. Here is the most important part of the workflow:
import foo
|
|
-- > [foo.py exists?] --- NO ----> [foo.pyc exists?] -- NO --> [ImportError]
| |
| YES
YES |--> [load foo.pyc]
|
|-> [look up in __pycache__]
That means, files from __pycache__ are only looked up, when a corresponding *.py-file can be found.