Multiprocessing Freeze Support in Python

You can add support for multiprocessing when freezing your code via the multiprocessing.freeze_support() function.

In this tutorial you will discover how to add freeze support for multiprocessing in your Python program.

What is Freezing Code

You can package your code with a Python runtime.

For example, you can package your program up as a .exe for Windows or .app for macOS that includes both your program and all its code, as well as the Python runtime and relevant standard libraries required to execute it.

It allows your program to run without the end user having to install anything else, such as Python.

It requires that your code be transformed into C-code, called freezing, performed automatically by a freezing tool.

It converts Python byte code to C arrays; with a C compiler you can embed all your modules into a new program, which is then linked with the standard Python modules. […] It then compiles the generated C code and links it with the rest of the Python interpreter to form a self-contained binary which acts exactly like your script.

The benefit to the user is that your program will work out of the box with the intended version of Python. A downside is that the application will be much larger, e.g. megabytes, and required updates to the program or the included Python runtime will require an update of the entire package.

It is common to freeze end-user applications written in Python intended to be distributed on Windows and macOS, although frozen versions of a program can be distributed to all platforms.

Python includes a tool to freeze Python scripts called freeze.py.

Alternately, you can freeze your program and create a distribution of your application using a third-party tool, such as:

• py2exe: py2exe is a Python Distutils extension which converts Python scripts into executable Windows programs, able to run without requiring a Python installation.Spice.
• PyInstaller: PyInstaller bundles a Python application and all its dependencies into a single package. The user can run the packaged app without installing a Python interpreter or any modules.
• cx_Freeze: cx_Freeze creates standalone executables from Python scripts, with the same performance, is cross-platform and should work on any platform that Python itself works on.

Now that we know what freezing code means, let’s look at a possible problem when using multiprocessing.

Multiprocessing and Freezing Code

When a program is frozen in order to be distributed, some features of Python are not included or disabled by default.

This is for performance and/or security reasons.

One feature that is disabled when freezing a Python program is multiprocessing.

That is, we cannot create new python processes via multiprocessing.Process instances when freezing our program for distribution.

Creating a process in a frozen application results in a RuntimeError.

How can we use multiprocessing in frozen code?

Add Multiprocessing Support When Freezing Code

We can add support for multiprocessing in our program when freezing code via the multiprocessing.freeze_support() function.

Add support for when a program which uses multiprocessing has been frozen to produce a Windows executable. (Has been tested with py2exe, PyInstaller and cx_Freeze.)

—  MULTIPROCESSING — PROCESS-BASED PARALLELISM

For example:

1 2 3

... # enable support for multiprocessing multiprocessing.freeze_support()

This function will allow a frozen program to create and start new processes via the multiprocessing.Process class when the program is frozen for distribution on Windows.

If the function is called and the program is not frozen for distribution, then it has no effect.

If the function is called on platforms other than Windows, it has no effect.

Calling freeze_support() has no effect when invoked on any operating system other than Windows. In addition, if the module is being run normally by the Python interpreter on Windows (the program has not been frozen), then freeze_support() has no effect.

—  MULTIPROCESSING — PROCESS-BASED PARALLELISM

This function must be called as the first line of the program, e.g. the first line after protecting the entry point.

One needs to call this function straight after the if __name__ == ‘__main__’ line of the main module.

—  MULTIPROCESSING — PROCESS-BASED PARALLELISM

For example:

1 2 3 4 5

# protect the entry point if __name__ == '__main__': # enable support for multiprocessing multiprocessing.freeze_support() # ...

Now that we know how to enable multiprocessing support in frozen code, let’s look at a worked example.

---

Free Python Multiprocessing Course

Download your FREE multiprocessing PDF cheat sheet and get BONUS access to my free 7-day crash course on the multiprocessing API.

Discover how to use the Python multiprocessing module including how to create and start child processes and how to use a mutex locks and semaphores.

Learn more
 

---

Example of Adding Freeze Support for Multiprocessing

We can add freeze support for multiprocessing.

In this example we will run a custom function in a child process. The main process will first add freeze support and then configure and start the child process.

First, we can define a function to run in a child process.

The function will report a message then sleep for two seconds.

The task() function below implements this.

1 2 3 4 5 6

# function executed in a new child process def task():     # report message     print('Hello from child process', flush=True)     # block for a moment     sleep(2)

Next, in the main process, we will first add support for multiprocessing while the program is frozen.

1 2 3

... # add freeze support for multiprocessing freeze_support()

We can create and configure a new multiprocessing.Process instance to run our custom function.

1 2 3

... # configure the child process child = Process(target=task)

Next, we can start the new child process, then wait for the child process to finish.

1 2 3 4 5

... # start the child process child.start() # wait for the child process to terminate child.join()

Tying this together, the complete example is listed below.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

# SuperFastPython.com # example of multiprocessing program with freeze support from time import sleep from multiprocessing import Process from multiprocessing import freeze_support # function executed in a new child process def task():     # report message     print('Hello from child process', flush=True)     # block for a moment     sleep(2) # protect the entry point if __name__ == '__main__':     # add freeze support for multiprocessing     freeze_support()     # configure the child process     child = Process(target=task)     # start the child process     child.start()     # wait for the child process to terminate     child.join()

First, save the above program as a Python script with the filename “app.py“.

We can then freeze the program.

You can use your preferred freezing tool, such as a third-party tool or freeze.py.

In this case, we will use the freeze tool provided with Python. This is in the Tools/ directory of your Python installation and has the name freeze.py.

For example:

1	/path/to/python/examples/Tools/freeze/freeze.py

If you need help with this tool, see documentation in the Python Wiki.

• Freeze, Python Wiki.

To freeze our program, call the freeze.py tool and specify the name of our app.py script.

For example:

1	python examples/Tools/freeze/freeze.py app.py

This will convert the program to C code.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

... freezing urllib.parse ... freezing urllib.request ... freezing urllib.response ... freezing uu ... freezing warnings ... freezing weakref ... freezing webbrowser ... freezing xml ... freezing xml.etree ... freezing xml.etree.ElementPath ... freezing xml.etree.ElementTree ... freezing xml.parsers ... freezing xml.parsers.expat ... freezing xmlrpc ... freezing xmlrpc.client ... freezing zipfile ... freezing zipimport ... generating table of frozen modules Now run "make" to build the target: app

Next, compile the C code.

This can be achieved by calling make.

Note, this assumes you have a C compiler and build system installed, which is typically the case by default for many modern development workstations, e.g. macOS and Linux.

For example:

1

make

This will compile the generated C-code into a binary for your platform.

Then run the program.

For example:

1

./app.py

This will execute the native program.

Internally, the program will start a new child Python process and report a message.

1	Hello from child process

You can learn more about multiprocessing in the tutorial:

• Multiprocessing in Python: The Complete Guide

Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps

Further Reading

This section provides additional resources that you may find helpful.

Python Multiprocessing Books

• Python Multiprocessing Jump-Start, Jason Brownlee (my book!)
• Multiprocessing API Interview Questions
• Multiprocessing API Cheat Sheet

I would also recommend specific chapters in the books:

• Effective Python, Brett Slatkin, 2019.
  • See: Chapter 7: Concurrency and Parallelism
• High Performance Python, Ian Ozsvald and Micha Gorelick, 2020.
  • See: Chapter 9: The multiprocessing Module
• Python in a Nutshell, Alex Martelli, et al., 2017.
  • See: Chapter: 14: Threads and Processes

Guides

• Python Multiprocessing: The Complete Guide
• Python Multiprocessing Pool: The Complete Guide
• Python ProcessPoolExecutor: The Complete Guide

APIs

• multiprocessing — Process-based parallelism
• PEP 371 - Addition of the multiprocessing package

References

• Thread (computing), Wikipedia.
• Process (computing), Wikipedia.