How to Distribute Commercial Python Applications

by Ville Laurikari on Monday, August 24, 2009

Bubble wrap.

Most of us in the software business are not in a position to release our source code to the public.  As much as I love free and open source software, I also understand it’s not part of all business models.  Python is enjoying commercial success, but perhaps less so in cases where end-users actually install the software.

Distributing applications written in Python can be tricky, especially if you don’t want to distribute your source code. I set out to find out how exactly do you distribute a Python program, sans source code, on various operating systems. It turns out there are quite a few alternatives to consider.

In each alternative, the first step is to compile your code to bytecode (.pyc or .pyo).  That’s what you’ll be shipping instead of the source code.

1. Using a preinstalled Python

Nowadays, many operating systems actually come with a sane Python interpreter preinstalled. Most Linux distributions ship with Python, and Mac OS X comes with Python, for example. Windows does not come with Python, nor do most commercial Unix systems.

Even if a preinstalled Python is available, it may not work for you.  Maybe it’s an older version, and you want to use the latest and greatest.  Another potential hurdle is that the Python version will be different on different operating systems. Mac OS X 10.5 (Leopard) comes with Python 2.5.1, Mac OS X 10.4 (Tiger) came with Python 2.3.5. Ubuntu 8.04 came with Python 2.5.2, Ubuntu 9.04 comes with Python 2.6.2… You get the point.

Sometimes modules are deprecated between versions and you’ll need to write stuff like this to suppress warnings:

try:
    import hashlib
    md5_new = hashlib.md5
except ImportError:
    import md5
    md5_new = md5.new

Varying Python versions may or may not be a problem for you application.  The example above is actually a fairly bening problem, and it’s easily solved.  If you ever need to support both Python 3 and Python 2, you will have bigger problems than just deprecated modules.

2. Freezing tools

Freezing tools compile Python programs to executables. If you’re distributing a small number of executables, a freeze tool might be just what you need.  There are quite a few of these around.  Disclaimer: I haven’t used most of these tools.  Caveat emptor.

py2exe

If you only care about Windows, py2exe fits the bill nicely.

py2exe is a Python Distutils extension which converts Python scripts into executable Windows programs, able to run without requiring a Python installation.

Applications such as BitTorrent and SpamBayes use py2exe – so it should be stable.  From what I’ve read, py2exe  works quite well.

py2app

py2app makes standard Mac OS X application bundles from Python scripts.  It’s “py2exe for Mac”.

cx_Freeze

cx_Freeze is similar to py2exe, but claims to be portable to both Windows and Unix systems.

bbfreeze

bbfreeze works on Windows and Unix, but not on OS X. Originally based on cx_Freeze.

Freeze

The Freeze tool ships with Python. It compiles Python programs to executables on Unix systems.

In my simple tests, it was able to handle a trivial hello world application pretty well, if we ignore the size of the resulting executable (several megabytes). Throwing anything more complex at it tended to fail miserably for various reasons. It seems to me that Freeze does not work very well out of the box.

PyInstaller

PyInstaller creates executables for Windows, Linux, and Irix.  Yeah, Irix.

3. Bundling CPython with your application

CPython is the default, most widely used implementation of Python.  It is written in C.  When people talk about the Python interpreter, they mean CPython unless they explicitly mention IronPython (for .NET and SilverLight), Jython (for JVM), Stackless Python (for concurrent programming), or something like that.

Sometimes the freezing tools aren’t a good choice.  They might lack platform coverage or could have other problems.  If you’re distributing a large collection of separate programs the overhead of bundling the full Python interpreter to each program will be prohibitive.  A good alternative is to bundle the Python interpreter with your application yourself.  You gotta love some DIY in the software business, right?

The first step is to bundle the CPython interpreter with your application. That’s fairly straightforward if you know what you’re doing. You simply include the Python interpreter with your code.

The second step is to cherry-pick what you need from the Python standard libraries. Tools like snakefood or modulegraph can help you do that. From what I can tell, these tools aren’t perfect, so be prepared to manually fix the results or patch the tools.

A third and optional step is to customize your Python distribution. For example, you might inline all bytecode directly into your executables instead of separate files. Then you can override the loading functions to pass the code directly from memory instead of loading from disk.

4. My recommendations

My first choice is to use the Python interpreter which comes with your target platform, if it has one.  Use distutils to install custom modules or extensions. If your application only supports Windows, use app2exe py2exe.

For complex scenarios, my recommendation is to go with bundling CPython yourself and steer clear of the freezers.  If a freezing tool works for you, awesome. You just saved yourself from a lot of trouble.  But if you need to cover a large number of platforms, or your application has special needs from the interpreter, then packaging CPython yourself seems to be the best way to go.

The added benefit of bundling CPython yourself is that you can customize the interpreter to your heart’s content.  Not that you’ll really need to, but it sounds like fun.  There’s nothing like making your own bubble wrap, complete with a custom hand-painted smiley face on each bubble.


Subscribe to Hacker Boss today!

Related posts:

  1. Approximate Regex Matching in Python
  2. Programming Problems in Disguise
  3. The Shell is Like a Dishwasher
  4. Platforms Come With a Culture

If you liked this, click here to receive new posts in a reader.
You should also follow me on Twitter here.

Comments on this entry are closed.

{ 17 comments }

Daniel August 25, 2009 at 18:56

Wow, I never know there are so many freezing tools. I’ve only used py2exe. I don’t think dealing with multiple 2.x Python versions would be a problem. Support both Python 3 and 2 is probably impossible in practice?

Ville Laurikari August 25, 2009 at 19:24

Daniel, the number of freezing tools surprised me too :)
My list probably is still missing some.

While supporting Python 2 and 3 is possible, I would advise against it. It seems like too much trouble than it’s worth.

Jonathan Hartley October 19, 2009 at 23:32

Hey. You recommend app2exe without ever mentioning what the heck it is. :-)

Ville Laurikari October 20, 2009 at 07:47

Oh. Oops. There’s no such thing as app2exe. I meant to say py2exe. Sorry about that, and thanks for pointing it out! Now fixed.

Adam Lintz November 15, 2009 at 19:02

Good article. Got here from reddit :) Thanks!

Ville Laurikari November 15, 2009 at 19:28

Thanks :) (Hello, reddit!) I’m writing a more detailed article on how exactly do you bundle the CPython interpreter with your application. Check back later if you’re interested in that.

Cosmic March 20, 2010 at 02:45

Thanks a lot! This will help me quite a bit with a program I was planning on distributing. :)

DG April 16, 2010 at 18:18

Great info. Has your CPython bundling article come to fruition? I would love to see it.

Viktor August 8, 2010 at 23:15

Wow, great article. Just what I was looking for. Thanks!

Cees Timmerman November 24, 2011 at 14:09

For speed and security, i would compile my program using Shed Skin, Iron Python, Cython, or PyPy. The psyco module also helps with speed if your code is old.

Eric December 13, 2012 at 23:32

Pyinstaller’s current list of supported platforms is:

Windows (32-bit and 64-bit),
Linux (32-bit and 64-bit),
Mac OS X (32-bit and 64-bit),
experimentally Solaris and AIX.

Chris Falck May 24, 2013 at 11:47

Thanks for this interesting article and for advising on your findings. Its much appreciated as I am currently looking to follow your advice, bundling CPython with some code I am creating for a client.
Any further advice is welcome

Pythoner May 17, 2014 at 11:01

It’s cool, but the only one works well for python3, which is cxfreeze, does not support Chinese!
Is there any solution?

Adam May 31, 2014 at 07:27

Beautiful post! Exactly what I was looking for and I wholeheartedly agree with the whole ‘just bundle the interpreter’ solution. Any chance you wrote that article on how to do it? I haven’t managed to find any decent documentation for it

Ville Laurikari June 12, 2014 at 11:33

Nope, never wrote that article – sorry.

Adam June 12, 2014 at 11:51

Hi Ville, thanks for the response. What a shame though. Any chance you can point me to some material on the topic? Any good tutorial/book/site?

Ville Laurikari June 26, 2014 at 11:28

Sorry for the delay. Your reply got caught in the nether. I’m on a biweekly routine of checking my “perhaps spam” messages.

I’m not aware of any recent tutorials or books on the subject. Then again, I haven’t looked recently.

What would I do?

First, I’d google for stuff, of course. I trust you’ve done all that already :)

If nothing useful comes up, I’d literally read the source. Download the Python source at https://www.python.org/downloads/source/ and see how it all works.

Yeah, it might take a day or two. Or a week or two, depending on how much you have to customise stuff. I wouldn’t know for sure before taking a good look.

Dropbox, for example, uses pyObjC on their Mac binaries, from what I can tell. http://ontwik.com/python/pycon-2011-how-dropbox-did-it-and-how-python-helped/

{ 5 trackbacks }

Previous post:

Next post: