A Primer on virtualenv

What is virtualenv and Why Should You Use It?

Put simply (and copied from the project page), virtualenv is a tool to create isolated Python environments. Why is this good? You can create a new Python environment to run a Python/Django/whatever app and install all package dependencies into the virtualenv without affecting your system’s site-packages. Need to upgrade a package to see how it affects your app? Create a new virtualenv, install/copy your app into it, run your tests, and delete it when you are done. Just like git makes branching inexpensive and easy, virtualenv makes creating and managing new Python environments inexpensive and easy.

Show Me Some Code!

First, you need to install virtualenv. The typical way to install is from PyPi using easy_install. You can also use pip or install from source. We’ll use easy_install since it is…easy:

$ sudo easy_install virtualenv
Password:

Searching for virtualenv
Best match: virtualenv 1.3.3
Downloading http://pypi.python.org/packages/2.5/v/virtualenv/virtualenv-1.3.3-py2.5.egg#md5=fae350c941cd9eadf5e9a407c37a2e03Processing virtualenv-1.3.3-py2.5.egg
Adding virtualenv 1.3.3 to easy-install.pth file
Installing virtualenv script to /usr/local/bin

Using /Library/Python/2.5/site-packages/virtualenv-1.3.3-py2.5.egg
Processing dependencies for virtualenv
Finished processing dependencies for virtualenv

Now, you’re ready to create your virtualenv. BUT FIRST…you have a decision to make. Do you want this virtualenv to use packages from your system site-packages on install them in the virtualenv’s site-packages? By default, virtualenv will symlink to your system’s site-packages if you install a package in the virtualenv that is already installed on your system. If you want a totally (well, as much as is possible) isolated virtualenv, you’ll want to do the latter. To do this, you pass in the –no-site-packages switch when creating your virtualenv.

So, let’s create a virtualenv named for the project we are working on (mycoolproject) and tell it not to use the system site-packages but keep it’s own copy:

$ pwd
/Users/chris/Documents/clients

$ virtualenv --no-site-packages mycoolproject
New python executable in mycoolproject/bin/python
Installing setuptools............done.

$ ls mycoolproject
bin     include lib

I told virtualenv to not use the system site-packages and to create the virtualenv in a directory named mycoolproject. This directory didn’t exist so it created it, however, if it did exist virtualenv would install it’s files within it. virtualenv created three directories to use for running Python and installing packages into the virtualenv.

So, we have a new virtualenv, now what? Activate your virtualenv. To do this, change to the virtualenv directory you just created and source the activation script from the bin directory in your virtualenv:

$ cd mycoolproject
chris@chris.local [~/Documents/clients/mycoolproject]
$ source bin/activate
(mycoolproject)chris@chris.local [~/Documents/clients/mycoolproject]
$

I’ve included my full command prompt here so you can see that when you activate a virtualenv, it modifies your current prompt to prepend the name of the virtualenv in parentheses so that you know at a glance you have it active.

So, what happened here? Other than the prompt change, the main changes the activate script makes are:

VIRTUAL_ENV="/Users/chris/Documents/clients/mycoolproject"
export VIRTUAL_ENV

_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH

This sets up an environment variable with the virtualenv path and also modifies your path so the bin directory in the virtualenv is first. This is important since running python from the command line when your virtualenv is active runs it from there instead of your system path. This means through the magic of the Python site module the site-packages in your virtualenv will be in sys.path and not your system’s site-packages.

Installing Packages in a virtualenv

So, this is where the fun part starts. If you look at the bin directory in your virtualenv, you’ll see easy_install which has been modified to put eggs and packages in the virtualenv’s site-packages directory. So, once you activate your virtualenv installing a package in it is as easy as always:

$ easy_install ipython

Notice that I didn’t have to sudo this time since the files will all be installed in the virtualenv’slib/python2.5/site-packages directory which was created as my user account so I have the appropriate permissions.

You can also use pip to install packages in a virtualenv since it is virtualenv-aware. This is very handy since you can easily create a pip requirements file and use that to easily install all the various packages the software you are running in your virtualenv will need. Check out thePinax project’s installation docs for an example of this.

Tips for Working With virtualenv

• Use the virtualenvwrapper script to make working with and changing to/from multiple virtualenvs easy.
• For Django, after you install Django in your virtualenv, copy the django-admin.py from the Django package to the bin directory of your virtualenv so that you can still type django-admin.py startproject without specifying the full path to django-admin.py.
• If you use the --no-site-packages option you are starting with a bare Python install and will need to install all the things you have forgotten about since the first time you set your system up like PIL, ipython, ipdb, readline, etc. This is where a pip requirements file is a very good thing.
• If you are on OS X installing PIL from pip or easy_install may not work. To get around this just symlink the PIL directory from /Library/Python/2.5/site-packages to lib/python2.5/site-packages in your virtualenv.

Questions, Corrections, Tips?

I’m not a virtualenv expert, I just play one on the internet. If you see anything wrong with what I’ve said or have any suggestions or tips, please let me know in the comments or email chris@ this domain.