Virtualenv and virtualenvwrapper for Python

It's often convenient to partition off your development space from your general system space: that way anything odd you do in your development space doesn't impact the wider system. There are a range of tools from chroot to virtualization that fulfill this sandbox need, but for Python development it makes sense to use something that is Python aware.

VirtualEnv example diagram
VirtualEnv environments

Virtualenv and Virtualenvwrapper let you create a separate environment where a parallel copy of all your Python dependencies can be installed. I particularly use these tools for Pelican. An, additional benefit of using virtualenv is you can use Pythons native installers rather than messing around with your system installer (e.g dpkg for me).

The diagram shows some of the ways you can use virtualenv. The first environment (Virtual Env 1) is a standard install of everything. With the second environment (Virtual Env 2) we've installed a different version of Python: commonly you'd do this to test that your code works with a later version. For the third one (Virtual Env 3) we've installed different application dependencies: again with this set-up you're isolating changes you're making to just one element of the stack making it easier to test.

Installing virtualenvwrapper

Virtualenvwrapper is an extension to virtualenv which makes it easier to use by storing all your environments in ~/.virtualenvs and providing a set of commands to manage the environments. As it's in the Ubuntu repositories the easiest way to install it is using apt-get:

$ sudo apt-get install virtualenvwrapper

The Ubuntu virtualenvwrapper install is enabled as a system wide capability, and it will automatically create a ~/.virtualenvs directory for you. See the README.Debian in /usr/share/doc/virtualenvwrapper/ for an explanation.

The next step is to define where the source code of your projects is going to be, in my case I use a 'workspace' directory for all my projects. To do that edit your .bashrc and add the following:

# Project definition needed for virtualenvwrapper
export PROJECT_HOME=$HOME/workspace

Then source .bashrc to get the new setting. To check it worked:

$ echo $PROJECT_HOME
/home/yourid/workspace

From now on all your projects will be created in /home/yourid/workspace/.

Create a project

Virtualenvwrapper has a lot of functionality that is worth exploring. Projects are really useful as they separate where your virtual environment is stored (under ~/.virtualenvs), from your project directories which have your code in them (in this case under ~/workspace/). This gives you a lot of flexibility to try different set-ups with your project code.

To create a project you do:

$ mkproject <name of project>

So something like:

$ mkproject virtenv1

This creates the virtual environment virtenv1 in .virtualenvs, it also creates the project workspace ~/workspace/virtenv1, finally it puts you into that virtualenv and moves your current working directory to the project space. Neat!

You can see that you're working under that environment because your prompt will be change to the name of the current project:

(virtenv1)$

Working on a project

To work on a project you use the workon command. If there are no arguments it will list the available environments:

$ workon
virtenv1
virtenv2

To work in a specific sandbox environment you use workon with the name of the project:

$ workon virtenv1
(virtenv1)$

To deactivate a sandbox and return to your standard environment:

(virtenv1)$ deactivate
$

Note that your prompt turns back to the standard one.

Finally, to delete a specific project you use the rmvirtualenv command.

Change to the project working directory

When an environment is activated you can use the command cdproject to change to the top of the project working directory:

(virtenv1)$ cdproject
~/workspace/virtenv1

Another clever trick 1 is to change the cd command so that rather than taking you to your home directory it takes you to the top of the virtualenv. To do this you have to take advantage of the userhooks in virtualenv, which let you alter what happens after activation, to set-up a shell command. Edit ~/.virtualenvs/postactivate and add:

    cd () {
        if (( $# == 0 ))
        then
            builtin cd $VIRTUAL_ENV
        else
            builtin cd "$@"
        fi
    }

    cd

Now when you use cd it will return you to the top of the virtualenv environment!

Attach a code directory to a virtualenv

If you have an existing code directory and want to attach it to a virtualenv to create a project then you use the setvirtualenvproject command:

$ setvirtualenvproject ~/.virtualenvs/newvirtualenv ~/workspace/existingcodedir

This allows you to switch between different versions of python or test out different dependencies with the same blog content or settings.

More virtualenv

Hopefully, that's enough of an introduction to successfully install virtualenvwrapper and to start using it. Aside from the documentation some good starting points for further exploration are:

Getting started with virtualenv by Bob Silverberg
Straightforward overview of virtualenv and the most common commands.
Tools of the Modern Python Hacker: Virtualenv, Fabric and Pip by Alex Clemesha
Great overview of the main Python web development tools - some of the commands have developed since this was written but strongly recommended.

If you think there's something really useful that I've missed or have some other way of using virtualenv then please comment away!


Posted in Python Thursday 03 October 2013
Tagged with blog pelican python