Keeping on-path Python scripts in the brave new virtualenv world

Update: It looks like uv supports per script run time environments specified in file which achieves the same thing as this tool and is more standard. I suggest you use this.

I have a confession to make. I used to use my system Python for a lot of things… and it wasn’t really that much of an issue. Sure for real work I would use virtualenvs but for the day-to-day tools that made my life easier, I would just shove a script on my path, use system Python to run it and things worked okay.

But it seems like these days are over, as Ubuntu has claimed ownership of python3 and told you that you can’t install things with pip . It’s an understandable move, but it does require me to reform my erstwhile ways… a bit.

If I was truly incorrigible, I would just create a virtualenv, link to its python, calling my version something like my-python and then use this for all my scripts. Or even activate a virtualenv in my bashrc. But… I felt like I should at least show a little willing so I’m using a different approach

Using PATH with a few virtualenvs

The approach that I decided to go with is still having Python scripts on my path - but have different directories on my path maintain their own Python. For example I have a directory with tools for command line video processing and in this directory I have version of python called video-python which I use in this directory.

I’ll do a quick tour of how this works. First I create a virtualenv in my directory (importantly, on my path) with this python3 -m venv env. Then I create a shim “python” called video-python which calls the Python in env/bin/python. I need to use a little bash magic here because symlinks appear to break virtualenvs. Here is the contents of

#!/bin/bash

here="$(dirname ${BASH_SOURCE[0]})"
$here/env/bin/python "$@"

I also do the same with pip to create video-pip.

Then in Python scripts that use this Python I have the following shbang (which works because video-python is on my path)

#!/usr/bin/env video-python

Turning this into a tool

Because I have quite a few directories like this, I wrapped up the logic to create these virtualenvs in a little tool called venv-python.

You can install this with pipx.

pipx install venv-python

You can then run this in the directory where you want to create a Python like so

cd bins/video
venv-python video

A note of the “correct” way of doing this

So there is another way of doing this, and it’s what I do when I make “big” tools that either need testing or want to be shared. That to create a Python project with an entry point in setup.py

    entry_points={
        'console_scripts': ['venv-python=venv_python.main:main']
    },
)

You can then install this using pipx with pipx install -f -e . . This allows you to edit the code underlying your script and works well enough. But you need to edit these entry points and rerun this command every thing you want some new scripts.

This is all just wiring so there are probably other tricks you can use.

Finishing up

I hope this is interesting, if it is you might like to follow me on X or my blog where I talk about reading tools and agency and also other technical stuff I do. I might suggest you could also like:

1. This post about sending code to ipython kernel in a virtualenv from Emacs

2. This command-line tool to use numpy from the shell

3. This library that customizes pdb to be more useable by adding custom commands.