Using Jupyter with custom environments

This post demonstrates running a “global” user-scoped Jupyter notebook that can be used to manage a collection of notebooks across many projects.

Using Jupyter in this way builds on top of a common workflow using IPython for interactive development, which has been a common process for me during development.

An example of when this is especially useful, would be when applying logic to a data construct which I am not familiar with by allowing interactive exploration of the data and then rolling up the code from that exercise into the actual code.

IPython is great, though I often find myself wanting to refer back to code I have previously written or to share that code with someone else.

Enter Jupyter.

In this context, Jupyter is basically a web-based IPython which provides an intuitive way to manage code snippets and experiments.

My plan is to use Jupyter as a central source of reference code. To be useful for that it has to be aware of the development projects I am working on which we’ll achieve by registering our development environments with a global Jupyter instance. Primarily I am using pipenv which this example demonstrates.

Firstly, install jupyter, I use pipx:

$ pipx install jupyter-core
$ pipx install notebook

Now go to your project and add ipykernel to the dev dependencies of the project you want to use in Jupyter:

$ cd workspace/project1
$ pipenv install --dev ipykernel

Next, add the project as an environment:

$ pipenv run python -m ipykernel install --user --name project1 --display-name "Project 1"
Installed kernelspec project1 in /home/jonathan/.local/share/jupyter/kernels/project1

$ grep project1 /home/jonathan/.local/share/jupyter/kernels/project1/kernel.json
"/home/jonathan/.local/share/virtualenvs/project1-vqFkHOwT/bin/python",

Notice in the output, that the kernel has been installed into the user configuration pointing at the pipenv managed virtual environment. This will allow you to select and experiment in your project with all dependencies available. Exactly what we wanted!

Finally, you can start your global Jupyter notebook or refresh a running notebook and see that the new kernel is available:

$ jupyter notebook

Kernels