Chances are, you probably don’t need to jump the hoops described below unless you have a very specific set of requirements in mind.

The goal is to create a reproducable and easy to configure environment to use Jupyter notebooks - to quickly experiment ideas and create notebooks that can be shared.

1. Install Jupyter in a virtual environment

install jupyter in virtualenv

Create and setup the virtual environment:

python3 -m virtualenv env-name
source env-name/bin/activate
pip3 install jupyter

Run Jupyter Notebook:

jupyter notebook

While it is very easy to install modules by sourcing the correct virtualenv and using pip, there are a few drawbacks to this approach. We have to install the Jupyter Notebook for each new environment created. If used as a server, this would become quite tedious and difficult to maintain - there may be a need to use multiple environemnts at once with different users.

2. Install Jupyter as global system package

install jupyter as ssytem package

We can centralize the Jupyter Notebook installation using the system package manager and then create virtual environments separately. For example, on Debian based systems, we could use apt:

sudo apt install python3 python3-virtualenv jupyter
jupyter notebook

This is convenient because there is only one Jupyter Notebook installation - it can be accessed on localhost:8888. However, as soon as we try to install modules in different virtual environments, it gets a bit tricky.

It is not possible to create and use a virtual environment from within a notebook itself. For example:

! python -m virtualenv venv
! source venv/bin/activate
! pip install numpy

When run in the notebook, “!” specifies that this is a shell command. This will create the virtual environment, but will not use that environment - when attempting to install modules with pip, you will likely get an externally-managed-environment error.

To understand this, we need to first discuss how Jupyter Notebook works. Internally, it has a component called the IPython Kernel. This is responsible for communicating with the Python interpreter on the system. The ipykernel is included by default when Jupyter Notebook is installed, but it can also be installed independently.

how ipython kernel integrates

Virtual environments cannot be nested! When using pip with the default ipython kernel, the python environment managed globally by the system is used.

To make a virtual environments available to Jupyter Notebook, the ipykernel module should be installed inside the virtual environment:

python3 -m virtualenv --system-site-packages env-name
source env-name/bin/activate
pip install ipykernel
python3 -m ipykernel install --user --name=env-name --display-name "Python (env-name)"

The --system-site-packages flag is particularly important if you depend on globally installed modules. For example, matplotlib has to be specially packaged for ARM platoforms like Raspberry Pi and installed using the system package manager. This flag ensures that these modules can be found.

Since the Jupyter Notebook installation is global, we can easily run the above commands from the interface by creating a new terminal.

jupyter new menu

When creating a new notebook, you can select Python (env-name) to use a particular virtual environment.

To list all kernels and remove a particular kernel:

jupyter-kernelspec list
jupyter-kernelspec uninstall env-name

We still have a bit of a problem - this is running directly on the server. If something goes wrong, it might affect other services. This is also not very easily portable.

3. Install Jupyter on Docker

Using docker, this can be containerized!

install jupyter in docker

This Dockerfile uses Alpine linux as the base image and installs python, pip, virtualenv and jupyter-notebook using the Alpine package manager apk. The last command starts the notebook. This probably isn’t very secure - you might want to modify the flags based on your security model.

FROM alpine:3.22
RUN apk add python3 py3-pip py3-virtualenv jupyter-notebook
EXPOSE 8888
CMD ["jupyter-notebook", "--ip=0.0.0.0", "--no-browser", "--allow-root", "--NotebookApp.token='your-very-long-and-secure-passphrase'"]

Now, creating the following docker-compose.yaml file and running docker compose up will build the image and expose Jupyter Notebook on localhost:4444. To manage the virtual environments, we’ll once again use a terminal and follow the steps in the previous section.

services:
  jupyter:
    container_name: jupyter
    build: .
    ports:
      - 4444:8888

To reproduce this configuration on a different service, we just need to copy the Dockerfile and docker-compose.yaml.