Install NVIDIA Container Toolkit on Ubuntu 24.04

The NVIDIA Container Toolkit is a set of tools and libraries that enable GPU support in containerized applications, such as those run with Docker or other container runtimes. It allows containers to access and use the NVIDIA GPUs available on the host system, which is crucial for running GPU-accelerated workloads like machine learning, or scientific computing. This tutorial explains how to install NVIDIA Container Toolkit on Ubuntu 24.04.

Prepare environment

Ensure that Docker is installed on the system. If you're using Ubuntu, you'll find the installation instructions in the post.

Install NVIDIA Container Toolkit

Download the GPG key and place it in the specified directory:

sudo wget -qO /etc/apt/keyrings/nvidia-container-toolkit.asc https://nvidia.github.io/libnvidia-container/gpgkey

Add the NVIDIA Container Toolkit repository:

echo "deb [signed-by=/etc/apt/keyrings/nvidia-container-toolkit.asc] https://nvidia.github.io/libnvidia-container/stable/deb/amd64 /" | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

Update the package lists:

sudo apt update

Install the NVIDIA Container Toolkit:

sudo apt install -y nvidia-container-toolkit

Restart the Docker service to apply changes:

sudo service docker restart

Testing NVIDIA Container Toolkit

Run a GPU-enabled container to verify the installation:

docker run -it --rm --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark

If the installation is successful, you should see an output similar to the following:

...
131072 bodies, total time for 10 iterations: 73.040 ms
= 2352.122 billion interactions per second
= 47042.441 single-precision GFLOP/s at 20 flops per interaction

Uninstall NVIDIA Container Toolkit

If you decided to completely remove the NVIDIA Container Toolkit and related dependencies, run the following command:

sudo apt purge --autoremove -y nvidia-container-toolkit

Restart Docker to finalize the removal:

sudo service docker restart

Remove GPG key and repository:

sudo rm -rf /etc/apt/keyrings/nvidia-container-toolkit.asc

sudo rm -rf /etc/apt/sources.list.d/nvidia-container-toolkit.list

Remove NVIDIA container runtime configuration:

sudo rm -rf /etc/nvidia-container-runtime