Custom Software with Docker

This article introduces Docker's key concepts and explains how it is utilized at Qarnot. It also outlines which profiles and constants can be configured to run a Docker image within a task.

# Introduction

At Qarnot, we leverage Docker to run tasks efficiently on our distributed infrastructure, enabling seamless scalability and deployment of custom software.

# What is a container ?

A container is a self-contained, lightweight, and executable package that includes everything required to run a piece of software—code, runtime, system tools, libraries, and settings. Containers ensure consistency and efficiency across different environments.

Containers have the following characteristics

Self-contained: Encapsulate the application and its dependencies to ensure consistent operation across environments.
Lightweight: Share the host system’s kernel, making them more resource-efficient than virtual machines.
Portable: Can run on any system supporting container execution, regardless of underlying OS or hardware.
Ephemeral: Designed for short-lived processes that can be quickly started, stopped, and destroyed.

Containers are now the standard for packaging applications in modern IT and cloud environments. At Qarnot, we rely on Docker, one of the most popular containerization platforms. To dive deeper into Docker, visit the Docker Documentation..

# Docker and Qarnot

Docker

Docker is the primary technology Qarnot uses to run computations on its platform. When you execute a task on Qarnot, it consists of three main components:

A Docker image: Packages the software and its dependencies.
User's data: Mounted inside the container when it starts.
Task settings: These include task parameters like network access, hardware configuration, and specific constraints, managed through Profiles
.

# Where to find docker images ?

There are several options for sourcing Docker images for use on Qarnot:

Public repositories: The Docker Hub or other public registries provide pre-built images not specific to Qarnot. Learn more about #[/choosing-your-software/bring-your-software-with-docker/off-the-shelf-docker-images](using off-the-shelf Docker Images).
Qarnot Container Registry: Qarnot's secured and integrated container registry .
Private registry: You can also use your own private Docker registry.

# Docker profiles

This section covers Docker profiles, which are necessary for using Docker containers on Qarnot. These profiles define how Docker interacts with Qarnot’s infrastructure, including network access, hardware, and cluster settings. If you want to learn how to use Docker images in your tasks, refer to our guide.

To use Docker in your tasks, choose a Docker-enabled profile. Below is a list of profiles and their descriptions:

Profile	Description
docker-batch	For non-networked tasks that run on a single node.
docker-network	For tasks that require network access.
docker-network-ssh	Allows SSH access in addition to network capabilities.
docker-cluster	Enables tasks to run in a cluster of nodes without network access.
docker-cluster-network	Clustered tasks with network capabilities.
docker-cluster-network-ssh	Cluster tasks with both network and SSH access.
docker-cluster-roce	Clustered tasks with high-performance network (RDMA over Converged Ethernet).
docker-cluster-roce-network-ssh	Combines RDMA, network, and SSH access for clustered tasks.
docker-nvidia-batch	Optimized for tasks running on NVIDIA GPUs without network access.
docker-nvidia-network	Optimized for NVIDIA GPU tasks that require network access.

# Constants to Use in Docker Profiles

When using Docker profiles on Qarnot, several constants must be configured to define the behavior of your task:

Constant	Default Value	Description
DOCKER_SRV	https://registry-1.docker.io	Docker registry to use. Change if using a custom registry.
DOCKER_REPO	library/ubuntu	Docker image repository. E.g., library/ubuntu.
DOCKER_TAG	latest	Docker image tag. E.g., latest, trusty, etc.
DOCKER_REGISTRY_LOGIN		Docker Hub login for private images.
DOCKER_REGISTRY_PASSWORD	${QARNOT_SECRET__DOCKER_REGISTRY_PASSWORD}	Docker Hub password for private images. Use the secret to hide sensitive data.
QARNOT_SECRET__DOCKER_REGISTRY_PASSWORD		Password for private Docker Hub images, hidden for security. using a “QARNOT_SECRET_” constant, these are read only, constant readable only by your task.
DOCKER_USER		The user for running commands. Leave blank for default.
DOCKER_HOST		Hostname inside the container.
DOCKER_CMD		Command to execute inside the container. Honors ENTRYPOINT and CMD in the Dockerfile.
DOCKER_SHUTDOWN_CMD	/bin/true	Command to run when the task is canceled, terminated after 30 seconds.
DOCKER_PROGRESS1-3		Regex for progress tracking. See developer documentation for details.
RESOURCES_PATH	/job	Path where task resources are available inside the container.
DOCKER_WORKDIR	${DOCKER_WORKDIR}	Task working directory. This does not change the resource/output path found in $TASK_PATH.
QARNOT_DISABLE_CPU_BOOST	false	Set to ‘true’ to disable CPU boost, reducing performance but providing consistency over time.

# Cluster and SSH Profiles

If you’re using a cluster profile (e.g., docker-cluster-xyz), the following constants are also important:

Constant	Description
DOCKER_CMD_MASTER	Command to run on the Master node (node 0). Honors ENTRYPOINT and CMD.
DOCKER_CMD_WORKER	Command for worker nodes (all others). Honors ENTRYPOINT and CMD.

If your profile includes SSH access (e.g., docker-network-ssh), you’ll need to configure SSH keys:

Constant	Description
DOCKER_SSH	Your public SSH key for access to the container.

For further information on using Docker on Qarnot, please refer to our detailed guides .

# Related Articles

To start using docker images on Qarnot please read the following guides