About Docker

Docker is an application that allows tools and their dependencies to be packaged into discrete runtime environments. These environments, containers, are built from images and are stored inside an image registry.

For an overview of Docker, please see the Docker website. Learn more about Docker images, containers and image registries below.

Images

A Docker image is a read-only file used to produce Docker containers. An image is composed of 'layers' of other images, and any change that is made to an image is carried out by adding a new layer.

The dependencies of Docker layers define an ordering, and any layer that does not depend on any other layer is called a base image. Typically this layer will describe the operating system for the environment.

Docker images can be uploaded to the CGC Image Registry, where they are referenced by repositories, which correspond to projects on the CGC. Once uploaded, you can run these tools on the CGC. Workflows will execute the tools in series, inside the containers that they are installed in.

Containers

A Docker container is a virtual private server. It can host software applications and their dependencies, which can be run from many diverse environments. Containers consist of an operating system, user-added files, and metadata.

Containers are built from images, but feature a read-write layer on top, so, unlike images, are interactive and can store state. This means that you can run applications inside them. After any executions in your Docker container, you can save a snapshot of the resulting state as another image.

Using Docker ensures that your tools will run on the CGC no matter what format they are in, and removes the possibility of conflicts between your local development environment and the execution environment of the CGC. Since there is no need to conform to any proprietary format when uploading your tools, you can easily run them elsewhere too.

Image Registries

An image registry stores Docker images. Each registry is organized into collections of images, called repositories.

Available registries include the CGC Image Registry and Docker Hub, Docker’s own image registry.

The CGC Image Registry

Docker images uploaded to the CGC Image Registry are further organized into repositories. Once the images are uploaded to the CGC Image Registry, you can run these tools on the CGC. Workflows will execute the tools in series inside their Docker containers.

Docker Hub - the Docker image registry

Docker hosts a number of images that you can use to build your containers. These are stored in an online registry called Docker Hub, where they are also organized into repositories. Docker Hub includes repositories that are are administrated by Docker, and contain images in which commonly-used software has been installed, including operating systems. These are typically used as base images. Docker Hub also hosts user repositories, containing images created and uploaded by Docker users. You are free to use images from any Docker Hub repository in building the image you upload to the CGC image registry.

You can also execute tools on the CGC that are contained in images stored in Docker Hub. However, storing your images in the CGC Image Registry rather than in Docker Hub will speed up processing time on the CGC, since the tools will be executed closer to the data they are processing.