Create and upload a Docker image

In order to use your own tools or workflows on the CGC, you need to install each tool in an individual Docker image, and then upload the image to the CGC Image Registry or to Docker Hub.

Prerequisites

Before you can create and upload a Docker image, you need to install Docker and make sure it is running.

Steps

To create and upload a Docker image:

  1. Run docker login cgc-images.sbgenomics.com and enter your CGC credentials. Don't forget: enter your authentication token when prompted for a password.
  2. Open a Docker base image.
  3. Install your tool in the image.
  4. Commit your image.
  5. Push your image to the CGC image registry.

Open a Docker image

If you are installing a tool, you'll need to open a new base image to install it on. On the other hand, if you are modifying a tool that you have already uploaded, you can open the image containing the tool from the CGC registry.
To install a tool, start by opening a base image. This can be any base image from Docker Hub, but starting with a plain operating system image like Ubuntu is generally recommended. To use this base image, enter:

docker run -ti ubuntu

To pull an image from the CGC registry, enter the repository that the image is stored in, followed by the image tag (if any), separated by a colon. The full repository path has the format cgc-images.sbgenomics.com/<username>/<repository_name>[:tag]. For example, if the user rfranklin wanted to open the image tagged 1.3 from her project picard, she would enter

docker run -ti cgc-images.sbgenomics.com/rfranklin/picard:1.3

Install your tools inside a Docker container

Inside a container you can install your chosen tool or modify an existing tool. Do this in the way that you would normally, using methods appropriate for your tool, e.g. apt-get.

📘

Installing Software

Methods to install software vary considerably across different tools and packages. Explaining these methods is out of scope for this documentation. For any given software package, consult its documentation regarding how to install. Follow those instructions within your Docker container. Then return to this tutorial.

When you've finished installing a tool, leave the container by typing exit.

root@container$ exit

Commit your image

After you exit the container, you can commit the image of it.
First, list all your local containers, so that you can commit an image (snapshot) of the ubuntu container that you just created. The -a option here lists all containers, include those that are not currently running; you should see a container that was just created recently. This is the one you want.

$ docker ps -a

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS               NAMES

c52b82b497eb        ubuntu              "/bin/bash"              2 minutes ago       Exited (0) 2 minutes ago                        kickass_liskov
ae194ed75819        debian              "/bin/bash"              7 hours ago         Exited (0) 7 hours ago                          jovial_swanson
34ff1377fbee        hello-world         "/hello"                 26 hours ago        Exited (0) 26 weeks ago                         high_almeida
54240578230c        c0bfb9c8e377        "/bin/sh -c '/usr/gam"   26 hours ago        Exited (0) 26 weeks ago                         serene_pare
517904a42f3d        docker/whalesay     "cowsay hhLinked Appl"   27 hours ago        Exited (0) 27 weeks ago                         romantic_bhaskara
1aad55d740cd        docker/whalesay     "cowsay docker/whales"   27 hours ago        Exited (0) 27 weeks ago                         cocky_bhaskara
7bfb18e0d18a        hello-world         "/hello"                 28 hours ago        Exited (0) 28 weeks ago                         stupefied_williams

Grab the CONTAINER ID of the ubuntu image that was created 2 minutes ago. It's c52b82b497eb.
Now, we'll commit an image of that container. This also gives you the opportunity to name your image. You must name it with the format cgc-images.sbgenomics.com/<username>/<repository_name>[:tag]. For example, if the user rfranklin wanted to commit her changes to a container in a repository named picard, with tag 1.4 she would use cgc-images.sbgenomics.com/rfranklin/picard:1.4.

Commit the image as follows:

$ docker commit c52b82b497eb cgc-images.sbgenomics.com/rfranklin/picard:1.4

If you want to confirm that the image has been named, you can list all of your local images:

$ docker images 

REPOSITORY                                TAG                 IMAGE ID            CREATED             VIRTUAL SIZE

cgc-images.sbgenomics.com/rfranklin/picard    1.4                 0fe5d1d1aaec        10 minutes ago      125.1 MB
rfranklin/test                            latest              0fe5d1d1aaec        18 minutes ago      125.1 MB
debian                                    latest              7a01cc5f27b1        7 hours ago         125.1 MB
ubuntu                                    latest              6cc0fc2a5ee3        8 months ago        187.9 MB
hello-world                               latest              0a6ba66e537a        8 months ago        960 B
docker/whalesay                           latest              ded5e192a685        8 months ago        247 MB

Push your image

To push your image to the CGC image registry, run the command docker push cgc-images.sbgenomics.com/<username>/<repository_name>[:tag], where <username>/<repository_name>[:tag] refers to the image that you have already committed. For example:

$ docker push cgc-images.sbgenomics.com/rfranklin/picard:1.4
The push refers to a repository [cgc-images.sbgenomics.com/rfranklin/picard] (len: 1)
container@root: pushed
1.4: digest: sha256:afc9023f29719ffd361cdcbc334fe4ec2c041997ee501a15a86ed7f6e9277008 size: 3990

The progress of the image upload will be shown in the terminal. When it has completed, you will see the message pushed.

Remove a local Docker image

If you want to delete a Docker image, use docker rmi cgc-images.sbgenomics.com/<username>/<repository_name>[:tag]. For example:

$ docker rmi cgc-images.sbgenomics.com/rfranklin/picard:1
Deleted 02c8c0913b94a09053dccded886512b77fbd2ecbe18930d1b059bb573f13afd1