In order to use your own tools or workflows on the CGC, you need to install each tool in an individual Docker image, and then upload the image to the CGC Image Registry or to Docker Hub.
Prerequisites
Before you can create and upload a Docker image, you need to install Docker and make sure it is running.
Steps
To create and upload a Docker image:
- Run
docker login cgc-images.sbgenomics.comand enter your CGC credentials. Don't forget: enter your authentication token when prompted for a password. - Open a Docker base image.
- Install your tool in the image.
- Commit your image.
- Push your image to the CGC image registry.
Open a Docker image
If you are installing a tool, you'll need to open a new base image to install it on. On the other hand, if you are modifying a tool that you have already uploaded, you can open the image containing the tool from the CGC registry.
To install a tool, start by opening a base image. This can be any base image from Docker Hub, but starting with a plain operating system image like Ubuntu is generally recommended. To use this base image, enter:
docker run -ti ubuntu
To pull an image from the CGC registry, enter the repository that the image is stored in, followed by the image tag (if any), separated by a colon. The full repository path has the format cgc-images.sbgenomics.com/<username>/<repository_name>[:tag]. For example, if the user rfranklin wanted to open the image tagged 1.3 from her project picard, she would enter
docker run -ti cgc-images.sbgenomics.com/rfranklin/picard:1.3
Install your tools inside a Docker container
Inside a container you can install your chosen tool or modify an existing tool. Do this in the way that you would normally, using methods appropriate for your tool, e.g. apt-get.
Installing Software
Methods to install software vary considerably across different tools and packages. Explaining these methods is out of scope for this documentation. For any given software package, consult its documentation regarding how to install. Follow those instructions within your Docker container. Then return to this tutorial.
When you've finished installing a tool, leave the container by typing exit.
root@container$ exit
Commit your image
After you exit the container, you can commit the image of it.
First, list all your local containers, so that you can commit an image (snapshot) of the ubuntu container that you just created. The -a option here lists all containers, include those that are not currently running; you should see a container that was just created recently. This is the one you want.
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c52b82b497eb ubuntu "/bin/bash" 2 minutes ago Exited (0) 2 minutes ago kickass_liskov
ae194ed75819 debian "/bin/bash" 7 hours ago Exited (0) 7 hours ago jovial_swanson
34ff1377fbee hello-world "/hello" 26 hours ago Exited (0) 26 weeks ago high_almeida
54240578230c c0bfb9c8e377 "/bin/sh -c '/usr/gam" 26 hours ago Exited (0) 26 weeks ago serene_pare
517904a42f3d docker/whalesay "cowsay hhLinked Appl" 27 hours ago Exited (0) 27 weeks ago romantic_bhaskara
1aad55d740cd docker/whalesay "cowsay docker/whales" 27 hours ago Exited (0) 27 weeks ago cocky_bhaskara
7bfb18e0d18a hello-world "/hello" 28 hours ago Exited (0) 28 weeks ago stupefied_williams
Grab the CONTAINER ID of the ubuntu image that was created 2 minutes ago. It's c52b82b497eb.
Now, we'll commit an image of that container. This also gives you the opportunity to name your image. You must name it with the format cgc-images.sbgenomics.com/<username>/<repository_name>[:tag]. For example, if the user rfranklin wanted to commit her changes to a container in a repository named picard, with tag 1.4 she would use cgc-images.sbgenomics.com/rfranklin/picard:1.4.
Commit the image as follows:
$ docker commit c52b82b497eb cgc-images.sbgenomics.com/rfranklin/picard:1.4
If you want to confirm that the image has been named, you can list all of your local images:
$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
cgc-images.sbgenomics.com/rfranklin/picard 1.4 0fe5d1d1aaec 10 minutes ago 125.1 MB
rfranklin/test latest 0fe5d1d1aaec 18 minutes ago 125.1 MB
debian latest 7a01cc5f27b1 7 hours ago 125.1 MB
ubuntu latest 6cc0fc2a5ee3 8 months ago 187.9 MB
hello-world latest 0a6ba66e537a 8 months ago 960 B
docker/whalesay latest ded5e192a685 8 months ago 247 MB
Push your image
To push your image to the CGC image registry, run the command docker push cgc-images.sbgenomics.com/<username>/<repository_name>[:tag], where <username>/<repository_name>[:tag] refers to the image that you have already committed. For example:
$ docker push cgc-images.sbgenomics.com/rfranklin/picard:1.4
The push refers to a repository [cgc-images.sbgenomics.com/rfranklin/picard] (len: 1)
container@root: pushed
1.4: digest: sha256:afc9023f29719ffd361cdcbc334fe4ec2c041997ee501a15a86ed7f6e9277008 size: 3990
The progress of the image upload will be shown in the terminal. When it has completed, you will see the message pushed.
Remove a local Docker image
If you want to delete a Docker image, use docker rmi cgc-images.sbgenomics.com/<username>/<repository_name>[:tag]. For example:
$ docker rmi cgc-images.sbgenomics.com/rfranklin/picard:1
Deleted 02c8c0913b94a09053dccded886512b77fbd2ecbe18930d1b059bb573f13afd1
Updated 8 months ago