In order to use your own tools or workflows on the CGC, you should install them in individual images, and then push these to the CGC Image Registry or to Docker Hub. You can use Docker to do this.

Installing Docker

Since Docker only runs on Linux, you will need access to a Linux machine to upload your images. If you are running Linux on your computer, you can install Docker locally. If you are running OS X, you can install Docker Machine. This is a tool that installs a Linux virtual machine that runs Docker.
Full installation instructions are below.

1. Installing Docker on Linux
2. Installing Docker on Mac OS 10.10.3 Yosemite or newer
3. Installing Docker on Mac OS 10.8 Mountain Lion or newer
4. Installing Docker on Windows 10 Pro, Enterprise and Education
5. Installing Docker on Windows 7 or Windows 8

Installing Docker on Linux

1. If you are running Linux on your local machine you will need to install Docker directly. You should be logged in as a user with sudo privileges. First, you will need to ensure that you have the command line utility cURL. Do this by opening a terminal and typing:

$ which curl

If cURL is not installed, update your package manager and install it, using:

$ sudo apt-get update
$ sudo apt-get install curl

2. Now that you have cURL, you can use it to get the latest Docker package:

$ curl -fsSL https://get.docker.com/ | sh

3. Add your account to the docker group.

sudo usermod -aG docker <your_username>

This step is required to be able to run Docker commands as a non-root user. You will have to log out and log back in for the change to take effect.

4. Now you should have Docker! Verify that it is installed by running the hello-world container:

$ docker run hello-world
 
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
b901d36b6f2f: Pull complete 
0a6ba66e537a: Pull complete 
Digest: sha256:8be990ef2aeb16dbcb9271ddfe2610fa6658d13f6dfb8bc72074cc1ca36966a7
Status: Downloaded newer image for hello-world:latest


Hello from Docker.

This message shows that your installation appears to be working correctly.

See the list of Docker commands below for instructions on using it to push images.

Installing Docker on OS X

Determine which version of OS X you are running and select an option:

• Mac OS 10.10.3 Yosemite or newer - Follow setup directions in the Docker for Mac section below.
• Mac OS 10.8 Mountain Lion or newer - Follow setup directions in the Docker Toolbox section below.

Docker for Mac

Docker for Mac runs as a native Mac application and uses xhyve to virtualize the Docker Engine environment and Linux kernel-specific features for the Docker daemon. To install Docker for Mac, follow these steps:

1. Download and install Docker for Mac.
   After the installation is complete, the Docker icon will be displayed in the menu bar.
2. Click the Docker icon and you will see the Docker is running information.

3. To use Docker, open a terminal and enter the Docker commands listed below.

📘

To make sure Docker for Mac is always running:

1. Click the Docker icon in the menu bar in the upper right corner of your screen.
2. Choose Preferences.
3. Enable Automatically start Docker when you log in.

Docker Toolbox

If you are running OS X Mountain Lion or newer, you can use Docker Machine, a tool that installs a lightweight Linux virtual machine made specifically for running Docker. It allows you to run Docker commands from your terminal on OS X, or from a 'Quickstart terminal' application on your computer.

1. Docker Machine is shipped with Docker Toolbox, which you can download from docker.com/products/docker-toolbox. To install Docker Toolbox, follow the installation instructions shown on the screen.

2. Once you have completed the installation steps for Docker Toolbox, you're ready to create a new Docker virtual machine. You can do this in two ways:
   (a) using the Docker Machine 'Quickstart Terminal' application, or
   (b) using the command line.

(a) If you are using the Quickstart Terminal app, then simply select it from your Launchpad.

This opens a terminal like the one shown below. If you chose this option, skip ahead to learn about essential Docker commands.

(b) Alternatively, you can create a Docker virtual machine using the command line. To do this, don't click to open the Docker 'Quickstart Terminal' application, but instead open a terminal. Type docker-machine --help to see a list of commands. We'll use docker-machine create to make our virtual machine, and name the machine default. Use the command docker-machine create --driver virtualbox default to do this.

$ docker-machine create --driver virtualbox default
 
Creating machine...
(default) OUT | Creating VirtualBox VM...
(default) OUT | Creating SSH key...
(default) OUT | Starting VirtualBox VM...
(default) OUT | Starting VM...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Detecting the provisioner...
Provisioning created instance...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
To see how to connect Docker to this machine, run: docker-machine env default

3. Use the option ls to list your Docker machines. You should see that your machine default is running:

$ docker-machine ls
NAME       ACTIVE   DRIVER       STATE     URL                         SWARM
default    *        virtualbox   Running   tcp://192.168.99.100:2376

4. Next, you need to set some environment variables for the shell of your Docker machine. You can see which variables need setting by using the env option followed by option argument default, for your default machine:

$ docker-machine env default 
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.101:2376"
export DOCKER_CERT_PATH="/Users/rfranklin/.docker/machine/machines/default2"
export DOCKER_MACHINE_NAME="default2"
# Run this command to configure your shell:
# eval "$(docker-machine env default)"

5. Set the variables using the command eval "$(docker-machine env default)". Note that you will need to do this every time you start a new session. To set the variables persistently, add the command to your .bashrc file (or equivalent file if using a different shell).

$ eval "$(docker-machine env default)"

6. That's it! Now you can use Docker commands, inside your Docker machine. Try docker run hello-world.

$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
b901d36b6f2f: Pull complete
0a6ba66e537a: Pull complete
Digest: sha256:8be990ef2aeb16dbcb9271ddfe2610fa6658d13f6dfb8bc72074cc1ca36966a7
Status: Downloaded newer image for hello-world:latest
 
 
Hello from Docker.
 
This message shows that your installation appears to be working correctly.

7. Whenever you want to use Docker, you need to start your Docker machine. Do this by either selecting the Docker Quickstart Terminal application from your Launchpad, or by using the docker-machine command (in which case, don't forget to set your environment variables) as follows.

$ docker-machine start default
$ eval "$(docker-machine env default)"

See the list of Docker commands below.

Installing Docker on Windows

Determine which version of Windows you are running and select an option:

• Windows 10 Pro, Enterprise, and Education (64bit) - Follow setup directions in the Docker for Windows section below.
• Windows 7 or Windows 8 - Follow setup directions in the Docker Toolbox section below.

Docker for Windows

Docker for Windows runs as a native Windows application. It uses the Hyper-V feature of the Windows OS to virtualize the Docker Engine environment and Linux kernel-specific features for the Docker daemon.

To install Docker for Windows, follow these steps:

1. Download and install Docker for Windows.
2. After installation is complete the screen prompting you to enable to the Hyper-V feature will be displayed. Click Yes to enable it.
3. Restart your computer.
4. Click the Docker for Windows icon on your desktop.

5. Open the terminal of your choice, and issue the following command: docker run hello-world. If the installation was successful you will see the confirmation message.

📘

To make sure Docker for Windows is always running:

1. Right-click Docker icon in the system tray.
2. Choose Settings.
3. Enable the Start Docker when you log in option.

See the list of Docker commands below.

Docker Toolbox

If you are running Windows 7 or Windows 8, access Docker via Docker Toolbox. Docker Toolbox uses Docker Machine to create a Linux virtual machine which hosts Docker on your Windows system.

To install Docker Toolbox, follow these steps:

1. Download Docker Toolbox.
2. Click DockerToolbox.exe and follow the onscreen instructions.

Once the installation is completed, click the Docker Quickstart Terminal icon on your desktop.

📘

Please bear in mind you will need to start Docker Toolbox every time you want to use Docker.

Docker commands

This page contains core Docker commands for running containers and pushing images to the CGC images registry.

👍

Depending on your operating system, first make sure Docker is started

• Docker on Mac OS 10.10.3 Yosemite or newer run Docker for Mac and start a terminal of your choice.
• Docker on Mac OS 10.8 Mountain Lion or newer run Docker Machine, by opening Docker Quickstart terminal or by using the command docker-machine start default.
• Windows 7 or 8: run Docker Quickstart Terminal.
• Windows 10: run Docker for Windows and start a terminal of your choice.
• Linux: skip this step.

Upload an image

The workflow for uploading an image is as follows:

👍

Depending on your operating system, first make sure Docker is running:

• Docker on Mac OS 10.10.3 Yosemite or newer run Docker for Mac and start a terminal of your choice.
• Docker on Mac OS 10.8 Mountain Lion or newer run Docker Machine, by opening Docker Quickstart terminal or by using the command docker-machine start default.
• Windows 7 or 8: run Docker Quickstart Terminal.
• Windows 10: run Docker for Windows and start a terminal of your choice.
• Linux: skip this step.

1. If you use OS X, first run Docker Machine, by opening the Docker Quickstart terminal or by using the command docker-machine start default. If you use Linux, skip this step;
2. Run docker login cgc-images.sbgenomics.com and enter your CGC credentials. Don't forget: enter your username in lower case, replacing spaces with hyphens, and enter your authentication token when prompted for a password.
3. Open a Docker base image with docker run -ti;
4. Install your tools on the image;
5. Commit your image;
6. Push your image to the CGC registry.

Open a Docker image

If you are installing a tool, you'll need to open a new base image to install it on. On the other hand, if you are modifying a tool that you have already uploaded, you can open the image containing the tool from the CGC registry.
To install a tool, start by opening a base image. This can be any base image from Docker Hub, but starting with a plain operating system image like Ubuntu is generally recommended. To use this base image, enter:

docker run -ti ubuntu

To pull an image from the Seven Bridges registry, enter the repository that the image is stored in, followed by the image tag (if any), separated by a colon. The full repository path has the format cgc-images.sbgenomics.com/<username>/<repository_name>[:tag]. For example, if the user rfranklin wanted to open the image tagged 1.3 from her project picard, she would enter

docker-run -ti cgc-images.sbgenomics.com/rfranklin/picard:1.3

Install your tools inside a Docker container

Inside a container you can install your chosen tool or modify an existing tool. Do this in the way that you would normally, using methods appropriate for your tool, e.g. apt-get. Files from the local host can be accessed in the directory /mountedcwd. So, if the software you want to install is stored in a remote repository, you can git pull your code and mount it into the container.

When you've finished installing a tool, leave the container by typing exit.

root@container$ exit

Commit the image

After you exit the container, you can commit the image of it.
First, list all your local containers, so that you can commit an image (snapshot) of the ubuntu container that you just created. The -a option here lists all containers, include those that are not currently running; you should see a container that was just created recently. This is the one you want.

$ docker ps -a

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS               NAMES

c52b82b497eb        ubuntu              "/bin/bash"              2 minutes ago       Exited (0) 2 minutes ago                        kickass_liskov
ae194ed75819        debian              "/bin/bash"              7 hours ago         Exited (0) 7 hours ago                          jovial_swanson
34ff1377fbee        hello-world         "/hello"                 26 hours ago        Exited (0) 26 weeks ago                         high_almeida
54240578230c        c0bfb9c8e377        "/bin/sh -c '/usr/gam"   26 hours ago        Exited (0) 26 weeks ago                         serene_pare
517904a42f3d        docker/whalesay     "cowsay hhLinked Appl"   27 hours ago        Exited (0) 27 weeks ago                         romantic_bhaskara
1aad55d740cd        docker/whalesay     "cowsay docker/whales"   27 hours ago        Exited (0) 27 weeks ago                         cocky_bhaskara
7bfb18e0d18a        hello-world         "/hello"                 28 hours ago        Exited (0) 28 weeks ago                         stupefied_williams

👍

Your local containers

If you have followed the Docker Quickstart, on their website, or used Docker in any other way, you might have already created some containers. You should see those listed too.

Grab the CONTAINER ID of the ubuntu image that was created 2 minutes ago. It's c52b82b497eb.
Now, we'll commit an image of that container. This also gives you the opportunity to name your image. You must name it with the format cgc-images.sbgenomics.com/<username>/<repository_name>[:tag]. For example, if the user rfranklin wanted to commit her changes to a container in a repository named picard, with tag 1.4 she would use cgc-images.sbgenomics.com/rfranklin/picard:1.4.

Commit the image as follows:

$ docker commit c52b82b497eb cgc-images.sbgenomics.com/rfranklin/picard:1.4

If you want to confirm that the image has been named, you can list all of your local images:

$ docker images 

REPOSITORY                                TAG                 IMAGE ID            CREATED             VIRTUAL SIZE

cgc-images.sbgenomics.com/rfranklin/picard    1.4                 0fe5d1d1aaec        10 minutes ago      125.1 MB
rfranklin/test                            latest              0fe5d1d1aaec        18 minutes ago      125.1 MB
debian                                    latest              7a01cc5f27b1        7 hours ago         125.1 MB
ubuntu                                    latest              6cc0fc2a5ee3        8 months ago        187.9 MB
hello-world                               latest              0a6ba66e537a        8 months ago        960 B
docker/whalesay                           latest              ded5e192a685        8 months ago        247 MB

Push your image

To push your image to the CGC image registry, run the command docker push cgc-images.sbgenomics.com/<username>/<repository_name>[:tag], where <username>/<repository_name>[:tag] refers to the image that you have already committed. For example:

$ docker push cgc-images.sbgenomics.com/rfranklin/picard:1.4
The push refers to a repository [cgc-images.sbgenomics.com/rfranklin/picard] (len: 1)
container@root: pushed
1.4: digest: sha256:afc9023f29719ffd361cdcbc334fe4ec2c041997ee501a15a86ed7f6e9277008 size: 3990

The progress of the image upload will be shown in the terminal. When it has completed, you will see the message pushed.

Remove a local Docker image

If you want to delete a Docker image, use docker rmi cgc-images.sbgenomics.com/<username>/<repository_name>[:tag]. For example:

$ docker rmi cgc-images.sbgenomics.com/rfranklin/picard:1
Deleted 02c8c0913b94a09053dccded886512b77fbd2ecbe18930d1b059bb573f13afd1