{"_id":"57bc3a7c531e000e0075d11c","project":"55faf11ba62ba1170021a9a7","__v":0,"user":"554340dfb7f4540d00fcef1d","version":{"_id":"55faf11ba62ba1170021a9aa","project":"55faf11ba62ba1170021a9a7","__v":38,"createdAt":"2015-09-17T16:58:03.490Z","releaseDate":"2015-09-17T16:58:03.490Z","categories":["55faf11ca62ba1170021a9ab","55faf8f4d0e22017005b8272","55faf91aa62ba1170021a9b5","55faf929a8a7770d00c2c0bd","55faf932a8a7770d00c2c0bf","55faf94b17b9d00d00969f47","55faf958d0e22017005b8274","55faf95fa8a7770d00c2c0c0","55faf96917b9d00d00969f48","55faf970a8a7770d00c2c0c1","55faf98c825d5f19001fa3a6","55faf99aa62ba1170021a9b8","55faf99fa62ba1170021a9b9","55faf9aa17b9d00d00969f49","55faf9b6a8a7770d00c2c0c3","55faf9bda62ba1170021a9ba","5604570090ee490d00440551","5637e8b2fbe1c50d008cb078","5649bb624fa1460d00780add","5671974d1b6b730d008b4823","5671979d60c8e70d006c9760","568e8eef70ca1f0d0035808e","56d0a2081ecc471500f1795e","56d4a0adde40c70b00823ea3","56d96b03dd90610b00270849","56fbb83d8f21c817002af880","573c811bee2b3b2200422be1","576bc92afb62dd20001cda85","5771811e27a5c20e00030dcd","5785191af3a10c0e009b75b0","57bdf84d5d48411900cd8dc0","57ff5c5dc135231700aed806","5804caf792398f0f00e77521","58458b4fba4f1c0f009692bb","586d3c287c6b5b2300c05055","58ef66d88646742f009a0216","58f5d52d7891630f00fe4e77","59a555bccdbd85001bfb1442"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"category":{"_id":"55faf958d0e22017005b8274","__v":8,"pages":["56268b3cc2781f0d00364bc0","5637bfa84dbdd919001b277d","5637e6fe97666c0d008656b6","5637e74acfaa870d00cdeb7a","5637e764f7e3990d007b2c52","5640baf604c3eb0d006fc43c","56cccb6894c8f00b00b83f24","56ddf68dba70f50e0033c6df"],"project":"55faf11ba62ba1170021a9a7","version":"55faf11ba62ba1170021a9aa","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2015-09-17T17:33:12.687Z","from_sync":false,"order":14,"slug":"bring-your-tools","title":"BRING YOUR TOOLS"},"parentDoc":null,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2016-08-23T11:58:52.910Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":5,"body":"[block:callout]\n{\n  \"type\": \"warning\",\n  \"title\": \"On this page\",\n  \"body\": \"* [Overview](#section-overview)\\n* [Format](#section-format)\\n* [Dockerfile sample](#section-dockerfile-sample)\\n* [Building an image from a Dockerfile and pushing it to the CGC image registry](#section-building-an-image-from-a-dockerfile-and-pushing-it-to-the-cgc-image-registry)\"\n}\n[/block]\n###Overview\nDockerfiles are text files that store the commands you would execute on the command line inside a container to create a Docker image. When using Dockerfiles, the process of building an image is automated as Docker reads the commands (instructions) from a Dockerfile and executes them in succession in order to create the final image.\n\nThe benefit of Dockerfiles is that they store the whole procedure on how an image is created. They are also significant as they help facilitate and automate the process of maintaining tools that are wrapped for use on the CGC. Specifically, Dockerfiles can contain instructions to install the required dependencies into the container that is loaded from the base image, add the required tool and tool-related files from its repository to the container, and install the tool into the container. This means that when changes are made to a tool and it needs to be wrapped for use on the CGC again, the image containing the tool can be built automatically based on the Dockerfile with no changes or only minor changes made to the Dockerfile itself.\n\n###Format\nA Dockerfile consists of two kind of items: **instructions followed by arguments** and **comments**. The basic Dockerfile format is shown below:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"# Comment\\nINSTRUCTION arguments\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nInstructions are not case-sensitive, but are usually written in uppercase so that they can be differentiated from arguments more easily. Comments have the hash symbol (#) at the beginning of the line. However, if the same symbol is located anywhere else in a line, the line will not be treated as a comment.\n\nAn example of an instruction is shown below:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"FROM ubuntu\",\n      \"language\": \"text\",\n      \"name\": null\n    }\n  ]\n}\n[/block]\nThis instruction and argument assign the ubuntu image as the base image that you will build upon.\n\n###Usage\nThis section will present some of the most common instructions used in Dockerfiles and the way in which they are usually used when wrapping tools for use on the CGC. For a full list of instructions and all of their possible formats and uses, please refer to the official [Dockerfile reference](https://docs.docker.com/engine/reference/builder/).\n\n**FROM**\nDocker runs instructions in the order in which they are listed in the Dockerfile. The first instruction and the first non-comment line in a Dockerfile **must be** `FROM` in order to specify the base image from which you will start building your new image. The instruction is entered in the following format:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"FROM <image>\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nor\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"FROM <image>:<tag>\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nor\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"FROM <image>:::at:::<digest>\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nThe `<tag>` part of the argument is used to specify a version of the image. This means that the instruction `FROM ubuntu:14.04` will automatically load the latest available version of Ubuntu 14.04 as the base image. On the other hand, `<digest>` is more specific as it is used to refer to an exact image which might not be the latest available version. For example, if you want to use a specific version of the Ubuntu 14.04 image which is not the latest available one, the instruction would be, for example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"FROM ubuntu@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nThe `<image>` argument is mandatory when using the `FROM` instruction, while `<tag>` or `<digest>` are optional. If they are not specified, the assumed tag will be `:latest` and the latest available version of the base image will be used. \n\nIf you want to use the `ubuntu` base image, your Dockerfile has to start with the following instruction:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"FROM ubuntu\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nLearn more about the [FROM instruction](https://docs.docker.com/engine/reference/builder/#/from).\n\n**MAINTAINER**\nThe `MAINTAINER` instruction is not mandatory, but is highly suggested so that containers can be sourced and authors can be contacted for information and support. This information provided in this instruction will also be used by Docker to populate the **Author** field for the generated image. The instruction is used in the following format:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"MAINTAINER <name>\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nThe `<name>` argument can include the name and additional information, such as an email address. For example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"MAINTAINER Rosalind Franklin rosalind.franklin@sbgenomics.com\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nAs the `MAINTAINER` instruction is used only to provide additional information rather than perform any action, it can be placed anywhere in the Dockerfile after the `FROM` instruction.\n\n**RUN**\nThe `RUN` instruction is the main executing instruction in a Dockerfile. This instruction is used in the following form:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"RUN <command>\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nThe `RUN` instruction executes the command that is provided as its argument. The results of execution are then committed to the current image and the resulting image is used for the next instruction listed in the Dockerfile. The example below shows how to use the `RUN` instruction to pull a specific version of SAMTools from its repository:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"RUN wget https://github.com/samtools/samtools/releases/download/1.2/samtools-1.2.tar.bz2\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nYou can also chain multiple commands within the same `RUN` instruction:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"RUN wget https://github.com/samtools/samtools/releases/download/1.2/samtools-1.2.tar.bz2 \\\\\\n    && tar jxf samtools-1.2.tar.bz2 \\\\\\n    && cd samtools-1.2 \\\\\\n    && make \\\\\\n    && make install\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nThis code block shows how commands are chained as an argument to a single `RUN` instruction. This instruction uses:\n* the `&&` connective to chain commands (indicates that a command will be executed only if the execution of the previous command succeeds),\n* the `\\` character (denotes a line break).\n\nLearn more about the [RUN instruction](https://docs.docker.com/engine/reference/builder/#/run).\n\n**CMD**\nThe `CMD` instruction can be used to execute a command. However, unlike `RUN` which is executed during build, the purpose of `CMD` is to provide the default command which is executed inside the container when it is created based on the image. This instruction can be used in the following format:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"CMD [\\\"command\\\",\\\"param1\\\",\\\"param2\\\"]\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nAlternatively, you can also use the shell form of the instruction:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"CMD command param1 param2 ...\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nIf there is more than one `CMD` instruction in a Dockerfile, only the last one is executed.\nContainers intended for use on the CGC have the following `CMD` instruction as that is how the container is invoked during execution of a task:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"CMD [\\\"/bin/bash\\\"]\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nIf you specify an additional argument after `docker run <image>`, the command specified as the argument will override any command set within the `CMD` instruction in the Dockerfile.\n\nLearn more about the [CMD instruction](https://docs.docker.com/engine/reference/builder/#/cmd).\n\n**ADD**\nThe `ADD` instruction is used to copy files, directories or remote file URLs from their original location `<source>` and to the container at the specified path `<destination>`. The `ADD` instruction has the following format:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ADD <source>...<destination>\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nYou can specify multiple `<source>` items. If those are files or folders, they must be located within the context of the build. The context of a build can either be on your local file system (the directory where you execute the command to build the image based on the Dockerfile) or it can be a URL (location of a Git repository, for example).\n\nYou can specify only those source paths that are within the context directory (including subdirectories), but not paths like `../directory/subdirectory`. The `<destination>` argument can either be an absolute path or a relative one.\n\nThe `<source>` parameter can also take wildcards in file names, for example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ADD sample?.txt /tmp/\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nThis will add the files named e.g. `sample1.txt`, `sample2.txt`, etc. in the `/tmp/` folder inside the container.\n\nThe basic allowed wildcards match those described on the [glob](page:glob) page. Learn more about [pattern matching](https://golang.org/pkg/path/filepath/#Match) in Dockerfiles.\n\nThe following rules also apply to the `ADD` instruction:\n* If `<source>` is a URL and `<destination>` does not end with a slash, then the file is downloaded from the URL and its contents are copied to `<destination>`. For example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ADD http://domain.com/sourcefile.txt /tmp/destfile\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nThis would save the entire contents of `sourcefile.txt` in the `/tmp/` folder within the container, as a file named `destfile`.\n* If `<source>` is a URL and `<destination>` ends with a slash, then the file keeps its original name and is downloaded to `<destination>/<filename>`. For instance:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ADD http://domain.com/file.txt /tmp/translate/\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nThis would create `file.txt` in the `/tmp/translate/` folder. For this instruction to be executed properly, the URL must point to the exact file.\nIf authentication is required to obtain the file from the URL, you will need to use an appropriate tool with the `RUN` instruction, since the `ADD` instruction does not support authentication.\n* If `<source>` is a local TAR archive in a recognized compression format (identity, gzip, bzip2 or xz), it will be unpacked in the destination directory. However, archives from remote sources (URLs) will not be unpacked.\n\n<a name=\"commonrules\" style=\"text-decoration: none; color: #474a54;\">The rules below apply both to the `ADD` and [COPY instruction](#copy):</a>\n* If `<source>` is a directory, the entire contents of the directory are copied (but not the directory itself).\n* If `<destination>` does not exist, it is created. This also applies to all missing directories in the `<destination>` path.\n* If multiple `<source>` items are specified (either explicitly or due to using a wildcard) then `<destination>` must be a directory and the path must end with a slash `/`.\n* If `<destination>` does not end with a slash, the contents of `<source>` will be written to `<destination>`. For example, if the specified instruction is `ADD sourcefile.txt /containertmp/destfile`, the entire contents of `sourcefile.txt` will be saved in the `/containertmp/` folder within the container as a file named `destfile`.\n\nLearn more about the [ADD instruction](https://docs.docker.com/engine/reference/builder/#/add).\n\n<a name=\"copy\" style=\"text-decoration: none; color: #474a54;\">**COPY**</a>\nThere are two major differences between `ADD` and `COPY`:\n* `ADD` can also take a URL as `<source>`.\n* If the `<source>` parameter of the `ADD` instruction is an archive in a recognized compression format, it will be unpacked. However, the `COPY` instruction will only copy the archive file, without unpacking it.\n\nThe `COPY` instruction is used to copy files or directories to the container at the specified path. `COPY` has the following format:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"COPY <source>...<destination>\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nYou can specify multiple source items. The items must be located within the context of the build - you are able to specify only those source paths that are within the context directory (including subdirectories), but not paths like `../directory/subdirectory`. The `<destination>` argument can either be an absolute path or a relative one.\n\nThe `<source>` parameter can also take wildcards in file names, for example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"COPY sample*.txt /script/\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nThis will copy the files named e.g. `sample12.txt`, `sampleabc.txt`, `sample_new.txt` etc. to the `/script/` folder inside the container.\n\nThe basic allowed wildcards match those described on the [glob](page:glob) page. Learn more about [pattern matching](https://golang.org/pkg/path/filepath/#Match) in Dockerfiles.\n\nThe `COPY` instruction shares a set of rules related to `<source>` and `<destination>` paths with the `ADD` instruction. Click <a href=\"#commonrules\">here</a> to see those rules.\n\nLearn more about the [COPY instruction](https://docs.docker.com/engine/reference/builder/#/copy).\n\n**ENV**\nThe `ENV` instruction is used to set the environment variable(s). These variables consist of key-value pairs which can be accessed within the container. This instruction has the following form:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ENV <key> <value>\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nor\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ENV <key>=<value>\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nThe second form of the instruction allows you to add multiple key-value pairs in the same instruction, by separating the pairs with a space:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ENV <key>=<value> <key>=<value> ...\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nEnvironment variables declared through the ENV instruction can also be used as variables by other instructions. For example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ENV PATH /code/tmp/\\nCOPY . $PATH\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nWhen used as a variable by another instruction, environment variables are written as either `$variablename` or `${variablename}`.\n\n**WORKDIR**\nThe `WORKDIR` instruction is used to set the default working directory for the container. Instructions such as `ADD`, `COPY`, `RUN` or `CMD` that are entered after the `WORKDIR` instruction in a Dockerfile will be executed in the defined working directory.\n[block:callout]\n{\n  \"type\": \"success\",\n  \"body\": \"You should always use the `WORKDIR` instruction instead of `RUN cd /directory/subdirectory/...` to set the working directory.\"\n}\n[/block]\nThis instruction can be used multiple times in a Dockerfile. If the directory that is specified as `WORKDIR` does not exist, it will be created.\nYou are also able to use variables previously set in the `ENV` instruction as arguments for `WORKDIR`. For example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ENV PATH /app\\nWORKDIR $PATH\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nDockerfiles for images that are intended for use on the CGC should include the following WORKDIR instruction:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"WORKDIR /\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n\n###Dockerfile sample\nThe following code represents a sample Dockerfile:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"# Specifying the ubuntu base image\\nFROM ubuntu\\n    \\n# Name and email of the person who maintains the file\\nMAINTAINER Rosalind Franklin rosalind.franklin@sbgenomics.com\\n  \\n# Set working directory as \\\"/\\\"\\nWORKDIR /\\n  \\n# Updating ubuntu and installing other necessary software\\nRUN apt-get update --yes \\\\\\n&& apt-get install wget build-essential zlib1g-dev libncurses5-dev vim --yes\\n   \\n# Pulling SAMTools from its repository, unpacking the archive and installing\\nRUN wget https://github.com/samtools/samtools/releases/download/1.2/samtools-1.2.tar.bz2 \\\\\\n&& tar jxf samtools-1.2.tar.bz2 \\\\\\n&& cd samtools-1.2 \\\\\\n&& make \\\\\\n&& make install\\n  \\n  \\n# Set command to bash\\nCMD [\\\"/bin/bash\\\"]\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n###Building an image from a Dockerfile and pushing it to the CGC image registry\nWhen you have created a Dockerfile, the image is built using the `docker build` command. The `docker build` command requires a Dockerfile and a context to build an image. It is common practice to put the Dockerfile at the root of the build context.\n[block:callout]\n{\n  \"type\": \"danger\",\n  \"body\": \"Do not use the root directory of your file system (`/`) as the build context. Since the first step of the `docker build` command is to send the build context to the docker daemon, this would cause transfer of your entire drive to the daemon.\"\n}\n[/block]\nThis is how the `docker build` command is run using the current directory as the context:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker build .\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nWhen building an image containing a tool to be used on the CGC, you need to specify a repository and a tag. The docker build command then has the following format:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker build -t cgc-images.sbgenomics.com/<user_name>/<project_name>:<tag> .\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nIn the above code, repository name is the `<user_name>/<project_name>` part. Note that `<user_name>` needs to be your username as displayed in the URL when you open a project on the CGC, e.g. in `https://cgc.sbgenomics.com/u/rosalind_franklin/my-project/`, the username is `rosalind_franklin`. Learn more about [repositories in the CGC image registry](doc:the-cgc-image-registry#section-repository-names).\n\nFor example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker build -t cgc-images.sbgenomics.com/rosalind_franklin/samtools:v1 .\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nAfter the build process has been completed successfully, the next step is to log in to the CGC image registry (cgc-images.sbgenomics.com) from the terminal:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker login cgc-images.sbgenomics.com\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n\n[block:callout]\n{\n  \"type\": \"danger\",\n  \"body\": \"You should enter your [authentication token](http://docs.cancergenomicscloud.org/docs/get-your-authentication-token) in response to the password prompt, not your CGC password.\"\n}\n[/block]\nFinally, you need to push the image you have created to the CGC image registry:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker push cgc-images.sbgenomics.com/rosalind_franklin/samtools:v1\",\n      \"language\": \"text\",\n      \"name\": null\n    }\n  ]\n}\n[/block]\nOnce the process has been completed, use the [Tool Editor](doc:the-tool-editor) to provide a description of the tool on the CGC.\n\n###Creating a Dockerfile for an existing Docker image\nIf you already have a Docker image and you want to create a Dockerfile for the image, [use Docker Hub to store your image](https://docs.docker.com/docker-hub/repos/#/pushing-a-repository-image-to-docker-hub), as follows:\n\n1. Log in to Docker Hub:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker login\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n2. When prompted, enter your Docker Hub credentials.\n\n3. Push your Docker image to Docker Hub:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker push <image>\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n4. Create a Dockerfile containing the following line:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"FROM <image>\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nIn this case, `<image>` is the reference to the image on Docker Hub, e.g. `rosalind_franklin/my_image`.\nYou are now able to build the image using the Dockerfile, in the way described above.","excerpt":"","slug":"upload-your-docker-image-with-a-dockerfile","type":"basic","title":"Upload your Docker image with a Dockerfile"}

Upload your Docker image with a Dockerfile


[block:callout] { "type": "warning", "title": "On this page", "body": "* [Overview](#section-overview)\n* [Format](#section-format)\n* [Dockerfile sample](#section-dockerfile-sample)\n* [Building an image from a Dockerfile and pushing it to the CGC image registry](#section-building-an-image-from-a-dockerfile-and-pushing-it-to-the-cgc-image-registry)" } [/block] ###Overview Dockerfiles are text files that store the commands you would execute on the command line inside a container to create a Docker image. When using Dockerfiles, the process of building an image is automated as Docker reads the commands (instructions) from a Dockerfile and executes them in succession in order to create the final image. The benefit of Dockerfiles is that they store the whole procedure on how an image is created. They are also significant as they help facilitate and automate the process of maintaining tools that are wrapped for use on the CGC. Specifically, Dockerfiles can contain instructions to install the required dependencies into the container that is loaded from the base image, add the required tool and tool-related files from its repository to the container, and install the tool into the container. This means that when changes are made to a tool and it needs to be wrapped for use on the CGC again, the image containing the tool can be built automatically based on the Dockerfile with no changes or only minor changes made to the Dockerfile itself. ###Format A Dockerfile consists of two kind of items: **instructions followed by arguments** and **comments**. The basic Dockerfile format is shown below: [block:code] { "codes": [ { "code": "# Comment\nINSTRUCTION arguments", "language": "text" } ] } [/block] Instructions are not case-sensitive, but are usually written in uppercase so that they can be differentiated from arguments more easily. Comments have the hash symbol (#) at the beginning of the line. However, if the same symbol is located anywhere else in a line, the line will not be treated as a comment. An example of an instruction is shown below: [block:code] { "codes": [ { "code": "FROM ubuntu", "language": "text", "name": null } ] } [/block] This instruction and argument assign the ubuntu image as the base image that you will build upon. ###Usage This section will present some of the most common instructions used in Dockerfiles and the way in which they are usually used when wrapping tools for use on the CGC. For a full list of instructions and all of their possible formats and uses, please refer to the official [Dockerfile reference](https://docs.docker.com/engine/reference/builder/). **FROM** Docker runs instructions in the order in which they are listed in the Dockerfile. The first instruction and the first non-comment line in a Dockerfile **must be** `FROM` in order to specify the base image from which you will start building your new image. The instruction is entered in the following format: [block:code] { "codes": [ { "code": "FROM <image>", "language": "text" } ] } [/block] or [block:code] { "codes": [ { "code": "FROM <image>:<tag>", "language": "text" } ] } [/block] or [block:code] { "codes": [ { "code": "FROM <image>@<digest>", "language": "text" } ] } [/block] The `<tag>` part of the argument is used to specify a version of the image. This means that the instruction `FROM ubuntu:14.04` will automatically load the latest available version of Ubuntu 14.04 as the base image. On the other hand, `<digest>` is more specific as it is used to refer to an exact image which might not be the latest available version. For example, if you want to use a specific version of the Ubuntu 14.04 image which is not the latest available one, the instruction would be, for example: [block:code] { "codes": [ { "code": "FROM ubuntu@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2", "language": "text" } ] } [/block] The `<image>` argument is mandatory when using the `FROM` instruction, while `<tag>` or `<digest>` are optional. If they are not specified, the assumed tag will be `:latest` and the latest available version of the base image will be used. If you want to use the `ubuntu` base image, your Dockerfile has to start with the following instruction: [block:code] { "codes": [ { "code": "FROM ubuntu", "language": "text" } ] } [/block] Learn more about the [FROM instruction](https://docs.docker.com/engine/reference/builder/#/from). **MAINTAINER** The `MAINTAINER` instruction is not mandatory, but is highly suggested so that containers can be sourced and authors can be contacted for information and support. This information provided in this instruction will also be used by Docker to populate the **Author** field for the generated image. The instruction is used in the following format: [block:code] { "codes": [ { "code": "MAINTAINER <name>", "language": "text" } ] } [/block] The `<name>` argument can include the name and additional information, such as an email address. For example: [block:code] { "codes": [ { "code": "MAINTAINER Rosalind Franklin rosalind.franklin@sbgenomics.com", "language": "text" } ] } [/block] As the `MAINTAINER` instruction is used only to provide additional information rather than perform any action, it can be placed anywhere in the Dockerfile after the `FROM` instruction. **RUN** The `RUN` instruction is the main executing instruction in a Dockerfile. This instruction is used in the following form: [block:code] { "codes": [ { "code": "RUN <command>", "language": "text" } ] } [/block] The `RUN` instruction executes the command that is provided as its argument. The results of execution are then committed to the current image and the resulting image is used for the next instruction listed in the Dockerfile. The example below shows how to use the `RUN` instruction to pull a specific version of SAMTools from its repository: [block:code] { "codes": [ { "code": "RUN wget https://github.com/samtools/samtools/releases/download/1.2/samtools-1.2.tar.bz2", "language": "text" } ] } [/block] You can also chain multiple commands within the same `RUN` instruction: [block:code] { "codes": [ { "code": "RUN wget https://github.com/samtools/samtools/releases/download/1.2/samtools-1.2.tar.bz2 \\\n && tar jxf samtools-1.2.tar.bz2 \\\n && cd samtools-1.2 \\\n && make \\\n && make install", "language": "text" } ] } [/block] This code block shows how commands are chained as an argument to a single `RUN` instruction. This instruction uses: * the `&&` connective to chain commands (indicates that a command will be executed only if the execution of the previous command succeeds), * the `\` character (denotes a line break). Learn more about the [RUN instruction](https://docs.docker.com/engine/reference/builder/#/run). **CMD** The `CMD` instruction can be used to execute a command. However, unlike `RUN` which is executed during build, the purpose of `CMD` is to provide the default command which is executed inside the container when it is created based on the image. This instruction can be used in the following format: [block:code] { "codes": [ { "code": "CMD [\"command\",\"param1\",\"param2\"]", "language": "text" } ] } [/block] Alternatively, you can also use the shell form of the instruction: [block:code] { "codes": [ { "code": "CMD command param1 param2 ...", "language": "text" } ] } [/block] If there is more than one `CMD` instruction in a Dockerfile, only the last one is executed. Containers intended for use on the CGC have the following `CMD` instruction as that is how the container is invoked during execution of a task: [block:code] { "codes": [ { "code": "CMD [\"/bin/bash\"]", "language": "text" } ] } [/block] If you specify an additional argument after `docker run <image>`, the command specified as the argument will override any command set within the `CMD` instruction in the Dockerfile. Learn more about the [CMD instruction](https://docs.docker.com/engine/reference/builder/#/cmd). **ADD** The `ADD` instruction is used to copy files, directories or remote file URLs from their original location `<source>` and to the container at the specified path `<destination>`. The `ADD` instruction has the following format: [block:code] { "codes": [ { "code": "ADD <source>...<destination>", "language": "text" } ] } [/block] You can specify multiple `<source>` items. If those are files or folders, they must be located within the context of the build. The context of a build can either be on your local file system (the directory where you execute the command to build the image based on the Dockerfile) or it can be a URL (location of a Git repository, for example). You can specify only those source paths that are within the context directory (including subdirectories), but not paths like `../directory/subdirectory`. The `<destination>` argument can either be an absolute path or a relative one. The `<source>` parameter can also take wildcards in file names, for example: [block:code] { "codes": [ { "code": "ADD sample?.txt /tmp/", "language": "text" } ] } [/block] This will add the files named e.g. `sample1.txt`, `sample2.txt`, etc. in the `/tmp/` folder inside the container. The basic allowed wildcards match those described on the [glob](page:glob) page. Learn more about [pattern matching](https://golang.org/pkg/path/filepath/#Match) in Dockerfiles. The following rules also apply to the `ADD` instruction: * If `<source>` is a URL and `<destination>` does not end with a slash, then the file is downloaded from the URL and its contents are copied to `<destination>`. For example: [block:code] { "codes": [ { "code": "ADD http://domain.com/sourcefile.txt /tmp/destfile", "language": "text" } ] } [/block] This would save the entire contents of `sourcefile.txt` in the `/tmp/` folder within the container, as a file named `destfile`. * If `<source>` is a URL and `<destination>` ends with a slash, then the file keeps its original name and is downloaded to `<destination>/<filename>`. For instance: [block:code] { "codes": [ { "code": "ADD http://domain.com/file.txt /tmp/translate/", "language": "text" } ] } [/block] This would create `file.txt` in the `/tmp/translate/` folder. For this instruction to be executed properly, the URL must point to the exact file. If authentication is required to obtain the file from the URL, you will need to use an appropriate tool with the `RUN` instruction, since the `ADD` instruction does not support authentication. * If `<source>` is a local TAR archive in a recognized compression format (identity, gzip, bzip2 or xz), it will be unpacked in the destination directory. However, archives from remote sources (URLs) will not be unpacked. <a name="commonrules" style="text-decoration: none; color: #474a54;">The rules below apply both to the `ADD` and [COPY instruction](#copy):</a> * If `<source>` is a directory, the entire contents of the directory are copied (but not the directory itself). * If `<destination>` does not exist, it is created. This also applies to all missing directories in the `<destination>` path. * If multiple `<source>` items are specified (either explicitly or due to using a wildcard) then `<destination>` must be a directory and the path must end with a slash `/`. * If `<destination>` does not end with a slash, the contents of `<source>` will be written to `<destination>`. For example, if the specified instruction is `ADD sourcefile.txt /containertmp/destfile`, the entire contents of `sourcefile.txt` will be saved in the `/containertmp/` folder within the container as a file named `destfile`. Learn more about the [ADD instruction](https://docs.docker.com/engine/reference/builder/#/add). <a name="copy" style="text-decoration: none; color: #474a54;">**COPY**</a> There are two major differences between `ADD` and `COPY`: * `ADD` can also take a URL as `<source>`. * If the `<source>` parameter of the `ADD` instruction is an archive in a recognized compression format, it will be unpacked. However, the `COPY` instruction will only copy the archive file, without unpacking it. The `COPY` instruction is used to copy files or directories to the container at the specified path. `COPY` has the following format: [block:code] { "codes": [ { "code": "COPY <source>...<destination>", "language": "text" } ] } [/block] You can specify multiple source items. The items must be located within the context of the build - you are able to specify only those source paths that are within the context directory (including subdirectories), but not paths like `../directory/subdirectory`. The `<destination>` argument can either be an absolute path or a relative one. The `<source>` parameter can also take wildcards in file names, for example: [block:code] { "codes": [ { "code": "COPY sample*.txt /script/", "language": "text" } ] } [/block] This will copy the files named e.g. `sample12.txt`, `sampleabc.txt`, `sample_new.txt` etc. to the `/script/` folder inside the container. The basic allowed wildcards match those described on the [glob](page:glob) page. Learn more about [pattern matching](https://golang.org/pkg/path/filepath/#Match) in Dockerfiles. The `COPY` instruction shares a set of rules related to `<source>` and `<destination>` paths with the `ADD` instruction. Click <a href="#commonrules">here</a> to see those rules. Learn more about the [COPY instruction](https://docs.docker.com/engine/reference/builder/#/copy). **ENV** The `ENV` instruction is used to set the environment variable(s). These variables consist of key-value pairs which can be accessed within the container. This instruction has the following form: [block:code] { "codes": [ { "code": "ENV <key> <value>", "language": "text" } ] } [/block] or [block:code] { "codes": [ { "code": "ENV <key>=<value>", "language": "text" } ] } [/block] The second form of the instruction allows you to add multiple key-value pairs in the same instruction, by separating the pairs with a space: [block:code] { "codes": [ { "code": "ENV <key>=<value> <key>=<value> ...", "language": "text" } ] } [/block] Environment variables declared through the ENV instruction can also be used as variables by other instructions. For example: [block:code] { "codes": [ { "code": "ENV PATH /code/tmp/\nCOPY . $PATH", "language": "text" } ] } [/block] When used as a variable by another instruction, environment variables are written as either `$variablename` or `${variablename}`. **WORKDIR** The `WORKDIR` instruction is used to set the default working directory for the container. Instructions such as `ADD`, `COPY`, `RUN` or `CMD` that are entered after the `WORKDIR` instruction in a Dockerfile will be executed in the defined working directory. [block:callout] { "type": "success", "body": "You should always use the `WORKDIR` instruction instead of `RUN cd /directory/subdirectory/...` to set the working directory." } [/block] This instruction can be used multiple times in a Dockerfile. If the directory that is specified as `WORKDIR` does not exist, it will be created. You are also able to use variables previously set in the `ENV` instruction as arguments for `WORKDIR`. For example: [block:code] { "codes": [ { "code": "ENV PATH /app\nWORKDIR $PATH", "language": "text" } ] } [/block] Dockerfiles for images that are intended for use on the CGC should include the following WORKDIR instruction: [block:code] { "codes": [ { "code": "WORKDIR /", "language": "text" } ] } [/block] ###Dockerfile sample The following code represents a sample Dockerfile: [block:code] { "codes": [ { "code": "# Specifying the ubuntu base image\nFROM ubuntu\n \n# Name and email of the person who maintains the file\nMAINTAINER Rosalind Franklin rosalind.franklin@sbgenomics.com\n \n# Set working directory as \"/\"\nWORKDIR /\n \n# Updating ubuntu and installing other necessary software\nRUN apt-get update --yes \\\n&& apt-get install wget build-essential zlib1g-dev libncurses5-dev vim --yes\n \n# Pulling SAMTools from its repository, unpacking the archive and installing\nRUN wget https://github.com/samtools/samtools/releases/download/1.2/samtools-1.2.tar.bz2 \\\n&& tar jxf samtools-1.2.tar.bz2 \\\n&& cd samtools-1.2 \\\n&& make \\\n&& make install\n \n \n# Set command to bash\nCMD [\"/bin/bash\"]", "language": "text" } ] } [/block] ###Building an image from a Dockerfile and pushing it to the CGC image registry When you have created a Dockerfile, the image is built using the `docker build` command. The `docker build` command requires a Dockerfile and a context to build an image. It is common practice to put the Dockerfile at the root of the build context. [block:callout] { "type": "danger", "body": "Do not use the root directory of your file system (`/`) as the build context. Since the first step of the `docker build` command is to send the build context to the docker daemon, this would cause transfer of your entire drive to the daemon." } [/block] This is how the `docker build` command is run using the current directory as the context: [block:code] { "codes": [ { "code": "docker build .", "language": "text" } ] } [/block] When building an image containing a tool to be used on the CGC, you need to specify a repository and a tag. The docker build command then has the following format: [block:code] { "codes": [ { "code": "docker build -t cgc-images.sbgenomics.com/<user_name>/<project_name>:<tag> .", "language": "text" } ] } [/block] In the above code, repository name is the `<user_name>/<project_name>` part. Note that `<user_name>` needs to be your username as displayed in the URL when you open a project on the CGC, e.g. in `https://cgc.sbgenomics.com/u/rosalind_franklin/my-project/`, the username is `rosalind_franklin`. Learn more about [repositories in the CGC image registry](doc:the-cgc-image-registry#section-repository-names). For example: [block:code] { "codes": [ { "code": "docker build -t cgc-images.sbgenomics.com/rosalind_franklin/samtools:v1 .", "language": "text" } ] } [/block] After the build process has been completed successfully, the next step is to log in to the CGC image registry (cgc-images.sbgenomics.com) from the terminal: [block:code] { "codes": [ { "code": "docker login cgc-images.sbgenomics.com", "language": "text" } ] } [/block] [block:callout] { "type": "danger", "body": "You should enter your [authentication token](http://docs.cancergenomicscloud.org/docs/get-your-authentication-token) in response to the password prompt, not your CGC password." } [/block] Finally, you need to push the image you have created to the CGC image registry: [block:code] { "codes": [ { "code": "docker push cgc-images.sbgenomics.com/rosalind_franklin/samtools:v1", "language": "text", "name": null } ] } [/block] Once the process has been completed, use the [Tool Editor](doc:the-tool-editor) to provide a description of the tool on the CGC. ###Creating a Dockerfile for an existing Docker image If you already have a Docker image and you want to create a Dockerfile for the image, [use Docker Hub to store your image](https://docs.docker.com/docker-hub/repos/#/pushing-a-repository-image-to-docker-hub), as follows: 1. Log in to Docker Hub: [block:code] { "codes": [ { "code": "docker login", "language": "text" } ] } [/block] 2. When prompted, enter your Docker Hub credentials. 3. Push your Docker image to Docker Hub: [block:code] { "codes": [ { "code": "docker push <image>", "language": "text" } ] } [/block] 4. Create a Dockerfile containing the following line: [block:code] { "codes": [ { "code": "FROM <image>", "language": "text" } ] } [/block] In this case, `<image>` is the reference to the image on Docker Hub, e.g. `rosalind_franklin/my_image`. You are now able to build the image using the Dockerfile, in the way described above.