{"__v":1,"_id":"57ff61d8e7218a1900673ceb","category":{"project":"55faf11ba62ba1170021a9a7","version":"55faf11ba62ba1170021a9aa","_id":"57ff5c5dc135231700aed806","__v":0,"sync":{"url":"","isSync":false},"reference":false,"createdAt":"2016-10-13T10:05:17.381Z","from_sync":false,"order":15,"slug":"tool-wrapping-troubleshooting","title":"TOOL WRAPPING TIPS AND TRICKS"},"parentDoc":null,"project":"55faf11ba62ba1170021a9a7","user":"5767bc73bb15f40e00a28777","version":{"__v":35,"_id":"55faf11ba62ba1170021a9aa","project":"55faf11ba62ba1170021a9a7","createdAt":"2015-09-17T16:58:03.490Z","releaseDate":"2015-09-17T16:58:03.490Z","categories":["55faf11ca62ba1170021a9ab","55faf8f4d0e22017005b8272","55faf91aa62ba1170021a9b5","55faf929a8a7770d00c2c0bd","55faf932a8a7770d00c2c0bf","55faf94b17b9d00d00969f47","55faf958d0e22017005b8274","55faf95fa8a7770d00c2c0c0","55faf96917b9d00d00969f48","55faf970a8a7770d00c2c0c1","55faf98c825d5f19001fa3a6","55faf99aa62ba1170021a9b8","55faf99fa62ba1170021a9b9","55faf9aa17b9d00d00969f49","55faf9b6a8a7770d00c2c0c3","55faf9bda62ba1170021a9ba","5604570090ee490d00440551","5637e8b2fbe1c50d008cb078","5649bb624fa1460d00780add","5671974d1b6b730d008b4823","5671979d60c8e70d006c9760","568e8eef70ca1f0d0035808e","56d0a2081ecc471500f1795e","56d4a0adde40c70b00823ea3","56d96b03dd90610b00270849","56fbb83d8f21c817002af880","573c811bee2b3b2200422be1","576bc92afb62dd20001cda85","5771811e27a5c20e00030dcd","5785191af3a10c0e009b75b0","57bdf84d5d48411900cd8dc0","57ff5c5dc135231700aed806","5804caf792398f0f00e77521","58458b4fba4f1c0f009692bb","586d3c287c6b5b2300c05055"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"updates":[],"next":{"pages":[],"description":""},"createdAt":"2016-10-13T10:28:40.089Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":0,"body":"This page will introduce a situation in which you may want to configure the standard procedure on the CGC for storing files.\n\n###Default handling of input and output files\nEach tool in a workflow can only write its output files to its own working directory. The tool has write access to further directories, but no file written to any other directory can be reported as an output file. When the outputs of one tool are used as inputs for a second 'downstream' tool, the downstream tool has read only access to the first tool's working directory. \n\nThis default handling of files has several implications, which it is often convenient to override:\n1. A tool cannot, in general, write to its input files since they are not in the tool's working directory. If you need your tool to write to its input files, you can copy them to the tool's working directory.\n2. A tool cannot in general report one of its input files as an output file. If you need your tool to pass through an input file as an output (without modifying the file), you can create a symbolic link from the input file to the tool's working directory.\n3. The file paths of a tool's input files must be given relative to the tool's working directory. If you need to include the file paths of a tool's input files as arguments, it is often easier to use a symbolic link from the input files to the tool's working directory to simplify the file paths.\n\n**Stage Input** is a setting on the Tool Editor that enables you to copy an input file to a tool's working directory, or to create a symbolic link from the file to the tool's working directory. See the [documentation on stage input](http://docs.cancergenomicscloud.org/docs/tool-input-ports#section-stage-input) for more details.\n\nThis page will consider example 3. We will use **Stage Input** in order to avoid using the full file path to a tool's input file that is located in an upstream tool's working directory. In particular, consider an index file produced by an aligner indexer that is used as a tool's input. Aligner indexers are a set of indexing tools that index reference files and output a TAR archive containing the reference and the index file(s). To use the index files from an aligner indexer as inputs for your tool, you will first need to make the TAR archive available in your tool's current working directory, and then unpack the archive file to get the individual index files. In this case, **Stage Input** makes the command line significantly simpler as the archive unpack command can use the file name only, rather than the full path to the file.\nThe procedures below will explain how to use **Stage Input** to make the TAR archive containing index files available in your tool's working directory. To use index files as inputs, you need to set a single file staged input port for your tool, which would create a symbolic link from your tool's working directory to the TAR archive. This setting is made in the Tool Editor, which you can access by navigating to the **Apps** tab in the project that contains your tool, and then clicking the pencil icon next to the tool.\n[block:callout]\n{\n  \"type\": \"warning\",\n  \"title\": \"On this page\",\n  \"body\": \"* Default handling of input and output files\\n* Make the indexing tool output available in your tool's current working directory\\n* Configure the tool to unpack the TAR archive\"\n}\n[/block]\n###Make the indexing tool output available in your tool's current working directory\n1. On the **Inputs** tab of the Tool Editor click the **+** button to add an input port.\n2. Set the **ID** of the input to e.g. **reference**. You can use another value as the ID (the field allows only alphanumeric characters and underscore), but note that you will need to modify the Javascript expression below to match the ID you have entered.\n3. Set the value in the **Type** field to **File**.\n4. In the **Label** field set the value which will be displayed in the visual interface.\n5. Set the value in the **File Types** field to **.TAR**.\n6. Under **Stage Input** select **Link**.\n7. Click **Save**.\nThis will create a symbolic link in your tool's working directory to the archive containing the index files.\n[block:callout]\n{\n  \"type\": \"info\",\n  \"body\": \"The procedure above can be adapted to create a symbolic link from other input files into a tool's working directory. To adapt the procedure, make sure to replace **.TAR** in step 5 with the extension of the input file(s) you are using.\"\n}\n[/block]\nOnce you have made the archive file available in your tool's working directory, configure your tool to unpack it.\n\n###Configure the tool to unpack the TAR archive\nThe following procedure explains how to configure your tool to unpack the input TAR archive.\n1. Navigate to the **Apps** tab in your project.\n2. Click the pencil icon next to the tool you want to configure.\n3. Navigate to the **General** tab in the Tool Editor.\n4. Click **+** in the **Base Command** section.\nIf the field(s) in the **Base Command** section have already been populated, copy the content of each field to the first blank field below it, until the very first field in the section becomes blank.\n5. Click **</>** next to the first field.\n6. Paste the following code:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"{\\n  var index_files_bundle = $job.inputs.reference.path.split('/').slice(-1)\\n  return 'tar -xf ' + index_files_bundle + ' ; '\\n}\",\n      \"language\": \"javascript\"\n    }\n  ]\n}\n[/block]\nThe first line of the expression retrieves the name of the archive file using the [`$job` object](doc:dynamic-expressions-in-tool-descriptions#section-the-job-object). The second line appends the retrieved file name to the command that will unpack the archive file.\n[block:callout]\n{\n  \"type\": \"warning\",\n  \"body\": \"This Javascript expression assumes that the ID of the input port that takes the TAR archive is `reference`. Please make sure to replace `reference` in the above code with the ID value of your tool's input port that takes the archive file.\"\n}\n[/block]\n7. Click **Save**.\n8. Click **Save** in the top-right corner of the Tool Editor.\nYour tool is now configured to unpack a TAR archive it receives as its input.","excerpt":"","slug":"make-files-available-in-your-tools-working-directory","type":"basic","title":"Make files available in your tool's working directory"}

Make files available in your tool's working directory


This page will introduce a situation in which you may want to configure the standard procedure on the CGC for storing files. ###Default handling of input and output files Each tool in a workflow can only write its output files to its own working directory. The tool has write access to further directories, but no file written to any other directory can be reported as an output file. When the outputs of one tool are used as inputs for a second 'downstream' tool, the downstream tool has read only access to the first tool's working directory. This default handling of files has several implications, which it is often convenient to override: 1. A tool cannot, in general, write to its input files since they are not in the tool's working directory. If you need your tool to write to its input files, you can copy them to the tool's working directory. 2. A tool cannot in general report one of its input files as an output file. If you need your tool to pass through an input file as an output (without modifying the file), you can create a symbolic link from the input file to the tool's working directory. 3. The file paths of a tool's input files must be given relative to the tool's working directory. If you need to include the file paths of a tool's input files as arguments, it is often easier to use a symbolic link from the input files to the tool's working directory to simplify the file paths. **Stage Input** is a setting on the Tool Editor that enables you to copy an input file to a tool's working directory, or to create a symbolic link from the file to the tool's working directory. See the [documentation on stage input](http://docs.cancergenomicscloud.org/docs/tool-input-ports#section-stage-input) for more details. This page will consider example 3. We will use **Stage Input** in order to avoid using the full file path to a tool's input file that is located in an upstream tool's working directory. In particular, consider an index file produced by an aligner indexer that is used as a tool's input. Aligner indexers are a set of indexing tools that index reference files and output a TAR archive containing the reference and the index file(s). To use the index files from an aligner indexer as inputs for your tool, you will first need to make the TAR archive available in your tool's current working directory, and then unpack the archive file to get the individual index files. In this case, **Stage Input** makes the command line significantly simpler as the archive unpack command can use the file name only, rather than the full path to the file. The procedures below will explain how to use **Stage Input** to make the TAR archive containing index files available in your tool's working directory. To use index files as inputs, you need to set a single file staged input port for your tool, which would create a symbolic link from your tool's working directory to the TAR archive. This setting is made in the Tool Editor, which you can access by navigating to the **Apps** tab in the project that contains your tool, and then clicking the pencil icon next to the tool. [block:callout] { "type": "warning", "title": "On this page", "body": "* Default handling of input and output files\n* Make the indexing tool output available in your tool's current working directory\n* Configure the tool to unpack the TAR archive" } [/block] ###Make the indexing tool output available in your tool's current working directory 1. On the **Inputs** tab of the Tool Editor click the **+** button to add an input port. 2. Set the **ID** of the input to e.g. **reference**. You can use another value as the ID (the field allows only alphanumeric characters and underscore), but note that you will need to modify the Javascript expression below to match the ID you have entered. 3. Set the value in the **Type** field to **File**. 4. In the **Label** field set the value which will be displayed in the visual interface. 5. Set the value in the **File Types** field to **.TAR**. 6. Under **Stage Input** select **Link**. 7. Click **Save**. This will create a symbolic link in your tool's working directory to the archive containing the index files. [block:callout] { "type": "info", "body": "The procedure above can be adapted to create a symbolic link from other input files into a tool's working directory. To adapt the procedure, make sure to replace **.TAR** in step 5 with the extension of the input file(s) you are using." } [/block] Once you have made the archive file available in your tool's working directory, configure your tool to unpack it. ###Configure the tool to unpack the TAR archive The following procedure explains how to configure your tool to unpack the input TAR archive. 1. Navigate to the **Apps** tab in your project. 2. Click the pencil icon next to the tool you want to configure. 3. Navigate to the **General** tab in the Tool Editor. 4. Click **+** in the **Base Command** section. If the field(s) in the **Base Command** section have already been populated, copy the content of each field to the first blank field below it, until the very first field in the section becomes blank. 5. Click **</>** next to the first field. 6. Paste the following code: [block:code] { "codes": [ { "code": "{\n var index_files_bundle = $job.inputs.reference.path.split('/').slice(-1)\n return 'tar -xf ' + index_files_bundle + ' ; '\n}", "language": "javascript" } ] } [/block] The first line of the expression retrieves the name of the archive file using the [`$job` object](doc:dynamic-expressions-in-tool-descriptions#section-the-job-object). The second line appends the retrieved file name to the command that will unpack the archive file. [block:callout] { "type": "warning", "body": "This Javascript expression assumes that the ID of the input port that takes the TAR archive is `reference`. Please make sure to replace `reference` in the above code with the ID value of your tool's input port that takes the archive file." } [/block] 7. Click **Save**. 8. Click **Save** in the top-right corner of the Tool Editor. Your tool is now configured to unpack a TAR archive it receives as its input.