{"__v":38,"_id":"5637f110daf0840d0011dcd5","category":{"__v":11,"_id":"5637e8b2fbe1c50d008cb078","pages":["5637e95797666c0d008656ba","5637ea64ee0ee60d0024ec0e","5637f110daf0840d0011dcd5","563d423e9799fb0d0004779b","5640ba77b9c4dd0d0097c827","5640bad42b14f70d0039b8cd","5640bae32b14f70d0039b8cf","5640bb0ceaede117005c99d8","565f6b770dc99e1900f24c73","56cc9eab8fa8b01b00b81f5a","56ddcb9ab4ac273200457c82"],"project":"55faf11ba62ba1170021a9a7","version":"55faf11ba62ba1170021a9aa","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2015-11-02T22:50:26.865Z","from_sync":false,"order":14,"slug":"describe-your-tool","title":"DESCRIBE YOUR TOOL"},"parentDoc":null,"project":"55faf11ba62ba1170021a9a7","user":"554340dfb7f4540d00fcef1d","version":{"__v":37,"_id":"55faf11ba62ba1170021a9aa","project":"55faf11ba62ba1170021a9a7","createdAt":"2015-09-17T16:58:03.490Z","releaseDate":"2015-09-17T16:58:03.490Z","categories":["55faf11ca62ba1170021a9ab","55faf8f4d0e22017005b8272","55faf91aa62ba1170021a9b5","55faf929a8a7770d00c2c0bd","55faf932a8a7770d00c2c0bf","55faf94b17b9d00d00969f47","55faf958d0e22017005b8274","55faf95fa8a7770d00c2c0c0","55faf96917b9d00d00969f48","55faf970a8a7770d00c2c0c1","55faf98c825d5f19001fa3a6","55faf99aa62ba1170021a9b8","55faf99fa62ba1170021a9b9","55faf9aa17b9d00d00969f49","55faf9b6a8a7770d00c2c0c3","55faf9bda62ba1170021a9ba","5604570090ee490d00440551","5637e8b2fbe1c50d008cb078","5649bb624fa1460d00780add","5671974d1b6b730d008b4823","5671979d60c8e70d006c9760","568e8eef70ca1f0d0035808e","56d0a2081ecc471500f1795e","56d4a0adde40c70b00823ea3","56d96b03dd90610b00270849","56fbb83d8f21c817002af880","573c811bee2b3b2200422be1","576bc92afb62dd20001cda85","5771811e27a5c20e00030dcd","5785191af3a10c0e009b75b0","57bdf84d5d48411900cd8dc0","57ff5c5dc135231700aed806","5804caf792398f0f00e77521","58458b4fba4f1c0f009692bb","586d3c287c6b5b2300c05055","58ef66d88646742f009a0216","58f5d52d7891630f00fe4e77"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"updates":["579179e57feea81700e5ab4c"],"next":{"pages":[],"description":""},"createdAt":"2015-11-02T23:26:08.556Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":7,"body":"The [tool editor](doc:the-tool-editor) allows you to describe the features and behavior of your command line tool. The resulting description is used to create an interface between your tool and other tools that can be run on the CGC.\n[block:callout]\n{\n  \"type\": \"warning\",\n  \"title\": \"On this page\",\n  \"body\": \"* [Overview](#section-overview)\\n* [Using dynamic expressions](#section-using-dynamic-expressions)\\n* [Predefined Javascript objects](#section-predefined-javascript-objects-)\\n* [Inspecting the `$job` object](#section-inspecting-the-job-object)\\n* [Examples of the `$job` object](#section-examples-of-the-job-object)\\n* [The `$self` object](#section-the-self-object)\\n* [Examples of `$self` usage](#section-examples-of-self-usage)\"\n}\n[/block]\n##Overview\n\nIf your tool has a certain behavior, controlled by a command line option argument, that you don't want to ever vary with executions, you can 'hard code' this into the tool description. On the other hand, if you want to chose the argument every time you run the tool, you can manually input it with each execution. But, in many cases, a middle way is more appropriate: although an argument should vary with executions, it is required do so in a deterministic way, dependent on other features of the tool execution.\n\nTo achieve this behavior, you can enter the argument in the tool description using a dynamic expression, rather than a literal. The dynamic expression can be given in terms of the inputs, outputs, or other aspects of a tool execution; the object that it refers to will then be determined at runtime on the basis of the values of these other objects.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Using dynamic expressions\nDynamic expressions are given in Javascript. You can enter a dynamic expression anywhere in the tool editor where you find the symbol **</>**. For example, you could enter the Javascript expression `1+2`, or a function body such as `{return 1+2}`.\n\nPlaces where you might want to use dynamic expressions include:\n  * Giving your tool's base command.\n  * Specifying the command options for your tool. Options related to tool inputs are set on the Inputs tab, and others are set in the Arguments field on the [General Information tab](general-tool-information#arg).\n  * Specifying stdin and stdout.\n  * Setting the resources allocated to your tool (memory and CPU number).\n  * Setting up the glob pattern to catch outputs, on the Outputs tab.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Predefined Javascript objects:\nSome Javascript objects have been defined in the scope of the tool execution, and are available for you to use in dynamic expressions to increase their expressivity. The objects, `$job` and `$self`, denote properties of the ongoing tool execution (the job), and the tool's inputs or outputs in a given execution. They are described below.\n\n###The $job object\nThe object `$job` refers to an ongoing tool execution. It has the properties `inputs` and `allocatedResources`, which, in turn, have further properties that describe the execution in question.\n\n* `inputs`\nThis is an object that denotes the inputs for the tool. [Recall that tool inputs include files and parameters](http://docs.cancergenomicscloud.org/docs/the-tool-editor#section-a-note-on-tool-inputs).The inputs object has the properties `<input_id>` and `<input_value>`.\n\n  *  `<input_id>` is the ID that you have labeled each input port with on the tool editor. \n  *  `<input_value>` refers to the object that is entered into the port named `<input_id>`.  `<input_value>` is set according to the following rule, illustrated in the table below: If the object entered into the port named `<input_id>` is any data type other than a file, then `<input_value>` is the object itself. If the object entered into the port `<input_id>` is a file, then `<input_value>` is four objects that describe, and uniquely determine, the file.\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"Data type input to the port named input_id\",\n    \"h-1\": \"`<input_value>`\",\n    \"0-0\": \"File\",\n    \"0-1\": \"The following four objects are the input_value:\\n  * `class`: this is always 'File'\\n  * `secondary files`: this is a list of secondary files (index files) attached to the file. It may be empty.\\n  * `path`: this is the file path.  \\n  * `size`: this is the file size.\",\n    \"1-0\": \"String\",\n    \"1-1\": \"The string entered to the input port.\",\n    \"2-0\": \"Enum\",\n    \"2-1\": \"The enumerator of the enumerated type that is entered to the input port labeled by `<input_id>`\",\n    \"3-0\": \"Int\",\n    \"3-1\": \"The integer entered to this input port.\",\n    \"4-0\": \"Float\",\n    \"4-1\": \"The float entered to this input port.\",\n    \"5-0\": \"Boolean\",\n    \"5-1\": \"The boolean value entered to this input port.\",\n    \"6-0\": \"Array\",\n    \"6-1\": \"The array entered to this input port.\"\n  },\n  \"cols\": 2,\n  \"rows\": 7,\n  \"sidebar\": true\n}\n[/block]\n* `allocatedResources`\nThis is the second property of the object `inputs`. it is an object with properties: `cpu` and `mem`:\n\n  * `cpu` represents the number of CPU cores that have been allocated to the tool for the execution in question.\n  * `mem` represents the memory in megabytes that has been allocated to the tool for the execution in question.\n[block:callout]\n{\n  \"type\": \"success\",\n  \"body\": \"Note that the required resources, [entered under 'Resources' on the General Information tab](general-tool-information#resources) differ from the `allocatedResources` object in `$job`: for optimization, more resources may be *allocated* to the tool than it actually *requires* to function. The resources actually allocated depend on the particular execution environment, whereas the required resources are a hard requirement of the tool. So, the value entered as a required resource will be less than or equal to the value of the `allocatedResources` object in `$job`.\",\n  \"title\": \"Required resources and `allocatedResources`\"\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Inspecting the $job object\nThe objects in `<input_value>` are defined at runtime, depending on which strings, ints, files, and so on are inputted to the tool. But we can use the Test tab to set dummy inputs, and then inspect the `$job` object.\nTo see the `$job` object for your tool description and test values:\n1. Click the cog icon for Settings in the top right hand corner of the Tool Editor\n2. Select the option </> Job JSON. \n\nYou will see a pop-up window displaying `$job`, expressed in Javascript Object Notation (JSON), for the test values entered. \n[block:image]\n{\n  \"images\": [\n    {\n      \"caption\": \"</> Job JSON\",\n      \"image\": [\n        \"https://files.readme.io/1ymlT0eaSGCtjkLSfSAR_Screen%20Shot%202015-05-20%20at%2017.06.14.jpeg\",\n        \"Screen Shot 2015-05-20 at 17.06.14.jpeg\",\n        \"490\",\n        \"366\",\n        \"#4284c4\",\n        \"\"\n      ]\n    }\n  ]\n}\n[/block]\nTo get a better understanding of what `input_id` and `input_value` denote, let's create a tool description in the editor that has input ports for different selected data types, and then view the `$job`.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Examples of the $job object\n###`$job` example 1: The $job for a a tool with ports for multiple data types\nWe'll create a tool that has input ports that take a range of different data types as input, and see how this affects `$job.inputs`.\n[block:image]\n{\n  \"images\": [\n    {\n      \"caption\": \"Example 1: entering input ports on the tool editor\",\n      \"image\": [\n        \"https://files.readme.io/s3FLzc3dTs2y7WTFUzqJ_Screen%20Shot%202015-09-21%20at%2016.46.18.png\",\n        \"Screen Shot 2015-09-21 at 16.46.18.png\",\n        \"1788\",\n        \"946\",\n        \"#487bb0\",\n        \"\"\n      ]\n    }\n  ]\n}\n[/block]\nFor example, create the following ports on the **Inputs** tab:\n  * Port ID: `my_file`, Input type: file\n  * Port ID: `my_string`, Input type: string\n  * Port ID: `my_enum`, Input type: enum; enumerations: red, yellow, green blue\n  * Port ID: `my_int`, Input type: int\n  * Port ID: `my_boolean`, Input type: boolean\n\nOn the Test Tab you can set the values of these ports. \n  * Set `my_string` to 'here is a test string'; \n  * Set `my_int` to 6;\n  * Set `my_boolean` to True (indicated by checking the box);\n  * Set `my_file `to test_file.etx;\n  * Set `my_enum` to yellow (selected from the enumerations we defined when creating the port).\nThese settings are shown in the screenshot of the **Test** tab, below:\n[block:image]\n{\n  \"images\": [\n    {\n      \"caption\": \"Example 1: entering values to be used as inputs\",\n      \"image\": [\n        \"https://files.readme.io/3NfCiC5aSCWKaMXmHBr9_Screen%20Shot%202015-09-21%20at%2017.01.41.jpeg\",\n        \"Screen Shot 2015-09-21 at 17.01.41.jpeg\",\n        \"2918\",\n        \"1144\",\n        \"#4c85bc\",\n        \"\"\n      ]\n    }\n  ]\n}\n[/block]\nNow that we have set the values for the objects inputted to the tool ports, let's look at the `$job` object. Click ** </> Job JSON** under the settings cog icon in the top right corner.\n\nThe resulting $job object in JSON looks like this:\n[block:image]\n{\n  \"images\": [\n    {\n      \"caption\": \"$job JSON for a tool with ports for multiple data types\",\n      \"image\": [\n        \"https://files.readme.io/R0HeI7uTRSqgNSawqB8R_Screen%20Shot%202015-09-21%20at%2017.13.51.jpeg\",\n        \"Screen Shot 2015-09-21 at 17.13.51.jpeg\",\n        \"1236\",\n        \"982\",\n        \"#4580bb\",\n        \"\"\n      ]\n    }\n  ]\n}\n[/block]\nYou can see that each input port is associated with the value of the object that is inputted to it, as set on the **Test** tab. \n\nThe input port for files, `my_file`, is associated with the four objects that describe the file inputted to that port: `path`, `class`, `size`, and `secondaryFiles`. These associated objects are the `<input_value>` for the input ports.\n\nThe `allocatedResources` object in the `$job` also contains the same values as set (by default) on the **Test** tab.\nIf we were to execute this tool, the values of `<input_value>` would change to the actual strings, ints, files, and so on inputted to the tool.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###`$job` example 2: The $job for a tool whose ports are `reads` and `trim_size`\nHere is a more concrete example. Suppose that you have a tool whose input port IDs are `reads` and `trim_size`. In this case, `reads` takes files as input type, and `trim_size` takes integers. If we enter '10' for `trim_size`, and 'reads_file.ext' for `reads` in the **Test** Tab then the job object would look like this:\n[block:image]\n{\n  \"images\": [\n    {\n      \"caption\": \"$job JSON for a tool with ports `reads` and `trim_size`\",\n      \"image\": [\n        \"https://files.readme.io/Ln8uJ8gsSACNXXlqKwOg_Screen%20Shot%202015-09-21%20at%2017.26.54.jpeg\",\n        \"Screen Shot 2015-09-21 at 17.26.54.jpeg\",\n        \"1244\",\n        \"872\",\n        \"#4480bc\",\n        \"\"\n      ]\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Examples of `$job` usage\nEarlier, we listed fields in which you may want to use the $job object in dynamic expressions. Here are a few examples:\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###`$job` usage example 1: Extracting the filesize\nYou may need to set the tool's required memory (on the General Information tab) to be a function of the size of the inputted file. In this case, you can use the object `$job.<input_id>.size` to pick out the size of the file inputted to the port `<input_id>`.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###`$job` usage example 2: Extracting the number of threads\n You may need to pass the number of CPUs allocated to the tool in as an argument, for instance, to be the value of the argument for thread number, `num-threads`. In this case, in the [**Arguments** field in the General Information tab](general-tool-information#arg), enter following object to pick out the CPUs allocated to the job at runtime:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$job.allocatedResources.cpu\",\n      \"language\": \"javascript\"\n    }\n  ]\n}\n[/block]\nThis usage is illustrated below:\n[block:image]\n{\n  \"images\": [\n    {\n      \"caption\": \"Example 2: Enter the number of threads as a argument, using `$job.`\",\n      \"image\": [\n        \"https://files.readme.io/8WqH36AUSq6SC3DsGavz_Screen%20Shot%202015-06-08%20at%2013.25.30.jpeg\",\n        \"Screen Shot 2015-06-08 at 13.25.30.jpeg\",\n        \"1838\",\n        \"464\",\n        \"#a04d4d\",\n        \"\"\n      ]\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###`$job` usage example 3: Extracting file names and paths\nYou can use `$job` to perform string manipulation on file paths to extract file names. If, for example, you are copying files to the current working directory of the job, you will need to specify: (a) the file name and (b) the file to be copied. (These are specified in the field marked [**Create Files** on the General Information tab](general-tool-information#create files); see the section on Attaching Index Files for information on this use case.)\n\n(a) The file name for the file input to the tool is obtained from the path property of the <input_id> object for the input port that takes files. The path of the input file is:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$job.inputs.<input_id>.path\",\n      \"language\": \"javascript\"\n    }\n  ]\n}\n[/block]\nTo get the file name, we want to perform string manipulation on the file path; break the path up by forward slashes, using Javascript's `slice` operation, and then select just the final slice of the path. This string manipulation can be performed using the following expression: \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$job.inputs.<input_id>.path.split('/').slice(-1)[0]\",\n      \"language\": \"javascript\"\n    }\n  ]\n}\n[/block]\n(b) We can refer to the file itself using the  `<input_id> of the input port that takes files. The full expression to refer to the file is:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$job.inputs.<input_id>\",\n      \"language\": \"javascript\"\n    }\n  ]\n}\n[/block]\nBelow is a screenshot showing how these expressions would be entered in the Create Files fields:\n[block:image]\n{\n  \"images\": [\n    {\n      \"caption\": \"Example 2: Using string manipulation on `$job` to enter files\",\n      \"image\": [\n        \"https://files.readme.io/EEQfjscGSV9d7weVAsNc_Screen%20Shot%202015-09-24%20at%2012.43.32.jpeg\",\n        \"Screen Shot 2015-09-24 at 12.43.32.jpeg\",\n        \"1664\",\n        \"326\",\n        \"#d65252\",\n        \"\"\n      ]\n    }\n  ]\n}\n[/block]\nFinally, the `$job` object is used to indicate errors in an imported tool. If a tool referred to in a tool description contains errors, such as a cyclical loop, these will be shown in the online editor and added to the JSON object displayed in the **`$job` JSON**, with the property `sbg:errors`.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##The `$self` object\nLike `$job`, the `$self` object is another hard-coded Javascript object you can use to create dynamic expressions in tool descriptions. `$self` is used to modify the value of the input or output that gets passed to the command line. Its reference is different, depending on whether it occurs in command line bindings for the tool input or output.\n\nIn the command line bindings for the tool inputs, $self is set to the value of the input. In other words, in this context `$self` is just `$job.inputs.<input_id>`.\nIn the command line bindings for the tool outputs, `$self` is  an object with the properties `path` and `size` that match the globbed file (see here for more information on globbing).\n\n##Examples of `$self` usage\nHere are a few examples of how you can use the `$self` object:\n\n`$self` in the Inputs tab\n\n  * Use `$self` for string manipulation. Suppose that your tool takes input files that are entered on the command line, but expects these to be given as filenames, not as file paths. Since `$self` refers to the tool input, in the case that files are the input type `$self` will refer to a full file path. By performing string manipulation on this file path, as in the example above, you can extract the filename. Then, enter this as the input value to the tool.\n  \n  * Another use case for `$self` is to manipulate an integer input. Suppose that your tool expects integer inputs in exponent notation, but inputs are given as integers. In this case, `$self` will refer to the integer inputted, and you can use an expression in terms of `$self` in the Inputs value field to give the integer in its exponent representation.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n`$self` in the Outputs tab\n\n  * In addition to using $self to refer to a tool's inputs, you can use `$self.path` on the **Outputs** tab in the metadata field to specify to the path of the file matching the glob pattern. You might want to annotate output files with the metadata field 'file type', whose value depends on the particular extension of the file being output by the tool. In this case, set the metadata key to `file_type` and set the value to a Javascript expression that picks out the part of the filepath following the dot, i.e., the file extension. The following expression will do:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$self.path.split('.').slice(-1)[0]\",\n      \"language\": \"javascript\"\n    }\n  ]\n}\n[/block]\n\n[block:callout]\n{\n  \"type\": \"success\",\n  \"body\": \"Metadata fields and their values are entered as key-value pairs on the outputs tab; for details, see the section Annotating output files with metadata.\",\n  \"title\": \"Annotating output files with metadata\"\n}\n[/block]\nThe example just described is shown on the screenshot below:\n[block:image]\n{\n  \"images\": [\n    {\n      \"caption\": \"Using `$self` to add `file-type` metadata to output files\",\n      \"image\": [\n        \"https://files.readme.io/x0USh4WRBatTENevhKtw_--1002.jpeg\",\n        \"--1002.jpeg\",\n        \"1838\",\n        \"1364\",\n        \"#cf5758\",\n        \"\"\n      ]\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>","excerpt":"Create arguments that vary depending on features of the job, inputs, or outputs","slug":"dynamic-expressions-in-tool-descriptions","type":"basic","title":"Dynamic Expressions in Tool Descriptions"}

Dynamic Expressions in Tool Descriptions

Create arguments that vary depending on features of the job, inputs, or outputs

The [tool editor](doc:the-tool-editor) allows you to describe the features and behavior of your command line tool. The resulting description is used to create an interface between your tool and other tools that can be run on the CGC. [block:callout] { "type": "warning", "title": "On this page", "body": "* [Overview](#section-overview)\n* [Using dynamic expressions](#section-using-dynamic-expressions)\n* [Predefined Javascript objects](#section-predefined-javascript-objects-)\n* [Inspecting the `$job` object](#section-inspecting-the-job-object)\n* [Examples of the `$job` object](#section-examples-of-the-job-object)\n* [The `$self` object](#section-the-self-object)\n* [Examples of `$self` usage](#section-examples-of-self-usage)" } [/block] ##Overview If your tool has a certain behavior, controlled by a command line option argument, that you don't want to ever vary with executions, you can 'hard code' this into the tool description. On the other hand, if you want to chose the argument every time you run the tool, you can manually input it with each execution. But, in many cases, a middle way is more appropriate: although an argument should vary with executions, it is required do so in a deterministic way, dependent on other features of the tool execution. To achieve this behavior, you can enter the argument in the tool description using a dynamic expression, rather than a literal. The dynamic expression can be given in terms of the inputs, outputs, or other aspects of a tool execution; the object that it refers to will then be determined at runtime on the basis of the values of these other objects. <div align="right"><a href="#top">top</a></div> ##Using dynamic expressions Dynamic expressions are given in Javascript. You can enter a dynamic expression anywhere in the tool editor where you find the symbol **</>**. For example, you could enter the Javascript expression `1+2`, or a function body such as `{return 1+2}`. Places where you might want to use dynamic expressions include: * Giving your tool's base command. * Specifying the command options for your tool. Options related to tool inputs are set on the Inputs tab, and others are set in the Arguments field on the [General Information tab](general-tool-information#arg). * Specifying stdin and stdout. * Setting the resources allocated to your tool (memory and CPU number). * Setting up the glob pattern to catch outputs, on the Outputs tab. <div align="right"><a href="#top">top</a></div> ##Predefined Javascript objects: Some Javascript objects have been defined in the scope of the tool execution, and are available for you to use in dynamic expressions to increase their expressivity. The objects, `$job` and `$self`, denote properties of the ongoing tool execution (the job), and the tool's inputs or outputs in a given execution. They are described below. ###The $job object The object `$job` refers to an ongoing tool execution. It has the properties `inputs` and `allocatedResources`, which, in turn, have further properties that describe the execution in question. * `inputs` This is an object that denotes the inputs for the tool. [Recall that tool inputs include files and parameters](http://docs.cancergenomicscloud.org/docs/the-tool-editor#section-a-note-on-tool-inputs).The inputs object has the properties `<input_id>` and `<input_value>`. * `<input_id>` is the ID that you have labeled each input port with on the tool editor. * `<input_value>` refers to the object that is entered into the port named `<input_id>`. `<input_value>` is set according to the following rule, illustrated in the table below: If the object entered into the port named `<input_id>` is any data type other than a file, then `<input_value>` is the object itself. If the object entered into the port `<input_id>` is a file, then `<input_value>` is four objects that describe, and uniquely determine, the file. [block:parameters] { "data": { "h-0": "Data type input to the port named input_id", "h-1": "`<input_value>`", "0-0": "File", "0-1": "The following four objects are the input_value:\n * `class`: this is always 'File'\n * `secondary files`: this is a list of secondary files (index files) attached to the file. It may be empty.\n * `path`: this is the file path. \n * `size`: this is the file size.", "1-0": "String", "1-1": "The string entered to the input port.", "2-0": "Enum", "2-1": "The enumerator of the enumerated type that is entered to the input port labeled by `<input_id>`", "3-0": "Int", "3-1": "The integer entered to this input port.", "4-0": "Float", "4-1": "The float entered to this input port.", "5-0": "Boolean", "5-1": "The boolean value entered to this input port.", "6-0": "Array", "6-1": "The array entered to this input port." }, "cols": 2, "rows": 7, "sidebar": true } [/block] * `allocatedResources` This is the second property of the object `inputs`. it is an object with properties: `cpu` and `mem`: * `cpu` represents the number of CPU cores that have been allocated to the tool for the execution in question. * `mem` represents the memory in megabytes that has been allocated to the tool for the execution in question. [block:callout] { "type": "success", "body": "Note that the required resources, [entered under 'Resources' on the General Information tab](general-tool-information#resources) differ from the `allocatedResources` object in `$job`: for optimization, more resources may be *allocated* to the tool than it actually *requires* to function. The resources actually allocated depend on the particular execution environment, whereas the required resources are a hard requirement of the tool. So, the value entered as a required resource will be less than or equal to the value of the `allocatedResources` object in `$job`.", "title": "Required resources and `allocatedResources`" } [/block] <div align="right"><a href="#top">top</a></div> ##Inspecting the $job object The objects in `<input_value>` are defined at runtime, depending on which strings, ints, files, and so on are inputted to the tool. But we can use the Test tab to set dummy inputs, and then inspect the `$job` object. To see the `$job` object for your tool description and test values: 1. Click the cog icon for Settings in the top right hand corner of the Tool Editor 2. Select the option </> Job JSON. You will see a pop-up window displaying `$job`, expressed in Javascript Object Notation (JSON), for the test values entered. [block:image] { "images": [ { "caption": "</> Job JSON", "image": [ "https://files.readme.io/1ymlT0eaSGCtjkLSfSAR_Screen%20Shot%202015-05-20%20at%2017.06.14.jpeg", "Screen Shot 2015-05-20 at 17.06.14.jpeg", "490", "366", "#4284c4", "" ] } ] } [/block] To get a better understanding of what `input_id` and `input_value` denote, let's create a tool description in the editor that has input ports for different selected data types, and then view the `$job`. <div align="right"><a href="#top">top</a></div> ##Examples of the $job object ###`$job` example 1: The $job for a a tool with ports for multiple data types We'll create a tool that has input ports that take a range of different data types as input, and see how this affects `$job.inputs`. [block:image] { "images": [ { "caption": "Example 1: entering input ports on the tool editor", "image": [ "https://files.readme.io/s3FLzc3dTs2y7WTFUzqJ_Screen%20Shot%202015-09-21%20at%2016.46.18.png", "Screen Shot 2015-09-21 at 16.46.18.png", "1788", "946", "#487bb0", "" ] } ] } [/block] For example, create the following ports on the **Inputs** tab: * Port ID: `my_file`, Input type: file * Port ID: `my_string`, Input type: string * Port ID: `my_enum`, Input type: enum; enumerations: red, yellow, green blue * Port ID: `my_int`, Input type: int * Port ID: `my_boolean`, Input type: boolean On the Test Tab you can set the values of these ports. * Set `my_string` to 'here is a test string'; * Set `my_int` to 6; * Set `my_boolean` to True (indicated by checking the box); * Set `my_file `to test_file.etx; * Set `my_enum` to yellow (selected from the enumerations we defined when creating the port). These settings are shown in the screenshot of the **Test** tab, below: [block:image] { "images": [ { "caption": "Example 1: entering values to be used as inputs", "image": [ "https://files.readme.io/3NfCiC5aSCWKaMXmHBr9_Screen%20Shot%202015-09-21%20at%2017.01.41.jpeg", "Screen Shot 2015-09-21 at 17.01.41.jpeg", "2918", "1144", "#4c85bc", "" ] } ] } [/block] Now that we have set the values for the objects inputted to the tool ports, let's look at the `$job` object. Click ** </> Job JSON** under the settings cog icon in the top right corner. The resulting $job object in JSON looks like this: [block:image] { "images": [ { "caption": "$job JSON for a tool with ports for multiple data types", "image": [ "https://files.readme.io/R0HeI7uTRSqgNSawqB8R_Screen%20Shot%202015-09-21%20at%2017.13.51.jpeg", "Screen Shot 2015-09-21 at 17.13.51.jpeg", "1236", "982", "#4580bb", "" ] } ] } [/block] You can see that each input port is associated with the value of the object that is inputted to it, as set on the **Test** tab. The input port for files, `my_file`, is associated with the four objects that describe the file inputted to that port: `path`, `class`, `size`, and `secondaryFiles`. These associated objects are the `<input_value>` for the input ports. The `allocatedResources` object in the `$job` also contains the same values as set (by default) on the **Test** tab. If we were to execute this tool, the values of `<input_value>` would change to the actual strings, ints, files, and so on inputted to the tool. <div align="right"><a href="#top">top</a></div> ###`$job` example 2: The $job for a tool whose ports are `reads` and `trim_size` Here is a more concrete example. Suppose that you have a tool whose input port IDs are `reads` and `trim_size`. In this case, `reads` takes files as input type, and `trim_size` takes integers. If we enter '10' for `trim_size`, and 'reads_file.ext' for `reads` in the **Test** Tab then the job object would look like this: [block:image] { "images": [ { "caption": "$job JSON for a tool with ports `reads` and `trim_size`", "image": [ "https://files.readme.io/Ln8uJ8gsSACNXXlqKwOg_Screen%20Shot%202015-09-21%20at%2017.26.54.jpeg", "Screen Shot 2015-09-21 at 17.26.54.jpeg", "1244", "872", "#4480bc", "" ] } ] } [/block] <div align="right"><a href="#top">top</a></div> ##Examples of `$job` usage Earlier, we listed fields in which you may want to use the $job object in dynamic expressions. Here are a few examples: <div align="right"><a href="#top">top</a></div> ###`$job` usage example 1: Extracting the filesize You may need to set the tool's required memory (on the General Information tab) to be a function of the size of the inputted file. In this case, you can use the object `$job.<input_id>.size` to pick out the size of the file inputted to the port `<input_id>`. <div align="right"><a href="#top">top</a></div> ###`$job` usage example 2: Extracting the number of threads You may need to pass the number of CPUs allocated to the tool in as an argument, for instance, to be the value of the argument for thread number, `num-threads`. In this case, in the [**Arguments** field in the General Information tab](general-tool-information#arg), enter following object to pick out the CPUs allocated to the job at runtime: [block:code] { "codes": [ { "code": "$job.allocatedResources.cpu", "language": "javascript" } ] } [/block] This usage is illustrated below: [block:image] { "images": [ { "caption": "Example 2: Enter the number of threads as a argument, using `$job.`", "image": [ "https://files.readme.io/8WqH36AUSq6SC3DsGavz_Screen%20Shot%202015-06-08%20at%2013.25.30.jpeg", "Screen Shot 2015-06-08 at 13.25.30.jpeg", "1838", "464", "#a04d4d", "" ] } ] } [/block] <div align="right"><a href="#top">top</a></div> ###`$job` usage example 3: Extracting file names and paths You can use `$job` to perform string manipulation on file paths to extract file names. If, for example, you are copying files to the current working directory of the job, you will need to specify: (a) the file name and (b) the file to be copied. (These are specified in the field marked [**Create Files** on the General Information tab](general-tool-information#create files); see the section on Attaching Index Files for information on this use case.) (a) The file name for the file input to the tool is obtained from the path property of the <input_id> object for the input port that takes files. The path of the input file is: [block:code] { "codes": [ { "code": "$job.inputs.<input_id>.path", "language": "javascript" } ] } [/block] To get the file name, we want to perform string manipulation on the file path; break the path up by forward slashes, using Javascript's `slice` operation, and then select just the final slice of the path. This string manipulation can be performed using the following expression: [block:code] { "codes": [ { "code": "$job.inputs.<input_id>.path.split('/').slice(-1)[0]", "language": "javascript" } ] } [/block] (b) We can refer to the file itself using the `<input_id> of the input port that takes files. The full expression to refer to the file is: [block:code] { "codes": [ { "code": "$job.inputs.<input_id>", "language": "javascript" } ] } [/block] Below is a screenshot showing how these expressions would be entered in the Create Files fields: [block:image] { "images": [ { "caption": "Example 2: Using string manipulation on `$job` to enter files", "image": [ "https://files.readme.io/EEQfjscGSV9d7weVAsNc_Screen%20Shot%202015-09-24%20at%2012.43.32.jpeg", "Screen Shot 2015-09-24 at 12.43.32.jpeg", "1664", "326", "#d65252", "" ] } ] } [/block] Finally, the `$job` object is used to indicate errors in an imported tool. If a tool referred to in a tool description contains errors, such as a cyclical loop, these will be shown in the online editor and added to the JSON object displayed in the **`$job` JSON**, with the property `sbg:errors`. <div align="right"><a href="#top">top</a></div> ##The `$self` object Like `$job`, the `$self` object is another hard-coded Javascript object you can use to create dynamic expressions in tool descriptions. `$self` is used to modify the value of the input or output that gets passed to the command line. Its reference is different, depending on whether it occurs in command line bindings for the tool input or output. In the command line bindings for the tool inputs, $self is set to the value of the input. In other words, in this context `$self` is just `$job.inputs.<input_id>`. In the command line bindings for the tool outputs, `$self` is an object with the properties `path` and `size` that match the globbed file (see here for more information on globbing). ##Examples of `$self` usage Here are a few examples of how you can use the `$self` object: `$self` in the Inputs tab * Use `$self` for string manipulation. Suppose that your tool takes input files that are entered on the command line, but expects these to be given as filenames, not as file paths. Since `$self` refers to the tool input, in the case that files are the input type `$self` will refer to a full file path. By performing string manipulation on this file path, as in the example above, you can extract the filename. Then, enter this as the input value to the tool. * Another use case for `$self` is to manipulate an integer input. Suppose that your tool expects integer inputs in exponent notation, but inputs are given as integers. In this case, `$self` will refer to the integer inputted, and you can use an expression in terms of `$self` in the Inputs value field to give the integer in its exponent representation. <div align="right"><a href="#top">top</a></div> `$self` in the Outputs tab * In addition to using $self to refer to a tool's inputs, you can use `$self.path` on the **Outputs** tab in the metadata field to specify to the path of the file matching the glob pattern. You might want to annotate output files with the metadata field 'file type', whose value depends on the particular extension of the file being output by the tool. In this case, set the metadata key to `file_type` and set the value to a Javascript expression that picks out the part of the filepath following the dot, i.e., the file extension. The following expression will do: [block:code] { "codes": [ { "code": "$self.path.split('.').slice(-1)[0]", "language": "javascript" } ] } [/block] [block:callout] { "type": "success", "body": "Metadata fields and their values are entered as key-value pairs on the outputs tab; for details, see the section Annotating output files with metadata.", "title": "Annotating output files with metadata" } [/block] The example just described is shown on the screenshot below: [block:image] { "images": [ { "caption": "Using `$self` to add `file-type` metadata to output files", "image": [ "https://files.readme.io/x0USh4WRBatTENevhKtw_--1002.jpeg", "--1002.jpeg", "1838", "1364", "#cf5758", "" ] } ] } [/block] <div align="right"><a href="#top">top</a></div>