{"_id":"5a82ffe79d713b004485e2e7","project":"55faf11ba62ba1170021a9a7","version":{"_id":"55faf11ba62ba1170021a9aa","project":"55faf11ba62ba1170021a9a7","__v":40,"createdAt":"2015-09-17T16:58:03.490Z","releaseDate":"2015-09-17T16:58:03.490Z","categories":["55faf11ca62ba1170021a9ab","55faf8f4d0e22017005b8272","55faf91aa62ba1170021a9b5","55faf929a8a7770d00c2c0bd","55faf932a8a7770d00c2c0bf","55faf94b17b9d00d00969f47","55faf958d0e22017005b8274","55faf95fa8a7770d00c2c0c0","55faf96917b9d00d00969f48","55faf970a8a7770d00c2c0c1","55faf98c825d5f19001fa3a6","55faf99aa62ba1170021a9b8","55faf99fa62ba1170021a9b9","55faf9aa17b9d00d00969f49","55faf9b6a8a7770d00c2c0c3","55faf9bda62ba1170021a9ba","5604570090ee490d00440551","5637e8b2fbe1c50d008cb078","5649bb624fa1460d00780add","5671974d1b6b730d008b4823","5671979d60c8e70d006c9760","568e8eef70ca1f0d0035808e","56d0a2081ecc471500f1795e","56d4a0adde40c70b00823ea3","56d96b03dd90610b00270849","56fbb83d8f21c817002af880","573c811bee2b3b2200422be1","576bc92afb62dd20001cda85","5771811e27a5c20e00030dcd","5785191af3a10c0e009b75b0","57bdf84d5d48411900cd8dc0","57ff5c5dc135231700aed806","5804caf792398f0f00e77521","58458b4fba4f1c0f009692bb","586d3c287c6b5b2300c05055","58ef66d88646742f009a0216","58f5d52d7891630f00fe4e77","59a555bccdbd85001bfb1442","5a2a81f688574d001e9934f5","5b080c8d7833b20003ddbb6f"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"category":{"_id":"55faf958d0e22017005b8274","__v":8,"pages":["56268b3cc2781f0d00364bc0","5637bfa84dbdd919001b277d","5637e6fe97666c0d008656b6","5637e74acfaa870d00cdeb7a","5637e764f7e3990d007b2c52","5640baf604c3eb0d006fc43c","56cccb6894c8f00b00b83f24","56ddf68dba70f50e0033c6df"],"project":"55faf11ba62ba1170021a9a7","version":"55faf11ba62ba1170021a9aa","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2015-09-17T17:33:12.687Z","from_sync":false,"order":15,"slug":"bring-your-tools","title":"BRING YOUR TOOLS"},"user":"5613e4f8fdd08f2b00437620","githubsync":"","__v":0,"parentDoc":null,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2018-02-13T15:10:31.192Z","link_external":false,"link_url":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":2,"body":"##Overview\n\nThe following command line tool changes are implemented for CWL v1.0 in comparison to sbg:draft-2.\n\n## No more $job for BiX8\n`$job` does not exist in JavaScript expressions. Available contexts are `$inputs` and `$runtime`.\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"sbg:draft-2\",\n    \"h-1\": \"CWL v1.0\",\n    \"0-0\": \"`$job.inputs.input_bam.path`\",\n    \"0-1\": \"`$(inputs.input_bam.path)`\",\n    \"1-0\": \"`$job.allocatedResources.cpu`\",\n    \"1-1\": \"`$(runtime.cores)`\"\n  },\n  \"cols\": 2,\n  \"rows\": 2\n}\n[/block]\n##Entering a JS expression\n\nExpressions are denoted by the syntax `$(...)` or `${...}`. A code fragment wrapped in the `$(...)` syntax is used as a one-line expression. A code fragment wrapped in `${...}` behaves like expressions in sbg:draft-2.\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"sbg:draft-2\",\n    \"h-1\": \"CWL v1.0\",\n    \"0-0\": \"`$job.inputs.input_bam.path + ‘.vcf’`\",\n    \"0-1\": \"`$(inputs.input_bam.path).vcf`\",\n    \"1-0\": \"`{ return $job.inputs.input_bam.path + ‘.vcf’ }`\",\n    \"1-1\": \"`${ return inputs.input_bam.path + ‘.vcf’}`\"\n  },\n  \"cols\": 2,\n  \"rows\": 2\n}\n[/block]\n##Commands relating to file paths (basename, dirname, nameroot, nameext)\n\nExperience improved commands relating to file paths. For instance, use `.nameroot` to get the input basename. Learn more from CWL's [documentation](http://www.commonwl.org/v1.0/CommandLineTool.html#File).\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"Expression\",\n    \"h-1\": \"Return\",\n    \"0-0\": \"`$(inputs.input_bam.path)`\",\n    \"0-1\": \"`/path/to/file.sorted.bam`\",\n    \"1-0\": \"`$(inputs.input_bam.basename)`\",\n    \"1-1\": \"`file.sorted.bam`\",\n    \"2-0\": \"`$(inputs.input_bam.dirname)`\",\n    \"2-1\": \"`/path/to`\",\n    \"3-0\": \"`$(inputs.input_bam.nameroot)`\",\n    \"3-1\": \"`file.sorted`\",\n    \"4-0\": \"`$(inputs.input_bam.nameext)`\",\n    \"4-1\": \"`bam`\"\n  },\n  \"cols\": 2,\n  \"rows\": 5\n}\n[/block]\n##One-line expressions\n\nIf the input is a BAM and the output is a VCF, you can define the output name as `$(inputs.input_bam.nameroot).vcf`.\n\n##No expressions in the base command\n\nAll the expressions must be inserted via arguments.\n\nThis means that if you want to insert pre-commands (such as un-TAR reference files), you need to define the whole command through arguments and leave the base command empty.\n\n##Expressions in secondary files\n\nSecondary files can now be defined with an expression.\n\nFor example, if the input is VCF or VCF.GZ, the secondary file is either .idx or .tbi.: `$(self.nameext == 'gz' ? \"tbi\" : \"idx\")`.\n\n##InitialWorkDirRequirement\n\nYou can create a file in `workdir` on runtime or make a file available in the `workdir` on runtime using `InitialWorkDirRequirement`.\n\n###Create literal content file example\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n \\n  - class: InitialWorkDirRequirement\\n    listing:\\n    - entry: $(JSON.stringify(inputs))\\n      entryname: cwl.inputs.json\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n###Create expression content file example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n \\n  - class: InitialWorkDirRequirement\\n    listing:\\n    - entry: |-\\n \\n         Some contents\\n \\n      entryname: dynamic_input.txt\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n###Stage an input example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n \\n - class: InitialWorkDirRequirement\\n   listing:\\n   - \\\"$(inputs.bam)\\\"\\n   - \\\"$(inputs.fastq_list)\\\"\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n###Example with staging inputs and creating files\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n \\n - class: InitialWorkDirRequirement\\n   listing:\\n   - entry: $(JSON.stringify(inputs))\\n     entryname: cwl.inputs.json\\n   - entry: |-\\n     ${\\n        return JSON.stringify(inputs)\\n     }\\n     entryname: '${return \\\"_1_cwl.inputs.json\\\"}'\\n   - \\\"$(inputs.bam)\\\"\\n   - \\\"$(inputs.fastq_list)\\\"\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n##ExpressionLibRequirement\n\nUse `ExpressionLibRequirement` to write a JS function in one place and use it in multiple places.\n\nDefine a function as:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n - class: InlineJavascriptRequirement\\n \\n   expressionLib:\\n     - var ext = function(){\\n       var x = inputs.vcf.nameext == 'gz' ? \\\"tbi\\\" : \\\"idx\\\";\\n       return x\\n     };\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\nAnd call it in a different place:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"secondaryFiles:\\n ${\\n \\n    return ext()\\n \\n  }\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n##SchemaDefRequirement\n\nCustom input and output structures can be defined in `SchemaDefRequirement`.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"class: SchemaDefRequirement\\ntypes:\\n- name: FileRecord\\n  type: record\\n  fields:\\n    - name: file\\n      type: File\\n    - name: metadata\\n      type: map   \\n      values: string\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n##shellQuote\n\nKeep this as false.\n\n##EnvVarRequirement\n\nInstead of starting a command with `export MY_CUSTOM_VARIABLE=DzoniJovanovic`, use `EnvVarRequirement`.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n EnvVarRequirement:\\n   envDef:\\n     FILIP: $(inputs.last_name)\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n##ExpressionTool\n\nSimilar to command-line tool except it has no command line and does not start a Docker container. The only purpose of this is to reshape stuff.\n\nExample:\n* https://github.com/NCI-GDC/gdc-dnaseq-cwl/blob/master/tools/decider_bwa_expression.cwl\n* https://github.com/NCI-GDC/gdc-dnaseq-cwl/blob/master/tools/sort_scatter_expression.cwl\n\n##Input/Output Type Directory\n\nThe input can be type directory.\n\n##Input/Output Union Type\n\nAn input or output can be defined to be one or more possible types.\n\nExample: Intervals input string or BED file\n\n##Input/Output Format Ontology\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"inputs:\\n aligned_sequences:\\n   type: File\\n   label: Aligned sequences in BAM format\\n   format: edam:format_2572\\n \\n \\n \\n$namespaces:\\n edam: http://edamontology.org/\\n$schemas:\\n - http://edamontology.org/EDAM_1.18.owl\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n##SoftwareRequirement\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"hints:\\n SoftwareRequirement:\\n   packages:\\n     interproscan:\\n       specs: [ \\\"https://identifiers.org/rrid/RRID:SCR_005829\\\" ]\\n       version: [ \\\"5.21-60\\\" ]\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]","excerpt":"","slug":"cwl-v10-improvements-over-sbgdraft-2","type":"basic","title":"CWL v.1.0 improvements over sbg:draft-2"}

CWL v.1.0 improvements over sbg:draft-2


##Overview The following command line tool changes are implemented for CWL v1.0 in comparison to sbg:draft-2. ## No more $job for BiX8 `$job` does not exist in JavaScript expressions. Available contexts are `$inputs` and `$runtime`. [block:parameters] { "data": { "h-0": "sbg:draft-2", "h-1": "CWL v1.0", "0-0": "`$job.inputs.input_bam.path`", "0-1": "`$(inputs.input_bam.path)`", "1-0": "`$job.allocatedResources.cpu`", "1-1": "`$(runtime.cores)`" }, "cols": 2, "rows": 2 } [/block] ##Entering a JS expression Expressions are denoted by the syntax `$(...)` or `${...}`. A code fragment wrapped in the `$(...)` syntax is used as a one-line expression. A code fragment wrapped in `${...}` behaves like expressions in sbg:draft-2. [block:parameters] { "data": { "h-0": "sbg:draft-2", "h-1": "CWL v1.0", "0-0": "`$job.inputs.input_bam.path + ‘.vcf’`", "0-1": "`$(inputs.input_bam.path).vcf`", "1-0": "`{ return $job.inputs.input_bam.path + ‘.vcf’ }`", "1-1": "`${ return inputs.input_bam.path + ‘.vcf’}`" }, "cols": 2, "rows": 2 } [/block] ##Commands relating to file paths (basename, dirname, nameroot, nameext) Experience improved commands relating to file paths. For instance, use `.nameroot` to get the input basename. Learn more from CWL's [documentation](http://www.commonwl.org/v1.0/CommandLineTool.html#File). [block:parameters] { "data": { "h-0": "Expression", "h-1": "Return", "0-0": "`$(inputs.input_bam.path)`", "0-1": "`/path/to/file.sorted.bam`", "1-0": "`$(inputs.input_bam.basename)`", "1-1": "`file.sorted.bam`", "2-0": "`$(inputs.input_bam.dirname)`", "2-1": "`/path/to`", "3-0": "`$(inputs.input_bam.nameroot)`", "3-1": "`file.sorted`", "4-0": "`$(inputs.input_bam.nameext)`", "4-1": "`bam`" }, "cols": 2, "rows": 5 } [/block] ##One-line expressions If the input is a BAM and the output is a VCF, you can define the output name as `$(inputs.input_bam.nameroot).vcf`. ##No expressions in the base command All the expressions must be inserted via arguments. This means that if you want to insert pre-commands (such as un-TAR reference files), you need to define the whole command through arguments and leave the base command empty. ##Expressions in secondary files Secondary files can now be defined with an expression. For example, if the input is VCF or VCF.GZ, the secondary file is either .idx or .tbi.: `$(self.nameext == 'gz' ? "tbi" : "idx")`. ##InitialWorkDirRequirement You can create a file in `workdir` on runtime or make a file available in the `workdir` on runtime using `InitialWorkDirRequirement`. ###Create literal content file example [block:code] { "codes": [ { "code": "requirements:\n \n - class: InitialWorkDirRequirement\n listing:\n - entry: $(JSON.stringify(inputs))\n entryname: cwl.inputs.json", "language": "yaml" } ] } [/block] ###Create expression content file example: [block:code] { "codes": [ { "code": "requirements:\n \n - class: InitialWorkDirRequirement\n listing:\n - entry: |-\n \n Some contents\n \n entryname: dynamic_input.txt", "language": "yaml" } ] } [/block] ###Stage an input example: [block:code] { "codes": [ { "code": "requirements:\n \n - class: InitialWorkDirRequirement\n listing:\n - \"$(inputs.bam)\"\n - \"$(inputs.fastq_list)\"", "language": "yaml" } ] } [/block] ###Example with staging inputs and creating files [block:code] { "codes": [ { "code": "requirements:\n \n - class: InitialWorkDirRequirement\n listing:\n - entry: $(JSON.stringify(inputs))\n entryname: cwl.inputs.json\n - entry: |-\n ${\n return JSON.stringify(inputs)\n }\n entryname: '${return \"_1_cwl.inputs.json\"}'\n - \"$(inputs.bam)\"\n - \"$(inputs.fastq_list)\"", "language": "yaml" } ] } [/block] ##ExpressionLibRequirement Use `ExpressionLibRequirement` to write a JS function in one place and use it in multiple places. Define a function as: [block:code] { "codes": [ { "code": "requirements:\n - class: InlineJavascriptRequirement\n \n expressionLib:\n - var ext = function(){\n var x = inputs.vcf.nameext == 'gz' ? \"tbi\" : \"idx\";\n return x\n };", "language": "yaml" } ] } [/block] And call it in a different place: [block:code] { "codes": [ { "code": "secondaryFiles:\n ${\n \n return ext()\n \n }", "language": "yaml" } ] } [/block] ##SchemaDefRequirement Custom input and output structures can be defined in `SchemaDefRequirement`. [block:code] { "codes": [ { "code": "class: SchemaDefRequirement\ntypes:\n- name: FileRecord\n type: record\n fields:\n - name: file\n type: File\n - name: metadata\n type: map \n values: string", "language": "yaml" } ] } [/block] ##shellQuote Keep this as false. ##EnvVarRequirement Instead of starting a command with `export MY_CUSTOM_VARIABLE=DzoniJovanovic`, use `EnvVarRequirement`. [block:code] { "codes": [ { "code": "requirements:\n EnvVarRequirement:\n envDef:\n FILIP: $(inputs.last_name)", "language": "yaml" } ] } [/block] ##ExpressionTool Similar to command-line tool except it has no command line and does not start a Docker container. The only purpose of this is to reshape stuff. Example: * https://github.com/NCI-GDC/gdc-dnaseq-cwl/blob/master/tools/decider_bwa_expression.cwl * https://github.com/NCI-GDC/gdc-dnaseq-cwl/blob/master/tools/sort_scatter_expression.cwl ##Input/Output Type Directory The input can be type directory. ##Input/Output Union Type An input or output can be defined to be one or more possible types. Example: Intervals input string or BED file ##Input/Output Format Ontology [block:code] { "codes": [ { "code": "inputs:\n aligned_sequences:\n type: File\n label: Aligned sequences in BAM format\n format: edam:format_2572\n \n \n \n$namespaces:\n edam: http://edamontology.org/\n$schemas:\n - http://edamontology.org/EDAM_1.18.owl", "language": "yaml" } ] } [/block] ##SoftwareRequirement [block:code] { "codes": [ { "code": "hints:\n SoftwareRequirement:\n packages:\n interproscan:\n specs: [ \"https://identifiers.org/rrid/RRID:SCR_005829\" ]\n version: [ \"5.21-60\" ]", "language": "yaml" } ] } [/block]