About the workflow editor

About the workflow editor

The workflow editor is a visual editor for creating and editing CWL workflows.

You can follow this simple tutorial to learn more about the workflow editor.

Workflow editor layout

This example shows a public workflow, RSEM with STAR workflow 1.3.1, in the workflow editor.

1. Navigation pane tabs allow you to select from My Projects and Public Apps. There is also a search box in this area, and for public apps (selected in this example), the option to group the apps by toolkit or by category.
2. Navigation pane showing (in this example) the public apps on the CGC, grouped by toolkit.
3. Tabs for workflow editor views (App Info, Visual Editor and Code).
4. Commands available for workflows (Revision history, Save, Run and additional actions where you can export your app to JSON or YAML and set hints).
5. Workflow editor pane showing a public workflow from the CGC, RSEM with STAR workflow 1.3.1.
6. Object inspector for tool node RSEM Prepare Reference in the workflow.
7. Editor functions ( to remove the node, + to zoom in, - to zoom out, to fit the image to the screen, to automatically arrange the layout and to export the workflow as an SVG image).
8. Workflow status bar, showing the CWL version and the error state.

Editor modes

The workflow editor has the following editing modes (displayed as tabs at the top of the editor window):

• Visual Editor view, which shows a visual representation of the workflow and allows you to edit the workflow by dragging and dropping apps and connections.
• Code view, which shows the underlying CWL.
• App Info view allows you to see and set general information about the workflow, for example, the author, and a description of the workflow. This information will be available to anyone who uses the workflow or places it as a component in a larger workflow.

Changes made in one view are immediately reflected in the other view so you can use whichever view is most convenient.

Creating a workflow

To create a workflow, follow the workflow creation procedure. You will be able to choose whether to use the older CWL sbg:draft-2 version, or the latest CWL v1.0 version. Read about CWL formats and versions to decide which CWL file format and version to choose.

📘

To ensure optimal consistency and functioning of your workflows, it is not recommended to mix CWL v1.0 and sbg:draft-2 apps in the same workflow.

Adding apps to a workflow

To add an existing app (tool or workflow) to a workflow, in the navigation pane, select the app you want to add and drag it into the workflow editor.

Connecting apps in a workflow

To connect an output from one app to an input in another, hover over the output port on the first app and drag it to the input port you want to connect it to on the second app. As you start dragging, any other ports that are of the right type to accept the connection are shown in green.

Note that, to avoid clutter, the editor only shows mandatory ports in the workflow. You can click on the app node to see a full list of ports in the object inspector. From here, you can click the toggle to show any other ports you want to include.

Adding workflow inputs and outputs

To define an input to the entire workflow, identify the app input node that you want to make into a workflow input. Hover over it and drag it to the left of the app node. This creates a workflow input as a node in the workflow editor.

Similarly, to define an output from the entire workflow, identify the app output node that you want to make into a workflow output. Hover over it and drag it to the right of the app node to create the workflow output as a node in in the workflow editor.

Deleting items

To delete an app, input, output or connection in the workflow, click to select the item you want to delete. Then click the bin icon in the bottom-right corner of the canvas. Optionally you can also press Backspace on the keyboard to do the same.

Setting item properties

To see and set more properties for an app node, workflow input or workflow output, double-click on the item. This displays additional information in the object inspector on the right hand side of the workflow editor.

The information shown in the object inspector depends on the item that has been selected. For example, if you select an input port, you will see the ID, and the data type. If you select a tool, you will see the tool file inputs and parameters.

Configure file inputs
The Files section in the object inspector shows all file inputs for the selected node in the workflow. The following options are available for a file input:

• Show/Hide - Select whether a file input will be shown in the visual editor. This option is available only for non-required file inputs.
• Link Merge Method - Defines how the input values are handled if the input port is connected to multiple sources.
  • Nested - The inputs are merged as an array consisting of array(s) and/or single values as its elements, depending on the provided source parameters.
  • Flattened - The input values are flattened into a single array of files by concatenating source parameters which are arrays and appending source parameters that are single elements.

Configure app parameters
The App Parameters section in the object inspector shows all non-file input parameters for the selected node in the workflow. The following options are available for an app parameter:

• Click the dropdown to the right of the parameter label to define how the parameter value is provided:
  • Default - Allows you to provide a parameter value that can’t be changed when the workflow is executed.
  • Exposed - The parameter is exposed for editing on each execution of the workflow.
  • Port - The parameter becomes an input port that you can connect to an adequate output port of another node to obtain the value. Or, you can grab the port and drag it to the left to allow parameter value to be defined when executing the workflow, similar to the Exposed option above.
• Link Merge Method - Defines how the input values are handled if the input port is connected to multiple input sources.
  • Nested - The input values are merged as an array consisting of array(s) and/or single elements, depending on the provided source parameters.
  • Flattened - The input values are flattened into a single array by concatenating source parameters that are arrays and appending source parameters that are single elements.
    For example, there are two arrays, [1,2,3] and [4,5,6] that are provided as values for an app parameter. If Nested is selected as the merge method, the resulting value of the parameter will be [[1,2,3],[4,5,6]], which preserves the original structure of input data. However, Flattened will result in the value being [1,2,3,4,5,6], where the two input arrays are merged and flattened into a single one.

Configure step parameters

Scatter

If you are inputting a list to a node, you can set the tool to iterate through the list and run one job per list element. This is known as "scattering".

Scatter is a "parallel map" or "for-each" construct, since one or more inputs are iterated over, while the rest remain fixed. It is the equivalent of "currying" the node with all other inputs, and running a parallel map over the input marked as scatter. Note that this input needs to be a list.

If you instruct a tool to scatter on an n element list, it will perform n jobs, and output an n-element array.

Configuring scattering in a CWL sbg:draft-2 workflow

To use scattering in an sbg:draft-2 workflow:

1. In the workflow editor, double-click the node whose input you want to scatter.
2. In the object inspector on the right click the Step tab.
3. Under Scatter select the input port you want to scatter.

When executing the workflow, this will iterate through all of the items provided on the selected input port and create one job per item.

Configuring scattering in a CWL 1.x workflow

To use scattering in a CWL 1.x workflow:

1. In the workflow editor, double-click the node whose input(s) you want to scatter.
2. In the object inspector on the right click the Step tab.
3. Under Scatter, select the input port(s) you want to scatter. To select or deselect multiple ports, hold the Ctrl (Cmd) key while making the selection.
4. If you scatter over multiple input ports, you need to select the Scatter Method. Read more about different scatter methods.