refresh v1.1 schema

mr-c · mr-c · commit 9e7e73dfa74a · 2019-02-27T17:24:56.000+02:00
diff --git a/cwltool/schemas/v1.1.0-dev1/CommandLineTool.yml b/cwltool/schemas/v1.1.0-dev1/CommandLineTool.yml
@@ -103,6 +103,13 @@ $graph:
           `Directory` listings should be loaded for use in expressions.
         * The position field of the [CommandLineBinding](#CommandLineBinding) can
           now be calculated from a CWL Expression.
+        * The exit code of a CommandLineTool invocation is now
+          available to expressions in `outputEval` as `runtime.exitCode`
+        * [Better explain](#map) the `map<…>` notation that has existed since v1.0.
+        * Fixed schema error where the `type` field inside the `inputs` and
+          `outputs` field was incorrectly listed as optional.
+        * For multi-Process CWL documents, if no particular process is named then
+          the process with the `id` of `#main` is chosen.
 
       See also the [CWL Workflow Description, v1.1.0-dev1 changelog](Workflow.html#Changelog).
 
@@ -325,13 +332,15 @@ $graph:
         - string
         - Expression
       doc: |
-        Evaluate an expression to generate the output value.  If `glob` was
-        specified, the value of `self` must be an array containing file objects
-        that were matched.  If no files were matched, `self` must be a zero
-        length array; if a single file was matched, the value of `self` is an
-        array of a single element.  Additionally, if `loadContents` is `true`,
-        the File objects must include up to the first 64 KiB of file contents
-        in the `contents` field.
+        Evaluate an expression to generate the output value.  If
+        `glob` was specified, the value of `self` must be an array
+        containing file objects that were matched.  If no files were
+        matched, `self` must be a zero length array; if a single file
+        was matched, the value of `self` is an array of a single
+        element.  Additionally, if `loadContents` is `true`, the File
+        objects must include up to the first 64 KiB of file contents
+        in the `contents` field.  The exit code of the process is
+        available in the expression as `runtime.exitCode`.
 
 - name: CommandLineBindable
   type: record
@@ -445,7 +454,6 @@ $graph:
   fields:
     - name: type
       type:
-        - "null"
         - CWLType
         - stdin
         - CommandInputRecordSchema
@@ -480,7 +488,6 @@ $graph:
   fields:
     - name: type
       type:
-        - "null"
         - CWLType
         - stdout
         - stderr
diff --git a/cwltool/schemas/v1.1.0-dev1/Workflow.yml b/cwltool/schemas/v1.1.0-dev1/Workflow.yml
@@ -88,6 +88,12 @@ $graph:
         * [Added](#LoadListingRequirement) `LoadListingRequirement`
           and [loadListing](#LoadContents) to control whether and how
           `Directory` listings should be loaded for use in expressions.
+        * [Better explain](#map) the `map<…>` notation that has existed since v1.0.
+        * Fixed schema error where the `type` field inside the `inputs` and
+          `outputs` fields for both `Workflow` and `ExpressionTool` was
+           incorrectly listed as optional.
+        * For multi-Process CWL documents, if no particular process is named then
+          the process with the `id` of `#main` is chosen.
 
       See also the [CWL Command Line Tool Description, v1.1.0-dev1 changelog](CommandLineTool.html#Changelog).
 
@@ -110,7 +116,6 @@ $graph:
   fields:
     - name: type
       type:
-        - "null"
         - CWLType
         - OutputRecordSchema
         - OutputEnumSchema
@@ -139,7 +144,6 @@ $graph:
   fields:
     - name: type
       type:
-        - "null"
         - CWLType
         - InputRecordSchema
         - InputEnumSchema
@@ -235,7 +239,6 @@ $graph:
         If not specified, the default method is "merge_nested".
     - name: type
       type:
-        - "null"
         - CWLType
         - OutputRecordSchema
         - OutputEnumSchema
diff --git a/cwltool/schemas/v1.1.0-dev1/concepts.md b/cwltool/schemas/v1.1.0-dev1/concepts.md
@@ -91,6 +91,185 @@ An implementation may formally validate the structure of a CWL document using
 SALAD schemas located at
 https://github.com/common-workflow-language/common-workflow-language/tree/master/v1.1.0-dev1
 
+### map
+
+Note: This section is non-normative.
+> type: array&lt;ComplexType&gt; |  
+> map&lt;`key_field`, ComplexType&gt;
+
+The above syntax in the CWL specifications means there are two or more ways to write the given value.
+
+Option one is a array and is the most verbose option.
+
+Option one generic example:
+```
+some_cwl_field:
+  - key_field: a_complex_type1
+    field2: foo
+    field3: bar
+  - key_field: a_complex_type2
+    field2: foo2
+    field3: bar2
+  - key_field: a_complex_type3
+```
+
+Option one specific example using [Workflow](Workflow.html#Workflow).[inputs](Workflow.html#WorkflowInputParameter):
+> array&lt;InputParameter&gt; |  
+> map&lt;`id`, `type` | InputParameter&gt;
+
+
+```
+inputs:
+  - id: workflow_input01
+    type: string
+  - id: workflow_input02
+    type: File
+    format: http://edamontology.org/format_2572
+```
+
+Option two is enabled by the `map<…>` syntax. Instead of an array of entries we
+use a mapping, where one field of the `ComplexType` (here named `key_field`)
+becomes the key in the map, and its value is the rest of the `ComplexType`
+without the key field. If all of the other fields of the `ComplexType` are
+optional and unneeded, then we can indicate this with an empty mapping as the
+value: `a_complex_type3: {}`
+
+Option two generic example:
+```
+some_cwl_field:
+  a_complex_type1:  # this was the "key_field" from above
+    field2: foo
+    field3: bar
+  a_complex_type2:
+    field2: foo2
+    field3: bar2
+  a_complex_type3: {}  # we accept the defualt values for "field2" and "field3"
+```
+
+Option two specific example using [Workflow](Workflow.html#Workflow).[inputs](Workflow.html#WorkflowInputParameter):
+> array&lt;InputParameter&gt; |  
+> map&lt;`id`, `type` | InputParameter&gt;
+
+
+```
+inputs:
+  workflow_input01:
+    type: string
+  workflow_input02:
+    type: File
+    format: http://edamontology.org/format_2572
+```
+
+Option two specific example using [SoftwareRequirement](#SoftwareRequirement).[packages](#SoftwarePackage):
+> array&lt;SoftwarePackage&gt; |  
+> map&lt;`package`, `specs` | SoftwarePackage&gt;
+
+
+```
+hints:
+  SoftwareRequirement:
+    packages:
+      sourmash:
+        specs: [ https://doi.org/10.21105/joss.00027 ]
+      screed:
+        version: [ "1.0" ]
+      python: {}
+```
+`
+Sometimes we have a third and even more compact option denoted like this:
+> type: array&lt;ComplexType&gt; |  
+> map&lt;`key_field`, `field2` | ComplexType&gt;
+
+For this example, if we only need the `key_field` and `field2` when specifying
+our `ComplexType`s (because the other fields are optional and we are fine with
+their default values) then we can abbreviate.
+
+Option three generic example:
+```
+some_cwl_field:
+  a_complex_type1: foo   # we accept the default value for field3
+  a_complex_type2: foo2  # we accept the default value for field3
+  a_complex_type3: {}    # we accept the default values for "field2" and "field3"
+```
+
+Option three specific example using [Workflow](Workflow.html#Workflow).[inputs](Workflow.html#WorkflowInputParameter):
+> array&lt;InputParameter&gt; |
+> map&lt;`id`, `type` | InputParameter&gt;
+
+
+```
+inputs:
+  workflow_input01: string
+  workflow_input02: File  # we accept the default of no File format
+```
+
+Option three specific example using [SoftwareRequirement](#SoftwareRequirement).[packages](#SoftwarePackage):
+> array&lt;SoftwarePackage&gt; |  
+> map&lt;`package`, `specs` | SoftwarePackage&gt;
+
+
+```
+hints:
+  SoftwareRequirement:
+    packages:
+      sourmash: [ https://doi.org/10.21105/joss.00027 ]
+      python: {}
+```
+
+
+What if some entries we want to mix the option 2 and 3? You can!
+
+Mixed option 2 and 3 generic example:
+```
+some_cwl_field:
+  my_complex_type1: foo   # we accept the default value for field3
+  my_complex_type2:
+    field2: foo2
+    field3: bar2          # we did not accept the default value for field3
+                          # so we had to use the slightly expanded syntax
+  my_complex_type3: {}    # as before, we accept the default values for both
+                          # "field2" and "field3"
+```
+
+Mixed option 2 and 3 specific example using [Workflow](Workflow.html#Workflow).[inputs](Workflow.html#WorkflowInputParameter):
+> array&lt;InputParameter&gt; |
+> map&lt;`id`, `type` | InputParameter&gt;
+
+
+```
+inputs:
+  workflow_input01: string
+  workflow_input02:     # we use the longer way
+    type: File          # because we want to specify the "format" too
+    format: http://edamontology.org/format_2572
+  workflow_input03: {}  # back to the short form as this entry
+                        # uses the default of no "type" just like the prior
+                        # examples
+```
+
+Mixed option 2 and 3 specific example using [SoftwareRequirement](#SoftwareRequirement).[packages](#SoftwarePackage):
+> array&lt;SoftwarePackage&gt; |  
+> map&lt;`package`, `specs` | SoftwarePackage&gt;
+
+
+```
+hints:
+  SoftwareRequirement:
+    packages:
+      sourmash: [ https://doi.org/10.21105/joss.00027 ]
+      screed:
+        specs: [ https://github.com/dib-lab/screed ]
+        version: [ "1.0" ]
+      python: {}
+```
+
+Note: The `map<…>` (compact) versions are optional, the verbose option #1 is
+always allowed, but for presentation reasons option 3 and 2 may be preferred
+by human readers.
+
+The normative explanation for these variations, aimed at implementors, is in the
+[Schema Salad specification](SchemaSalad.html#Identifier_maps).
+
 ## Identifiers
 
 If an object contains an `id` field, that is used to uniquely identify the
@@ -187,15 +366,19 @@ of [process requirements](#Requirements_and_hints).
 The generic execution sequence of a CWL process (including workflows and
 command line line tools) is as follows.
 
-1. Load, process and validate a CWL document, yielding a process object.
-2. Load input object.
-3. Validate the input object against the `inputs` schema for the process.
-4. Validate process requirements are met.
-5. Perform any further setup required by the specific process type.
-6. Execute the process.
-7. Capture results of process execution into the output object.
-8. Validate the output object against the `outputs` schema for the process.
-9. Report the output object to the process caller.
+1. Load input object.
+1. Load, process and validate a CWL document, yielding one or more process objects.
+1. If there are multiple process objects (due to [`$graph`](SchemaSalad.html#Document_graph))
+and which process object to start with is not specified in the input object (via
+a [`cwl:tool`](#Executing_CWL_documents_as_scripts) entry) or by any other means
+(like a URL fragment) then choose the process with the `id` of "#main" or "main".
+1. Validate the input object against the `inputs` schema for the process.
+1. Validate process requirements are met.
+1. Perform any further setup required by the specific process type.
+1. Execute the process.
+1. Capture results of process execution into the output object.
+1. Validate the output object against the `outputs` schema for the process.
+1. Report the output object to the process caller.
 
 ## Requirements and hints
 
diff --git a/cwltool/schemas/v1.1.0-dev1/invocation.md b/cwltool/schemas/v1.1.0-dev1/invocation.md
@@ -146,6 +146,9 @@ providing the fields `successCodes`, `temporaryFailCodes`, and
 `permanentFailCodes`.  An implementation may choose to default unspecified
 non-zero exit codes to either `temporaryFailure` or `permanentFailure`.
 
+The exit code of the process is available to expressions in
+`outputEval` as `runtime.exitCode`.
+
 ## Output binding
 
 If the output directory contains a file named "cwl.output.json", that file
