docs: add/update

Signed-off-by: Yves Bastide <yves@botify.com>

Author: ybastide

docs/mkdocs.yml

@@ -76,6 +76,7 @@ nav:
       - Error Handling: features/error_handling.md
       - Continue As New: features/continue_as_new.md
       - Middleware: features/middleware.md
+      - Proxy: features/proxy.md
   - Development: development.md
   - Contributing: contributing.md
   - License: license.md

docs/src/features/canvas.md

@@ -139,12 +139,12 @@ from simpleflow.canvas import Chain, FuncGroup, Group
 @activity.with_attributes(task_list="quickstart", version="example")
 def partition_data(data_location):
     # Partition a list of things to do into parallelizable sub-parts
-    pass
+    ...
 
 
 @activity.with_attributes(task_list="quickstart", version="example")
 def execute_on_sub_part(sub_part):
-    pass
+    ...
 
 
 class AWorkflow(Workflow):
@@ -169,5 +169,11 @@ return an empty Group. Since this has been a long-standing policy, a new
 `_allow_none` argument relaxes this constraint.
 
 !!! warning
-    This is a new experimental option: a better one might be to enforce
-    that nothing is returned.
+    This is a new experimental option: a better one may be to allow
+    a `None` return value.
+
+### Overriding future classes
+
+Both the `Group` and `Canvas` instances delegate their work to a
+`GroupFuture` (resp. `ChainFuture`) instance by default.
+Passing a `future_class` argument allows overriding this.

docs/src/features/command_line.md

@@ -9,13 +9,14 @@ List Workflow Executions
 ------------------------
 
     $ simpleflow workflow.list TestDomain
+    Workflow ID    Workflow Type    Status
     basic-example-1438722273  basic  OPEN
 
 
 Workflow Execution Status
 -------------------------
 
-    $ simpleflow --header workflow.info TestDomain basic-example-1438722273
+    $ simpleflow workflow.info TestDomain basic-example-1438722273
     domain      workflow_type.name    workflow_type.version      task_list  workflow_id               run_id                                          tag_list      execution_time  input
     TestDomain  basic                 example                               basic-example-1438722273  22QFVi362TnCh6BdoFgkQFlocunh24zEOemo1L12Yl5Go=                          1.70  {'args': [1], 'kwargs': {}}
 
@@ -25,11 +26,11 @@ Tasks Status
 
 You can check the status of the workflow execution with:
 
-    $ simpleflow --header workflow.tasks DOMAIN WORKFLOW_ID [RUN_ID] --nb-tasks 3
-    $ simpleflow --header workflow.tasks TestDomain basic-example-1438722273
+    $ simpleflow workflow.tasks DOMAIN WORKFLOW_ID [RUN_ID] --nb-tasks 3
+    $ simpleflow workflow.tasks TestDomain basic-example-1438722273
     Tasks                     Last State    Last State Time             Scheduled Time
     examples.basic.increment  scheduled     2015-08-04 23:04:34.510000  2015-08-04 23:04:34.510000
-    $ simpleflow --header workflow.tasks TestDomain basic-example-1438722273
+    $ simpleflow workflow.tasks TestDomain basic-example-1438722273
     Tasks                     Last State    Last State Time             Scheduled Time
     examples.basic.double     completed     2015-08-04 23:06:19.200000  2015-08-04 23:06:17.738000
     examples.basic.delay      completed     2015-08-04 23:08:18.402000  2015-08-04 23:06:17.738000
@@ -41,7 +42,7 @@ Profiling
 
 You can profile the execution of the workflow with:
 
-    $ simpleflow --header workflow.profile TestDomain basic-example-1438722273
+    $ simpleflow workflow.profile TestDomain basic-example-1438722273
     Task                                 Last State    Scheduled           Time Scheduled  Start               Time Running  End                 Percentage of total time
     activity-examples.basic.double-1     completed     2015-08-04 23:06              0.07  2015-08-04 23:06            1.39  2015-08-04 23:06                        1.15
     activity-examples.basic.increment-1  completed     2015-08-04 23:04            102.20  2015-08-04 23:06            0.79  2015-08-04 23:06                        0.65

docs/src/features/continue_as_new.md

@@ -1,7 +1,8 @@
 # Continue As New
 
-In long-running workflow executions, the history can hit the 25,000-events hard SWF limit. This causes execution
-termination. To prevent this, the workflow can itself close the current execution and start another one by submitting
+In long-running workflow executions, the history can hit the 25,000-events hard SWF limit.
+This causes execution termination.
+To prevent this, the workflow can itself close the current execution and start another one by submitting
 `self.continue_as_new(*args, **kwargs)`: it is then restarted with a new run ID and an empty history.
 
 See `examples/continue_as_new.py` for a demonstration of this pattern:

docs/src/features/program_tasks.md

@@ -1,7 +1,7 @@
 Execution of Tasks as Programs
 ==============================
 
-The `simpleflow.execute` module allows to define functions that will be
+The `simpleflow.execute` module allows defining functions that will be
 executed as a program.
 
 There are two modes:

docs/src/features/proxy.md

@@ -0,0 +1,34 @@
+# Worker Proxy Support
+
+When deploying a fleet of workers on an instance, their number
+may cause the maximum number of connections to SWF to be reached
+and let some of these workers unable to fetch activities.
+
+A proxy can be used to prevent this. Simpleflow can now use one,
+and provides a simplistic single-threaded proxy.
+
+## Starting the proxy
+
+The `simpleflow proxy.start` command starts an HTTP proxy server.
+It processes the CONNECT proxy method; the rest of the connection
+is encrypted as before (it doesn't handle HTTPS to MITM the
+communication).
+
+## Using the proxy
+
+The `simpleflow worker.start` command accepts a new `-x, --proxy` argument.
+
+## Example
+
+```shell
+# Running the `pirate` example with multiple processes.
+# Starting the decider, then the workflow, are left as exercises.
+[screen 1] $ simpleflow proxy.start
+Serving HTTP Proxy on ::1:4242
+[screen 2] $ SWF_PROXY=localhost:4242 PYTHONPATH=$PWD SWF_DOMAIN=TestDomain \
+ simpleflow worker.start -N 1 -t pirate
+```
+
+## Environment setting
+
+Both commands honor a new `SWF_PROXY` environment variable.

docs/src/features/swf_layer.md

@@ -22,7 +22,7 @@ Settings
 --------
 
 !!! bug
-    The informations in this "Settings" section may be outdated, they need some love.
+    The information in this "Settings" section may be outdated, it needs some love.
 
 Optional:
 
@@ -84,7 +84,7 @@ local and remote objects.
     retention_period=60
 )
 
-# a Domain model local instance has been created, but nothing has been
+# A Domain model local instance has been created, but nothing has been
 # sent to amazon. To do so, you have to save it.
 >>> D.save()
 ```
@@ -133,8 +133,8 @@ ModelDiff()
 
 ### QuerySets
 
-Models can be retrieved and instantiated via querysets. To continue over the django comparison,
-they’re behaving like django managers.
+Models can be retrieved and instantiated via querysets. To continue over the Django comparison,
+they’re behaving like Django managers.
 
 ```python
 # As querying for models needs a valid connection to amazon service,

docs/src/features/task_lists.md

@@ -1,7 +1,6 @@
 # Task Lists
 
-Task lists are often used to route different tasks to specific groups
-of workers.
+Task lists are used to route different tasks to specific groups of workers.
 The decider and activity task lists are distinct, even if they have the same name.
 
 For SWF activities, the task list is typically specified with `@activity.with_attributes`:
@@ -56,7 +55,7 @@ class MyWorkflow(Workflow):
     ...
 
     @classmethod
-    def get_task_list(cls, task_list, *args, **kwargs):
+    def get_task_list(cls, *args, task_list, **kwargs):
         return task_list
 
     def run(self, x, task_list, *args, **kwargs):