Skip to content

Commit 8d23671

Browse files
committed
docs: add/update
Signed-off-by: Yves Bastide <yves@botify.com>
1 parent d37f282 commit 8d23671

File tree

8 files changed

+61
-19
lines changed

8 files changed

+61
-19
lines changed

docs/mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ nav:
7676
- Error Handling: features/error_handling.md
7777
- Continue As New: features/continue_as_new.md
7878
- Middleware: features/middleware.md
79+
- Proxy: features/proxy.md
7980
- Development: development.md
8081
- Contributing: contributing.md
8182
- License: license.md

docs/src/features/canvas.md

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -139,12 +139,12 @@ from simpleflow.canvas import Chain, FuncGroup, Group
139139
@activity.with_attributes(task_list="quickstart", version="example")
140140
def partition_data(data_location):
141141
# Partition a list of things to do into parallelizable sub-parts
142-
pass
142+
...
143143

144144

145145
@activity.with_attributes(task_list="quickstart", version="example")
146146
def execute_on_sub_part(sub_part):
147-
pass
147+
...
148148

149149

150150
class AWorkflow(Workflow):
@@ -169,5 +169,11 @@ return an empty Group. Since this has been a long-standing policy, a new
169169
`_allow_none` argument relaxes this constraint.
170170

171171
!!! warning
172-
This is a new experimental option: a better one might be to enforce
173-
that nothing is returned.
172+
This is a new experimental option: a better one may be to allow
173+
a `None` return value.
174+
175+
### Overriding future classes
176+
177+
Both the `Group` and `Canvas` instances delegate their work to a
178+
`GroupFuture` (resp. `ChainFuture`) instance by default.
179+
Passing a `future_class` argument allows overriding this.

docs/src/features/command_line.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,13 +9,14 @@ List Workflow Executions
99
------------------------
1010

1111
$ simpleflow workflow.list TestDomain
12+
Workflow ID Workflow Type Status
1213
basic-example-1438722273 basic OPEN
1314

1415

1516
Workflow Execution Status
1617
-------------------------
1718

18-
$ simpleflow --header workflow.info TestDomain basic-example-1438722273
19+
$ simpleflow workflow.info TestDomain basic-example-1438722273
1920
domain workflow_type.name workflow_type.version task_list workflow_id run_id tag_list execution_time input
2021
TestDomain basic example basic-example-1438722273 22QFVi362TnCh6BdoFgkQFlocunh24zEOemo1L12Yl5Go= 1.70 {'args': [1], 'kwargs': {}}
2122

@@ -25,11 +26,11 @@ Tasks Status
2526

2627
You can check the status of the workflow execution with:
2728

28-
$ simpleflow --header workflow.tasks DOMAIN WORKFLOW_ID [RUN_ID] --nb-tasks 3
29-
$ simpleflow --header workflow.tasks TestDomain basic-example-1438722273
29+
$ simpleflow workflow.tasks DOMAIN WORKFLOW_ID [RUN_ID] --nb-tasks 3
30+
$ simpleflow workflow.tasks TestDomain basic-example-1438722273
3031
Tasks Last State Last State Time Scheduled Time
3132
examples.basic.increment scheduled 2015-08-04 23:04:34.510000 2015-08-04 23:04:34.510000
32-
$ simpleflow --header workflow.tasks TestDomain basic-example-1438722273
33+
$ simpleflow workflow.tasks TestDomain basic-example-1438722273
3334
Tasks Last State Last State Time Scheduled Time
3435
examples.basic.double completed 2015-08-04 23:06:19.200000 2015-08-04 23:06:17.738000
3536
examples.basic.delay completed 2015-08-04 23:08:18.402000 2015-08-04 23:06:17.738000
@@ -41,7 +42,7 @@ Profiling
4142

4243
You can profile the execution of the workflow with:
4344

44-
$ simpleflow --header workflow.profile TestDomain basic-example-1438722273
45+
$ simpleflow workflow.profile TestDomain basic-example-1438722273
4546
Task Last State Scheduled Time Scheduled Start Time Running End Percentage of total time
4647
activity-examples.basic.double-1 completed 2015-08-04 23:06 0.07 2015-08-04 23:06 1.39 2015-08-04 23:06 1.15
4748
activity-examples.basic.increment-1 completed 2015-08-04 23:04 102.20 2015-08-04 23:06 0.79 2015-08-04 23:06 0.65

docs/src/features/continue_as_new.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
# Continue As New
22

3-
In long-running workflow executions, the history can hit the 25,000-events hard SWF limit. This causes execution
4-
termination. To prevent this, the workflow can itself close the current execution and start another one by submitting
3+
In long-running workflow executions, the history can hit the 25,000-events hard SWF limit.
4+
This causes execution termination.
5+
To prevent this, the workflow can itself close the current execution and start another one by submitting
56
`self.continue_as_new(*args, **kwargs)`: it is then restarted with a new run ID and an empty history.
67

78
See `examples/continue_as_new.py` for a demonstration of this pattern:

docs/src/features/program_tasks.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
Execution of Tasks as Programs
22
==============================
33

4-
The `simpleflow.execute` module allows to define functions that will be
4+
The `simpleflow.execute` module allows defining functions that will be
55
executed as a program.
66

77
There are two modes:

docs/src/features/proxy.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Worker Proxy Support
2+
3+
When deploying a fleet of workers on an instance, their number
4+
may cause the maximum number of connections to SWF to be reached
5+
and let some of these workers unable to fetch activities.
6+
7+
A proxy can be used to prevent this. Simpleflow can now use one,
8+
and provides a simplistic single-threaded proxy.
9+
10+
## Starting the proxy
11+
12+
The `simpleflow proxy.start` command starts an HTTP proxy server.
13+
It processes the CONNECT proxy method; the rest of the connection
14+
is encrypted as before (it doesn't handle HTTPS to MITM the
15+
communication).
16+
17+
## Using the proxy
18+
19+
The `simpleflow worker.start` command accepts a new `-x, --proxy` argument.
20+
21+
## Example
22+
23+
```shell
24+
# Running the `pirate` example with multiple processes.
25+
# Starting the decider, then the workflow, are left as exercises.
26+
[screen 1] $ simpleflow proxy.start
27+
Serving HTTP Proxy on ::1:4242
28+
[screen 2] $ SWF_PROXY=localhost:4242 PYTHONPATH=$PWD SWF_DOMAIN=TestDomain \
29+
simpleflow worker.start -N 1 -t pirate
30+
```
31+
32+
## Environment setting
33+
34+
Both commands honor a new `SWF_PROXY` environment variable.

docs/src/features/swf_layer.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ Settings
2222
--------
2323

2424
!!! bug
25-
The informations in this "Settings" section may be outdated, they need some love.
25+
The information in this "Settings" section may be outdated, it needs some love.
2626

2727
Optional:
2828

@@ -84,7 +84,7 @@ local and remote objects.
8484
retention_period=60
8585
)
8686

87-
# a Domain model local instance has been created, but nothing has been
87+
# A Domain model local instance has been created, but nothing has been
8888
# sent to amazon. To do so, you have to save it.
8989
>>> D.save()
9090
```
@@ -133,8 +133,8 @@ ModelDiff()
133133

134134
### QuerySets
135135

136-
Models can be retrieved and instantiated via querysets. To continue over the django comparison,
137-
they’re behaving like django managers.
136+
Models can be retrieved and instantiated via querysets. To continue over the Django comparison,
137+
they’re behaving like Django managers.
138138

139139
```python
140140
# As querying for models needs a valid connection to amazon service,

docs/src/features/task_lists.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
# Task Lists
22

3-
Task lists are often used to route different tasks to specific groups
4-
of workers.
3+
Task lists are used to route different tasks to specific groups of workers.
54
The decider and activity task lists are distinct, even if they have the same name.
65

76
For SWF activities, the task list is typically specified with `@activity.with_attributes`:
@@ -56,7 +55,7 @@ class MyWorkflow(Workflow):
5655
...
5756

5857
@classmethod
59-
def get_task_list(cls, task_list, *args, **kwargs):
58+
def get_task_list(cls, *args, task_list, **kwargs):
6059
return task_list
6160

6261
def run(self, x, task_list, *args, **kwargs):

0 commit comments

Comments
 (0)