Skip to content

Commit f23b963

Browse files
nrcnrc
authored
Merge pull request #235 from nrc/guide-plan
A plan for the guide
2 parents 177aec4 + 58eeade commit f23b963

File tree

11 files changed

+417
-1
lines changed

11 files changed

+417
-1
lines changed

‎src/SUMMARY.md‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,35 @@
1111

1212
- [Introduction](part-guide/intro.md)
1313
- [Concurrent programming](part-guide/concurrency.md)
14-
- [Async and Await](part-guide/async-await.md)
14+
- [Async and await](part-guide/async-await.md)
15+
- [Advanced async/await topics](part-guide/adv-async-await.md)
16+
- [IO and issues with blocking](part-guide/io.md)
17+
- [Concurrency primitives](part-guide/concurrency-primitives.md)
18+
- [Channels, locking, and synchronization](part-guide/sync.md)
19+
- [Tools for async programming](part-guide/tools.md)
20+
- [Destruction and clean-up](part-guide/dtors.md)
21+
- [Futures](part-guide/futures.md)
22+
- [Runtimes](part-guide/runtimes.md)
23+
- [Timers and signal handling](part-guide/times-signals.md)
24+
- [Async iterators (streams)](part-guide/streams.md)
1525

1626
# Part 2: reference
1727

28+
- [Implementing futures and streams]()
29+
- [Alternate runtimes]()
30+
- [Implementing your own runtime]()
31+
- [async in sync, sync in async]()
32+
- [Async IO: readiness vs completion, and io_uring]()
33+
- [Design patterns]()
34+
- [Cancellation]() (cancellation safety)
35+
- [Starvation]()
36+
- [Pinning]()
37+
- [Async and FFI]()
38+
- [Comparing async programming in Rust to other languages]()
39+
- [The implementation of async/await in rustc]()
40+
- structured concurrency?
41+
42+
1843
# Old chapters
1944

2045
- [Getting Started](01_getting_started/01_chapter.md)

‎src/part-guide/adv-async-await.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# More async/await topics
2+
3+
## Unit tests
4+
5+
## Blocking and cancellation
6+
7+
- Two important concepts to be aware of early, we'll revisit in more detail as we go along
8+
- Cancellation
9+
- How to do it
10+
- drop a future
11+
- cancellation token
12+
- abort functions
13+
- Why it matters, cancellation safety (forward ref)
14+
- Blocking
15+
- IO and computation can block
16+
- why it's bad
17+
- how to deal is a forward ref to io chapter
18+
19+
## `Send + 'static` bounds on futures
20+
21+
- Why they're there, multi-threaded runtimes
22+
- spawn local to avoid them
23+
- What makes an async fn `Send + 'static` and how to fix bugs with it
24+
25+
## Async traits
26+
27+
- syntax
28+
- The `Send + 'static` issue and working around it
29+
- trait_variant
30+
- explicit future
31+
- return type notation (https://blog.rust-lang.org/inside-rust/2024/09/26/rtn-call-for-testing.html)
32+
- overriding
33+
- future vs async notation for methods
34+
- object safety
35+
- capture rules (https://blog.rust-lang.org/2024/09/05/impl-trait-capture-rules.html)
36+
- history and async-trait crate
37+
38+
39+
## Async blocks and closures
40+
41+
- async block syntax
42+
- what it means
43+
- using an async block in a function returning a future
44+
- subtype of async method
45+
- closures
46+
- coming soon (https://github.com/rust-lang/rust/pull/132706, https://blog.rust-lang.org/inside-rust/2024/08/09/async-closures-call-for-testing.html)
47+
- async blocks in closures vs async closures
48+
- errors in async blocks
49+
- https://rust-lang.github.io/async-book/07_workarounds/02_err_in_async_blocks.html
50+
51+
## Recursion
52+
53+
- Allowed (relatively new), but requires some explicit boxing
54+
- forward reference to futures, pinning
55+
- https://rust-lang.github.io/async-book/07_workarounds/04_recursion.html
56+
- https://blog.rust-lang.org/2024/03/21/Rust-1.77.0.html#support-for-recursion-in-async-fn
57+
- async-recursion macro (https://docs.rs/async-recursion/latest/async_recursion/)
58+
59+
60+
## Lifetimes and borrowing
61+
62+
- Mentioned the static lifetime above
63+
- Lifetime bounds on futures (`Future + '_`, etc.)
64+
- Borrowing across await points
65+
- I don't know, I'm sure there are more lifetime issues with async functions ...

‎src/part-guide/concurrency-primitives.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Concurrency primitives
2+
3+
- concurrent composition of futures
4+
- c.f., sequential composition with await, composition of tasks with spawn
5+
- concurrent/task behaviour
6+
- behaviour on error
7+
- streams as alternative, forward ref
8+
- different versions in different runtimes/other crates
9+
- focus on the Tokio versions
10+
11+
## Join
12+
13+
- Tokio/futures-rs join macro
14+
- c.f., joining tasks
15+
- join in futures-concurrency
16+
- FuturesUnordered
17+
- like a dynamic version of join
18+
- forward ref to stream
19+
20+
## Race/select
21+
22+
- Tokio select macro
23+
- cancellation issues
24+
- different behaviour of futures-rs version
25+
- race in futures-concurrency

‎src/part-guide/dtors.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Destruction and clean-up
2+
3+
- Object destruction and recap of Drop
4+
- General clean up requirements in software
5+
- Async issues
6+
- Might want to do stuff async during clean up, e.g., send a final message
7+
- Might need to clean up stuff which is still being used async-ly
8+
- Might want to clean up when an async task completes or cancels and there is no way to catch that
9+
- State of the runtime during clean-up phase (esp if we're panicking or whatever)
10+
- No async Drop
11+
- WIP
12+
- forward ref to completion io topic
13+
14+
## Cancellation
15+
16+
- How it happens (recap of adv-async-await.md)
17+
- drop a future
18+
- cancellation token
19+
- abort functions
20+
- What we can do about 'catching' cancellation
21+
- logging or monitoring cancellation
22+
- How cancellation affects other futures tasks (forward ref to cancellation safety chapter, this should just be a heads-up)
23+
24+
## Panicking and async
25+
26+
- Propagation of panics across tasks (spawn result)
27+
- Panics leaving data inconsistent (tokio mutexes)
28+
- Calling async code when panicking (make sure you don't)
29+
30+
## Patterns for clean-up
31+
32+
- Avoid needing clean up (abort/restart)
33+
- Don't use async for cleanup and don't worry too much
34+
- async clean up method + dtor bomb (i.e., separate clean-up from destruction)
35+
- centralise/out-source clean-up in a separate task or thread or supervisor object/process
36+
37+
## Why no async Drop (yet)
38+
39+
- Note this is advanced section and not necessary to read
40+
- Why async Drop is hard
41+
- Possible solutions and there issues
42+
- Current status

‎src/part-guide/futures.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Futures
2+
3+
We've talked a lot about futures in the preceding chapters; they're a key part of Rust's async programming story! In this chapter we're going to get into some of the details of what futures are and how they work, and some libraries for working directly with futures.
4+
5+
## The `Future` and `IntoFuture` traits
6+
7+
- Future
8+
- Output assoc type
9+
- No real detail here, polling is in the next section, reference adv sections on Pin, executors/wakers
10+
- IntoFuture
11+
- Usage - general, in await, async builder pattern (pros and cons in using)
12+
- Boxing futures, `Box<dyn Future>` and how it used to be common and necessary but mostly isn't now, except for recursion, etc.
13+
14+
## Polling
15+
16+
- what it is and who does it, Poll type
17+
- ready is final state
18+
- how it connects with await
19+
- drop = cancel
20+
- for futures and thus tasks
21+
- implications for async programming in general
22+
- reference to chapter on cancellation safety
23+
24+
### Fusing
25+
26+
## futures-rs crate
27+
28+
- History and purpose
29+
- see streams chapter
30+
- helpers for writing executors or other low-level futures stuff
31+
- pinning and boxing
32+
- executor as a partial runtime (see alternate runtimes in reference)
33+
- TryFuture
34+
- convenience futures: pending, ready, ok/err, etc.
35+
- combinator functions on FutureExt
36+
- alternative to Tokio stuff
37+
- functions
38+
- IO traits
39+
40+
## futures-concurrency crate
41+
42+
https://docs.rs/futures-concurrency/latest/futures_concurrency/
43+
44+

‎src/part-guide/io.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# IO and issues with blocking
2+
3+
## Blocking and non-blocking IO
4+
5+
- High level view
6+
- How async IO fits with async concurrency
7+
- Why blocking IO is bad
8+
- forward ref to streams for streams/sinks
9+
10+
## Read and Write
11+
12+
- async Read and Write traits
13+
- part of the runtime
14+
- how to use
15+
- specific implementations
16+
- network vs disk
17+
- tcp, udp
18+
- file system is not really async, but io_uring (ref to that chapter)
19+
- practical examples
20+
- stdout, etc.
21+
- pipe, fd, etc.
22+
23+
24+
## Memory management
25+
26+
- Issues with buffer management and async IO
27+
- Different solutions and pros and cons
28+
- zero-copy approach
29+
- shared buffer approach
30+
- Utility crates to help with this, Bytes, etc.
31+
32+
## Advanced topics on IO
33+
34+
- buf read/write
35+
- Read + Write, split, join
36+
- copy
37+
- simplex and duplex
38+
- cancelation
39+
40+
## The OS view of IO
41+
42+
- Different kinds of IO and mechanisms, completion IO, reference to completion IO chapter in adv section
43+
- different runtimes can faciliate this
44+
- mio for low-level interface
45+
46+
47+
## Other blocking operations
48+
49+
- Why this is bad
50+
- Long running CPU work
51+
- Using Tokio for just CPU work: https://thenewstack.io/using-rustlangs-async-tokio-runtime-for-cpu-bound-tasks/
52+
- Solutions
53+
- spawn blocking
54+
- thread pool
55+
- etc.
56+
- yielding to the runtime
57+
- not the same as Rust's yield keyword
58+
- await doesn't yield
59+
- implicit yields in Tokio

‎src/part-guide/runtimes.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Runtimes and runtime issues
2+
3+
## Running async code
4+
5+
- Explicit startup vs async main
6+
- tokio context concept
7+
- block_on
8+
- runtime as reflected in the code (Runtime, Handle)
9+
- runtime shutdown
10+
11+
## Threads and tasks
12+
13+
- default work stealing, multi-threaded
14+
- revisit Send + 'static bounds
15+
- yield
16+
- spawn-local
17+
- spawn-blocking (recap), block-in-place
18+
- tokio-specific stuff on yielding to other threads, local vs global queues, etc
19+
20+
## Configuration options
21+
22+
- thread pool size
23+
- single threaded, thread per core etc.
24+
25+
## Alternate runtimes
26+
27+
- Why you'd want to use a different runtime or implement your own
28+
- What kind of variations exist in the high-level design
29+
- Forward ref to adv chapters

‎src/part-guide/streams.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Async iterators (FKA streams)
2+
3+
- Stream as an async iterator or as many futures
4+
- WIP
5+
- current status
6+
- futures and Tokio Stream traits
7+
- nightly trait
8+
- lazy like sync iterators
9+
- pinning and streams (forward ref to pinning chapter)
10+
- fused streams
11+
12+
## Consuming an async iterator
13+
14+
- while let with async next
15+
- for_each, for_each_concurrent
16+
- collect
17+
- into_future, buffered
18+
19+
## Stream combinators
20+
21+
- Taking a future instead of a closure
22+
- Some example combinators
23+
- unordered variations
24+
25+
## Implementing an async iterator
26+
27+
- Implementing the trait
28+
- Practicalities and util functions
29+
- async_iter stream macro
30+
31+
## Sinks
32+
33+
- https://docs.rs/futures/latest/futures/sink/index.html
34+
35+
## Future work
36+
37+
- current status
38+
- https://rust-lang.github.io/rfcs/2996-async-iterator.html
39+
- async next vs poll
40+
- async iteration syntax
41+
- (async) generators
42+
- lending iterators
43+

‎src/part-guide/sync.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Channels, locking, and synchronization
2+
3+
note on runtime specificness of sync primitves
4+
5+
Why we need async primitives rather than use the sync ones
6+
7+
## Channels
8+
9+
- basically same as the std ones, but await
10+
- communicate between tasks (same thread or different)
11+
- one shot
12+
- mpsc
13+
- other channels
14+
- bounded and unbounded channels
15+
16+
## Locks
17+
18+
- async Mutex
19+
- c.f., std::Mutex - can be held across await points (borrowing the mutex in the guard, guard is Send, scheduler-aware? or just because lock is async?), lock is async (will not block the thread waiting for lock to be available)
20+
- even a clippy lint for holding the guard across await (https://rust-lang.github.io/rust-clippy/master/index.html#await_holding_lock)
21+
- more expensive because it can be held across await
22+
- use std::Mutex if you can
23+
- can use try_lock or mutex is expected to not be under contention
24+
- lock is not magically dropped when yield (that's kind of the point of a lock!)
25+
- deadlock by holding mutex over await
26+
- tasks deadlocked, but other tasks can make progress so might not look like a deadlock in process stats/tools/OS
27+
- usual advice - limit scope, minimise locks, order locks, prefer alternatives
28+
- no mutex poisoning
29+
- lock_owned
30+
- blocking_lock
31+
- cannot use in async
32+
- applies to other locks (should the above be moved before discussion of mutex specifically? Probably yes)
33+
- RWLock
34+
- Semaphore
35+
- yielding
36+
37+
## Other synchronization primitives
38+
39+
- notify, barrier
40+
- OnceCell
41+
- atomics

0 commit comments

Comments
 (0)