11# Closure Expansion in rustc
22
3- Let's start with a few examples
3+ This section describes how rustc handles closures. Closures in Rust are
4+ effectively "desugared" into structs that contain the values they use (or
5+ references to the values they use) from their creator's stack frame. rustc has
6+ the job of figuring out which values a closure uses and how, so it can decide
7+ whether to capture a given variable by shared reference, mutable reference, or
8+ by move. rustc also has to figure out which the closure traits ([ ` Fn ` ] [ fn ] ,
9+ [ ` FnMut ` ] [ fn_mut ] , or [ ` FnOnce ` ] [ fn_once ] ) a closure is capable of
10+ implementing.
11+
12+ [ fn ] : https://doc.rust-lang.org/std/ops/trait.Fn.html
13+ [ fn_mut ] :https://doc.rust-lang.org/std/ops/trait.FnMut.html
14+ [ fn_once ] : https://doc.rust-lang.org/std/ops/trait.FnOnce.html
15+
16+ Let's start with a few examples:
417
518### Example 1
19+
20+ To start, let's take a look at how the closure in the following example is desugared:
21+
622``` rust
723fn closure (f : impl Fn ()) {
824 f ();
@@ -14,22 +30,37 @@ fn main() {
1430 println! (" Value of x after return {}" , x );
1531}
1632```
17- Let's say the above is the content of a file called immut.rs. If we compile immut.rs using the command
18- ```
19- rustc +stage1 immut.rs -Zdump-mir=all
33+
34+ Let's say the above is the content of a file called ` immut.rs ` . If we compile
35+ ` immut.rs ` using the following command. The [ ` -Zdump-mir=all ` ] [ dump-mir ] flag will cause
36+ ` rustc ` to generate and dump the [ MIR] [ mir ] to a directory called ` mir_dump ` .
37+ ``` console
38+ > rustc +stage1 immut.rs -Zdump-mir=all
2039```
21- we will see a newly generated directory in our current working directory called mir_dump, which will
22- contain several files. If we look at file ` rustc.main.-------.mir_map.0.mir ` , we will find, among
40+
41+ [ mir ] : ./mir/index.md
42+ [ dump-mir ] : ./mir/passes.md
43+
44+ After we run this command, we will see a newly generated directory in our
45+ current working directory called ` mir_dump ` , which will contain several files.
46+ If we look at file ` rustc.main.-------.mir_map.0.mir ` , we will find, among
2347other things, it also contains this line:
2448
2549``` rust,ignore
26- _4 = &_1; // bb0[6]: scope 1 at immut.rs:7:13: 7:36
27- _3 = [closure@immut.rs:7:13: 7:36] { x: move _4 }; // bb0[7]: scope 1 at immut.rs:7:13: 7:36
50+ _4 = &_1;
51+ _3 = [closure@immut.rs:7:13: 7:36] { x: move _4 };
2852```
29- Here in first line ` _4 = &_1; ` , the mir_dump tells us that x was borrowed as an immutable reference.
30- This is what we would hope as our closure just reads x.
53+
54+ Note that in the MIR examples in this chapter, ` _1 ` is ` x ` .
55+
56+ Here in first line ` _4 = &_1; ` , the ` mir_dump ` tells us that ` x ` was borrowed
57+ as an immutable reference. This is what we would hope as our closure just
58+ reads ` x ` .
3159
3260### Example 2
61+
62+ Here is another example:
63+
3364``` rust
3465fn closure (mut f : impl FnMut ()) {
3566 f ();
@@ -46,13 +77,16 @@ fn main() {
4677```
4778
4879``` rust,ignore
49- _4 = &mut _1; // bb0[6]: scope 1 at mut.rs:7:13: 10:6
50- _3 = [closure@mut.rs:7:13: 10:6] { x: move _4 }; // bb0[7]: scope 1 at mut.rs:7:13: 10:6
80+ _4 = &mut _1;
81+ _3 = [closure@mut.rs:7:13: 10:6] { x: move _4 };
5182```
5283This time along, in the line ` _4 = &mut _1; ` , we see that the borrow is changed to mutable borrow.
53- fair enough as the closure increments x by 10.
84+ Fair enough! The closure increments ` x ` by 10.
5485
5586### Example 3
87+
88+ One more example:
89+
5690``` rust
5791fn closure (f : impl FnOnce ()) {
5892 f ();
@@ -70,57 +104,66 @@ fn main() {
70104``` rust,ignore
71105_6 = [closure@move.rs:7:13: 9:6] { x: move _1 }; // bb16[3]: scope 1 at move.rs:7:13: 9:6
72106```
73- Here, x is directly moved into the closure and the access to it will not be permitted after the
107+ Here, ` x ` is directly moved into the closure and the access to it will not be permitted after the
74108closure.
75109
110+ ## Inferences in the compiler
76111
77112Now let's dive into rustc code and see how all these inferences are done by the compiler.
78113
79114Let's start with defining a term that we will be using quite a bit in the rest of the discussion -
80- * upvar* . An ** upvar** is a variable that is local to the function, where the closure is defined. So,
115+ * upvar* . An ** upvar** is a variable that is local to the function where the closure is defined. So,
81116in the above examples, ** x** will be an upvar to the closure. They are also sometimes referred to as
82117the * free variables* meaning they are not bound to the context of the closure.
83- ` src/librustc/ty/query/mod.rs ` defines a query called * freevars* for this purpose.
118+ [ ` src/librustc/ty/query/mod.rs ` ] [ freevars ] defines a query called * freevars* for this purpose.
84119
85- So, we know that other than lazy invocation, one other thing that the distinguishes a closure from a
86- normal function is that it can use the upvars. Because, it borrows these upvars from its surrounding
87- context, therfore the compiler has to determine the upvar's borrow type. The compiler starts with
120+ [ freevars ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/query/queries/struct.freevars.html
121+
122+ Other than lazy invocation, one other thing that the distinguishes a closure from a
123+ normal function is that it can use the upvars. It borrows these upvars from its surrounding
124+ context; therfore the compiler has to determine the upvar's borrow type. The compiler starts with
88125assigning an immutable borrow type and lowers the restriction (that is, changes it from
89126** immutable** to ** mutable** to ** move** ) as needed, based on the usage. In the Example 1 above, the
90127closure only uses the variable for printing but does not modify it in any way and therefore, in the
91- mir_dump, we find the borrow type for the upvar x to be immutable. In example 2, however the
92- closure modifies x and increments it by some value. Because of this mutation, the compiler, which
93- started off assigning x as an immutable reference type, has to adjust it as mutable reference.
128+ ` mir_dump ` , we find the borrow type for the upvar ` x ` to be immutable. In example 2, however, the
129+ closure modifies ` x ` and increments it by some value. Because of this mutation, the compiler, which
130+ started off assigning ` x ` as an immutable reference type, has to adjust it as a mutable reference.
94131Likewise in the third example, the closure drops the vector and therefore this requires the variable
95- x to be moved into the closure. Depending on the borrow kind, the closure has to implement the
96- appropriate trait. Fn trait for immutable borrow, FnMut for mutable borrow and FnOnce for move
97- semantics.
132+ ` x ` to be moved into the closure. Depending on the borrow kind, the closure has to implement the
133+ appropriate trait: ` Fn ` trait for immutable borrow, ` FnMut ` for mutable borrow,
134+ and ` FnOnce ` for move semantics.
135+
136+ Most of the code related to the closure is in the
137+ [ ` src/librustc_typeck/check/upvar.rs ` ] [ upvar ] file and the data structures are
138+ declared in the file [ ` src/librustc/ty/mod.rs ` ] [ ty ] .
98139
99- Most of the code related to the closure is in the src/librustc_typeck/ check/upvar.rs file and the
100- data structures are declared in the file src/librustc/ ty/mod.rs.
140+ [ upvar ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/ check/upvar/index.html
141+ [ ty ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ ty/index.html
101142
102143Before we go any further, let's discuss how we can examine the flow of coontrol through the rustc
103- codebase. For the closure part specifically, I would set the RUST_LOG as under and collect the
104- output in a file
144+ codebase. For closures specifically, set the ` RUST_LOG ` env variable as below and collect the
145+ output in a file:
105146
106- ```
107- RUST_LOG=rustc_typeck::check::upvar rustc +stage1 -Zdump-mir=all <.rs file to compile> 2> <file
108- where the output will be dumped>
147+ ``` console
148+ > RUST_LOG=rustc_typeck::check::upvar rustc +stage1 -Zdump-mir=all \
149+ <.rs file to compile> 2> <file where the output will be dumped>
109150```
110151
111- This uses the stage1 compiler.
152+ This uses the stage1 compiler and enables ` debug! ` logging for the
153+ ` rustc_typeck::check::upvar ` module.
112154
113155The other option is to step through the code using lldb or gdb.
114156
115- ```
116- 1. rust-lldb build/x86_64-apple-darwin/stage1/bin/rustc test.rs
117- 2. b upvar.rs:134 // Setting the breakpoint on a certain line in the upvar.rs file
118- 3. r // Run the program until it hits the breakpoint
119- ```
157+ 1 . ` rust-lldb build/x86_64-apple-darwin/stage1/bin/rustc test.rs `
158+ 2 . In lldb:
159+ 1 . ` b upvar.rs:134 ` // Setting the breakpoint on a certain line in the upvar.rs file`
160+ 2 . ` r ` // Run the program until it hits the breakpoint
161+
162+ Let's start with [ ` upvar.rs ` ] [ upvar ] . This file has something called
163+ the [ ` euv::ExprUseVisitor ` ] [ euv ] which walks the source of the closure and
164+ invokes a callbackfor each upvar that is borrowed, mutated, or moved.
120165
121- Let's start with the file: ` upvar.rs ` . This file has something called the euv::ExprUseVisitor which
122- walks the source of the closure and it gets called back for each upvar that is borrowed, mutated or
123- moved.
166+ [ euv ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/expr_use_visitor/struct.ExprUseVisitor.html
124167
125168``` rust
126169fn main () {
@@ -132,28 +175,39 @@ fn main() {
132175}
133176```
134177
135- In the above example, our visitor will be called twice, for the lines marked 1 and 2, once as a
136- shared borrow and another one as a mutable borrow. It will also tell as what was borrowed. The
137- callbacks get invoked at the delegate. The delegate is of type ` struct InferBorrowKind ` which has a
138- few fields but the one we are interested in is the ` adjust_upvar_captures ` which is of type
139- ` FxHashMap<UpvarId, UpvarCapture<'tcx>> ` which tells us for each upvar, which mode of borrow did we
140- require. The modes of borrow can be ByValue (moved) or ByRef (borrowed) and for ByRef borrows, it
141- can be one among shared, shallow, unique or mut as defined in the ` src/librustc/mir/mod.rs `
142-
143- The method callbacks are the method implementations of the euv::Delegate trait for InferBorrowKind.
144- ** consume** callback is for * move* of a variable, ** borrow** callback if there is a * borrow* of some
145- kind, shared or mutable and ** mutate** when we see an * assignment* of something. We will see that
146- all these callbacks have a common argument * cmt* which stands for category, Mutability and Type and
147- is defined in * src/librustc/middle/mem_categorization.rs* . Borrowing from the code comments * cmt * is
148- a complete categorization of a value indicating where it originated and how it is located, as well
149- as the mutability of the memory in which the value is stored.** Based on the callback (consume,
150- borrow etc.), we will call the relevant * adjust_upvar_borrow_kind_for_ <something >* and pass the cmt
151- along. Once the borrow type is adjusted, we store it in the table, which basically says for this
152- closure, these set of borrows were made.
178+ In the above example, our visitor will be called twice, for the lines marked 1 and 2, once for a
179+ shared borrow and another one for a mutable borrow. It will also tell us what was borrowed.
153180
154- ```
181+ The callbacks are defined by implementing the [ ` Delegate ` ] [ delegate ] trait. The
182+ [ ` InferBorrowKind ` ] [ ibk ] type implements ` Delegate ` and keeps a map that
183+ records for each upvar which mode of borrow was required. The modes of borrow
184+ can be ` ByValue ` (moved) or ` ByRef ` (borrowed). For ` ByRef ` borrows, it can be
185+ ` shared ` , ` shallow ` , ` unique ` or ` mut ` as defined in the
186+ [ ` src/librustc/mir/mod.rs ` ] [ mir_mod ] .
187+
188+ [ mir_mod ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/mir/index.html
189+
190+ ` Delegate ` defines a few different methods (the different callbacks):
191+ ** consume** : for * move* of a variable, ** borrow** for a * borrow* of some kind
192+ (shared or mutable), and ** mutate** when we see an * assignment* of something.
193+
194+ All of these callbacks have a common argument * cmt* which stands for Category,
195+ Mutability and Type and is defined in
196+ [ ` src/librustc/middle/mem_categorization.rs ` ] [ cmt ] . Borrowing from the code
197+ comments, "` cmt ` is a complete categorization of a value indicating where it
198+ originated and how it is located, as well as the mutability of the memory in
199+ which the value is stored". Based on the callback (consume, borrow etc.), we
200+ will call the relevant * adjust_upvar_borrow_kind_for_ <something >* and pass the
201+ ` cmt ` along. Once the borrow type is adjusted, we store it in the table, which
202+ basically says what borrows were made for each closure.
203+
204+ ``` rust,ignore
155205self.tables
156206 .borrow_mut()
157207 .upvar_capture_map
158208 .extend(delegate.adjust_upvar_captures);
159209```
210+
211+ [ delegate ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/expr_use_visitor/trait.Delegate.html
212+ [ ibk ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/check/upvar/struct.InferBorrowKind.html
213+ [ cmt ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/mem_categorization/index.html
0 commit comments