Simplify lightweight clones, including into closures and async blocks #3680
Conversation
joshtriplett
added
T-lang
|
Personally, I don't feel that the non-closure/block use cases of this are really strong enough to warrant adding this, and the closure/block use case can be fixed with The example let obj: Arc<LargeComplexObject> = new_large_complex_object();
some_function(obj.use); // Pass a separate use of the object to `some_function`
obj.method(); // The object is still owned afterwardscould just be written as Which can presumably be solved either by making LLVM smarter about atomics for the specific case of The ergonomics of needing to clone in a block are annoying though, I agree, but that's a smaller feature by being able to do: spawn(async clone {
func1(x).await;
func2(y).await;
});and similarly for closures. |
The problem with a Even without the
I think there's potential value there (and I've captured this in the RFC); for instance, we could omit a clone of a |
|
I'm afraid I'm pretty negative about this RFC.
|
|
I've added a new paragraph in the "Rationale and alternatives" section explaining why
|
|
@Diggsey wrote:
You can; I'm not suggesting that we couldn't provide a syntax for that, too. However, people have asked for the ability to distinguish between expensive and lightweight clones. And a lightweight clone is less of a big deal, making it safer to have a lighter-weight syntax and let users mostly not worry about it. We could additionally provide syntax for performing expensive clones; I've mentioned one such syntax in the future work section, but we could consider others as well if that's a common use case.
That assumes that users want to call
This is only true if we omitted the proposed elision behavior, or if we decide that it's acceptable for methods to have elision semantics attached to them. I agree that in either of those cases there's no particular reason to use a special syntax rather than a method.
This is a reasonable point. I personally don't think this would cause problems, but at a minimum I'll capture this in the alternatives section, and we could consider changing the elision behavior to make it required. The annoying thing about making it required is that we then have to implement it before shipping the feature and we can never make it better after shipping the feature. I don't think that's a good tradeoff. Ultimately, though, I think the elisions aren't the most important part of this feature, and this feature is well worth shipping without the elisions, so if the elisions fail to reach consensus we can potentially ship the feature without the elisions. (Omitting the elisions entirely is already called out as an alternative.)
See the previous points about people wanting to distinguish lightweight clones specifically. This is a load-bearing point: I can absolutely understand that if you disagree with the motivation of distinguishing lightweight clones, the remainder of the RFC then does not follow. The RFC is based on the premise that people do in fact want to distinguish lightweight clones specifically.
I've added this as an alternative, but I don't think that would be nearly as usable. |
|
@Diggsey wrote:
While I don't think this is dangerous, I do think it's not the ideal solution, and I'd love to find a better way to specify this. The goal is to |
|
Thank you for working on this RFC! PyO3 necessarily makes heavy use of Python reference counting so users working on Rust + Python projects may benefit significantly from making this more ergonomic. The possibility to elide operations where unnecessary is also very interesting; while it's a new idea to me, performance optimizations are always great! I have some questions:
|
|
Since "Precise capturing" #3617 also abuses the |
|
We should really not overload the usage of the keyword Isn't it easier to understand if we've some macro for the multiple clones that run before the code that consumes them, but still inside some distinguished scope? In this, the We use this multi-clones pattern outside async code too, so this non-async specific approach benefits everyone. |
…tsakis Ergonomic ref counting: optimize away clones when possible This PR build on top of rust-lang/rust#134797. It optimizes codegen of ergonomic ref-counting when the type being `use`d is only known to be copy after monomorphization. We avoid codening a clone and generate bitwise copy instead. RFC: rust-lang/rfcs#3680 Tracking issue: rust-lang/rust#132290 Project goal: rust-lang/rust-project-goals#107 r? `@nikomatsakis` This PR could better sit on top of rust-lang/rust#131650 but as it did not land yet I've decided to just do minimal changes. It may be the case that doing what I'm doing regress the performance and we may need to go the full route of rust-lang/rust#131650. cc `@saethlin` in this regard.
…tsakis Ergonomic ref counting: optimize away clones when possible This PR build on top of rust-lang/rust#134797. It optimizes codegen of ergonomic ref-counting when the type being `use`d is only known to be copy after monomorphization. We avoid codening a clone and generate bitwise copy instead. RFC: rust-lang/rfcs#3680 Tracking issue: rust-lang/rust#132290 Project goal: rust-lang/rust-project-goals#107 r? `@nikomatsakis` This PR could better sit on top of rust-lang/rust#131650 but as it did not land yet I've decided to just do minimal changes. It may be the case that doing what I'm doing regress the performance and we may need to go the full route of rust-lang/rust#131650. cc `@saethlin` in this regard.
I can't see why |
I cannot find |
|
Very good proposal, thank you! The rationale is spot on. I'd rather use more explicit names, even if they are longer. I imagine I would see Likewise, the The I would not worry too much about compatibility with older editions and having to import some extra traits/macros. I'm fine with typing more as long as I get an implementation where every field does what's expected of it (move, fast clone, copy) with or without prodding (but without extra variables and extra scopes, please) and doesn't do what's not expected (slow clone, unwanted move). |
|
While replying to someone on Lobste.rs, I realized part of why I think this is ill-fitted for Rust: It's a recurring pattern in Rust that things like Ideas like making automatic clone/refcount incrementing a property of a type rather than a property of the call site feel like the memory consumption analogue to silently acquiring locks as needed, despite the risk of deadlocks, or silently bubbling up On an abstract level, Rust seems pretty consistent about acquire-y operations being explicit but release-y operations being allowed to be implicit via In fact, auto-borrow of EDIT: Basically, as I see it, Rust is explicit in the acquisition, to encourage and aid you in being conservative in how much you acquire, when, and it's implicit and end-of-scope automatic in the release as a "Rust didn't originally have NLL" conservative choice for minimizing how long and how much you keep held. They're two sides of the same "hold as few resources as possible for as short a time as possible" coin and they harmonize well with how much can be checked through compile-time lifetime analysis. ...so, at worst, I wouldn't want to see "lightweight" clones made "simpler" at each call site than (I'm pretty sure I've already stated my belief that EDIT: To be clear, I wrote this based on either me or someone else having already addressed the stance I take on why something like |
|
The idiom that we want to simplify is this: The previously described The form The form It seems to me that this could be solved by a new prefix operator This operator could be read as "clone from outer context" (or "lightweight clone from outer context" if we want to restrict its usage to those types) and would only be allowed in function-defining contexts such as closures and async blocks. This seems to be best of both worlds - an explicit operator that is easy to type and visually appears right next to the variable name. Its advantages are even clearer when you have a mix of variables that you want to clone and not clone: is rewritten into It also could support cloning multiple times in a block: would be rewritten as |
This is still a big improvement over the status quo: much less verbose than Good enough IMO, though |
|
Closure captures also allows you to introduce some kind of sigil syntax sugar through macros to annotate which bindings are "automatically captured". I imagine something like this could be useful for dioxus like code: #[component]
fn Compoonent(state: Arc<State>) {
let x = || ^state.things();
}
// The macro can do simple syntax transformation into:
#[component]
fn Compoonent(state: Arc<State>) {
let x = use(state) || state.things();
} |
|
My issue with the proposed |
|
Any time I see this pop up I keep thinking that for the years I used C++ I never had this problem, I just used the explicit capture syntax. My vote is for copying that success, something like: clone (a) || {
// a is cloned
}
clone, move (a, b) || {
// a and b are moved, rest are cloned
}This matches Rust Analyzer's closure capture hints, and is very similar to the reclone crate , almost identical to the closure crate. It also fits right beside the existing move syntax at the start of the lambda. Fundamentally - putting all the stuff that a lambda captures all in one place feels much more easy to read and explicit than scanning through the function body for any code that calls |
|
As stated before the |
|
I think one alternative which hasn't been explored is tail position capture information. As a parallel, when writing a function, one writes: fn a_function<A, B, C, D>(an_argument: A, another_argument: B, yet_another_argument: C) -> Vec<D>
where
A: SomeTrait,
B: AnotherTrait,
C: YetAnotherTrait;The main advantage of doing so is that the function signature (arguments & result type) are obvious at a glance, without the fluff getting in the way. The same consideration applies to capture information. When considering a slightly more realistic example -- scaled up with more likely identifier lengths, number of captured identifiers, and number of arguments -- the upfront syntax makes it harder to see where the signature really begins: move(a_capture.clone(), b_capture.clone(), c_capture.clone(), d_capture.clone()) |an_argument, another_argument, yet_another_argument| -> AResultType {
...
}Formatting may help, though it looks a bit awkward to me: move(a_capture.clone(), b_capture.clone(), c_capture.clone(), d_capture.clone())
|an_argument, another_argument, yet_another_argument| -> AResultType {
...
}A tail position syntax, however, puts the signature upfront, regardless of formatting: |an_argument, another_argument, yet_another_argument| -> AResultType
with
clone(a_capture, b_capture, c_capture, d_capture),
{
...
}|an_argument, another_argument, yet_another_argument| -> AResultType with clone(a_capture, b_capture, c_capture, d_capture),
{
...
}It's less clear what parsing ambiguities do arise from this choice, though. It's possible that a contextual |
Since this reads as "move the clone", is there any chance to have it directly inline, as in: // Instead of:
let c = move(long_name.clone()) |arg| long_name.method(arg);
// Something like:
let c = |arg| long_name.clone().move.method(arg);? |
|
@iago-lito Then you'd have to scan the closure body to see what's being cloned, which makes it hard to follow / breaks the explicit flow. |
|
Agreed. The ergonomic gain only makes sense in small closures like this one. I guess the alternative we could document to avoid having to repeat let c = move(l = long_name.clone()) |arg| l.method(arg); |
|
One step further in ergonomics beyond explicit capture clauses would be allowing the explicit activation of behavioral traits, which would result in different move semantics. For example, given a let a: Handle
let b: Handle
let closure = use<Handle> || {
// a and b are clones here, because we are explicitly calling the activation of the Handle trait.
}Would be equivalent to the explicitly doing this: let a: Handle
let b: Handle
let closure = move(a = a.clone(), b= b.clone()) || {
// a and b are clones here
}The first example could also be explicitly restricted with The main advantage, other than being still quite explicit, is that you could add different capture semantics by just introducing more traits. But that could also be understood as a disadvantage, since this complicates the understanding of Traits a bit more.
|
|
Seeing https://smallcultfollowing.com/babysteps/blog/2025/11/05/bikeshedding-handle/ made me realize another reason I'm opposed to this: Testing something like this in Rust itself, rather than in a third-party crate, feels like an abuse of power... as if boats had skipped making fehler a crate where the ecosystem could properly express that it's not a good fit, and gone straight to getting it in Rust. Why does this get to be fast-tracked into the 2025H2 goals where it can potentially become an "Oh, sorry. Too late to get rid of it. Crates are depending on it now." when things like fehler and delegate, which I see as equally ill-fitted to Rust's design philosophy, have to demonstrate their appropriateness as third-party crates first? (Especially when the perception that "things in the standard library are more trustworthy because supply chain attacks" will skew people's willingness to start depending on it compared to a proper trial run as a third-party crate.) EDIT: I wouldn't mind as much if this were something that could be forbidden on a codebase-by-codebase basis using a Clippy lint. It'd be a drift toward "C++ is many little dialects and a maintainable codebase must use policy and tooling to keep contributors within one", but that's still tolerable at this stage. ...but the effects of "making costs less explicit" in a transitive dependency propagate throughout the codebase and show up as a flamegraph or heap trace that has no obvious places to fix. |
|
The proposed syntax of So, I agreed to folks that proposed something like this: let closure = use (move a, b = b.clone(), c_copy = c.clone()) || {}But, remember that this reduces the ergonomic of the case where user just want to move their variables. How about upgrades the current // Strict mode
let closure = move (a, b = b.clone(), c_copy = c.clone(), d_typed: Box<_> = Box::new(d)) || {}
// Backward compatible loose mode
let closure = move || {}Since it is always a Formatting, in my opinion, the current move syntax is already awkward. By estimating rustfmt current behavior, it would be something like this (I prefer the verbatim fn main() {
let closure = move (
a, a1, a2, a3,
b = b.clone(),
c_copy = c.clone(),
d_typed: Box<_> = Box::new(d)
) |arg_a,
arg_b,
long_arg_c,
some_typed_d: Option<u32>,
arg_e,
arg_f,
arg_long_long_g| {
println!("{arg_a} {arg_b} {long_arg_c}");
};
} |
|
@ssokolow I think you may be misunderstanding what is happening regarding this feature. I do think there may be an experimental nightly feature being developped in the compiler for this, but that's a normal thing we do to get a feel for features. We have processes for that: the experiment stays nightly-only with a "this feature is incomplete, be warned" warning to discourage people from depending on it. It does sometimes happen that a feature is useful and complete enough that people start depending on it, but we're far from that with this one. And we have in the past ripped out such experiments from the compiler if they're not getting approved, that's the whole point of nightly features. The reason this isn't a third-party crate is that we're talking about a new language feature. This can't just be implemented with a macro, or indeed we would have started with that. If you're worried about the
Because the lang team felt that this particular issue was important to solve, based on what they've seen of what users stumble on. Note that the project goal isn't about this particular RFC, it's about finding some solution to the problem of ergonomic Arc etc. I wouldn't even say that this RFC is the leading proposal right now. Happy to continue this discussion on Zulip if you see something I don't. |
OK, I admit I lost perspective on that part. Once I've decided whether I'm up to creating a Zulip account, you may see me there. |
|
let closure = a.clone(), b.clone() in move || {
a.something();
b.something();
};let a.clone(), b.clone() in closure = move || {
a.something();
b.something();
}; |
😕 ok that makes me confused, if we are back to the drawing board why is this RFC still open |
Closing this would mean a decision that this really isn't the right solution; I didn't want to give the impression that there's any consensus like that. I mostly read blogs and issues and Zulip chats sometimes and it looked to me like the "explicit captures" idea was gaining traction is all. I don't feel like this one has been discarded at all. |
|
There actually some blog posts about this by @nikomatsakis:
It contains what I've proposed above for syntax.
Is there other RFC related to this? |
Not that I'm aware of no, the proposals I've seen are mostly Niko's posts :) |
There is discussion on some kind of related idea on zulip though, using |
The let g = move || f(e().super);
<=>
let g = {
let _temp = e();
move || f(_temp);
};This is basically the abuse of Now IMO postfix let g = move |x| x.clone().super; // ??? |
58 participants
Provide a feature to simplify performing lightweight clones (such as of
Arc/Rc), particularly cloning them into closures or async blocks, whilestill keeping such cloning visible and explicit.
A very common source of friction in asynchronous or multithreaded Rust
programming is having to clone various
Arc<T>reference-counted objects intoan async block or task. This is particularly common when spawning a closure as
a thread, or spawning an async block as a task. Common patterns for doing so
include:
All of these patterns introduce noise every time the program wants to spawn a
thread or task, or otherwise clone an object into a closure or async block.
Feedback on Rust regularly brings up this friction, seeking a simpler solution.
This RFC proposes solutions to minimize the syntactic weight of
lightweight-cloning objects, particularly cloning objects into a closure or
async block, while still keeping an indication of this operation.
This RFC is part of the "Ergonomic ref-counting" project goal, owned by
@jkelleyrtp. Thanks to @jkelleyrtp and @nikomatsakis for reviewing. Thanks to
@nikomatsakis for key insights in this RFC, including the idea to use
use.Rendered