Commit 8679208
committed
Auto merge of rust-lang#109670 - lqd:init-mask, r=oli-obk
Make init mask lazy for fully initialized/uninitialized const allocations
There are a few optimization opportunities in the `InitMask` and related const `Allocation`s (e.g. by taking advantage of the fact that it's a bitset that represents initialization, which is often entirely initialized or uninitialized in a single call, or gradually built up, etc).
There's a few overwrites to the same state, multiple writes in a row to the same indices, the RLE scheme for `memcpy` doesn't always compress, etc.
Here, we start with:
- avoiding materializing the bitset's blocks if the allocation is fully initialized/uninitialized
- dealloc blocks when fully overwriting, including when participating in `memcpy`s
- take care of the fixme about allocating blocks of 0s before overwriting them to the expected value
- expanding unit test coverage of the init mask
This should be most visible on benchmarks and crates where const allocations dominate the runtime (like `ctfe-stress-5` of course), but I was especially looking at the worst cases from rust-lang#93215.
This first change allows the majority of `set_range` calls to stay with a lazy init mask when bootstrapping rustc (not that the init mask is a big part of the process in cpu time or memory usage).
r? `@oli-obk`
I have another in-progress branch where I'll switch the singular initialized/uninitialized value to a watermark, recording the point after which everything is uninitialized. That will take care of cases where full initialization is monotonic and done in multiple steps (e.g. an array of a type without padding), which should then allow the vast majority of const allocations' init masks to stay lazy during bootstrapping (though interestingly I've seen such gradual initialization in both left-to-right and right-to-left directions, and I don't think a single watermark can handle both).File tree
6 files changed
+447
-103
lines changed- compiler/rustc_middle/src/mir/interpret
- allocation
- init_mask
- tests/codegen
- remap_path_prefix
6 files changed
+447
-103
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
6 | | - | |
7 | 5 | | |
8 | 6 | | |
9 | 7 | | |
| |||
0 commit comments