Auto merge of #103214 - Nilstrieb:set-theory, r=petrochenkov

Use Set instead of Vec in transitive_relation

Helps with #103195. It doesn't fix the underlying quadraticness but it makes it _a lot_ faster to an extent where even doubling the amount of nested references still takes less than two seconds (50s on nightly).

I want to see whether this causes regressions (because the vec was usually quite small) or improvements (as lookup for bigger sets is now much faster) in real code.

Author: bors