Add UCB Pattern Search #1115
Add UCB Pattern Search #1115
Conversation
meta-cla
bot
added
the
CLA Signed
|
|
||
| def encode_dim(self) -> int: | ||
| """ | ||
| Returns the dimension of the output of encode |
There was a problem hiding this comment.
I don't understand what this is? What is it encoding?
There was a problem hiding this comment.
This is just the dimension of the encoding
There was a problem hiding this comment.
Changed the name to be more clear
| """ | ||
| raise NotImplementedError | ||
|
|
||
| def encode(self, value: object) -> list[float]: |
There was a problem hiding this comment.
Can we reuse encode_scalar here? If not, I'd like to combine this with encode_scalar since they are solving the same problem.
There was a problem hiding this comment.
@jansel This is pretty much identical to encode_scalar for integer and poweroftwo. But the previous encode_scalar did not have any functionality for ListOf or PermutationFragments. How should we handle those two?
There was a problem hiding this comment.
I'm happy to rename everything to encode_scalar, but given ListOf and PermutationFragments, I think we should allow encode to output a list of floats
| self.cat_dims = [] | ||
| offset = 0 | ||
| for spec in self.config_gen.flat_spec: | ||
| n_dims = spec.encode_dim() | ||
| if spec.is_categorical(): | ||
| # All dimensions of this encoder are categorical | ||
| self.cat_dims.extend(range(offset, offset + n_dims)) | ||
| offset += n_dims |
There was a problem hiding this comment.
This seems kind of error prone, can we have a cleaner way to do it?
…into ucb_pattern_search
| # Initialize config encoder | ||
| self.frac_selected = frac_selected | ||
|
|
||
| # compute offsets from the flat_spec |
2 participants
Includes the UCB Pattern Search autotuner, which modifies PatternSearch to search through configs using a Gaussian Process and the Upper Confidence Bound (UCB) acquisition function.
This improves over PatternSearch in kernel latency and autotuning wall-clock time on B200 for a set of benchmark kernels. DifferentialEvolution can improve further upon this, but takes substantially longer. DESurrogate in #1096, has comparable performance.
Kernel latency:
Autotuning Wall-clock Speedup:
Autotuning Convergence Time:
