Skip to content

Conversation

@darsnack
Copy link
Member

@darsnack darsnack commented Feb 17, 2022

This refactors the squeeze and excite (SE) layer and inverted bottleneck layer to make them easier to use with other models. Previously, each MobileNet variant used its own variant of the inverted bottleneck layer. In #113, many of the functions are being reused. So, the motivation for this PR is to make the codepaths shared across these models.

There is now a single invertedresidual function that is used by both MobileNet v2 + v3. The activation function can be any function, and the configs/builders for each variant select the specific ones for those models. It also accepts the reduction factor for the SE layer as a keyword. One upside of this that the MobileNet v3 config can now allow a user to specify arbitrary reduction factors (previously it was 4 or no SE).

I've moved and renamed selayer to squeeze_excite. The former is fine for primarily internal use, but now this function is intended to be a reusable block. No functional changes.

I also moved and renamed _make_divisible to _round_channels just to make it more clear. Maybe there's a better name, but it is internal.

@darsnack darsnack requested a review from ToucheSir February 17, 2022 20:06
@darsnack darsnack merged commit dae5b87 into FluxML:master Feb 18, 2022
@darsnack darsnack deleted the invert-bottleneck branch February 18, 2022 18:28
@darsnack darsnack mentioned this pull request Feb 18, 2022
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants