YaResnet, MXResNet, Mish

Author: ayasyrev

README.md

@@ -25,15 +25,15 @@ And by creating constructor object, then modify it and then create model.
 
 First import constructor class, then create model constructor oject.
 
-```
+```python
 from model_constructor.net import *
 ```
 
-```
+```python
 model = Net()
 ```
 
-```
+```python
 model
 ```
 
@@ -46,7 +46,7 @@ model
 
 Now we have model consructor, defoult setting as xresnet18. And we can get model after call it.
 
-```
+```python
 model.c_in
 ```
 
@@ -57,7 +57,7 @@ model.c_in
 
 
 
-```
+```python
 model.c_out
 ```
 
@@ -68,7 +68,7 @@ model.c_out
 
 
 
-```
+```python
 model.stem_sizes
 ```
 
@@ -79,7 +79,7 @@ model.stem_sizes
 
 
 
-```
+```python
 model.layers
 ```
 
@@ -90,7 +90,7 @@ model.layers
 
 
 
-```
+```python
 model.expansion
 ```
 
@@ -101,7 +101,7 @@ model.expansion
 
 
 
-```
+```python
 model()
 ```
 
@@ -277,14 +277,14 @@ model()
 If you want to change model, just change constructor parameters.  
 Lets create xresnet50.
 
-```
+```python
 model.expansion = 4
 model.layers = [3,4,6,3]
 ```
 
 Now we can look at model body and if we call constructor - we have pytorch model!
 
-```
+```python
 model.body
 ```
 
@@ -623,7 +623,7 @@ model.body
 
 
 
-```
+```python
 model.block_szs
 ```
 
@@ -644,20 +644,20 @@ But now lets create model as mxresnet50 from fastai forums tread https://forums.
 
 Lets create mxresnet constructor.
 
-```
+```python
 mxresnet = Net()
 ```
 
 Then lets modify stem.
 
-```
+```python
 mxresnet.stem_sizes = [3,32,64,64]
 ```
 
 Now lets change activation function to Mish.
 Here is link to forum disscussion https://forums.fast.ai/t/meet-mish-new-activation-function-possible-successor-to-relu
 
-```
+```python
 class Mish(nn.Module):
     def __init__(self):
         super().__init__()
@@ -666,7 +666,7 @@ class Mish(nn.Module):
         return x *( torch.tanh(F.softplus(x))) 
 ```
 
-```
+```python
 mxresnet.expansion = 4
 mxresnet.layers = [3,4,6,3]
 mxresnet.act_fn = Mish()
@@ -677,7 +677,7 @@ Now we have mxresnet50 constructor.
 We can inspect some parts of it.  
 And after call it we got model.
 
-```
+```python
 mxresnet
 ```
 
@@ -688,7 +688,7 @@ mxresnet
 
 
 
-```
+```python
 mxresnet.stem.conv_1
 ```
 
@@ -703,7 +703,7 @@ mxresnet.stem.conv_1
 
 
 
-```
+```python
 mxresnet.body.l_0.bl_0
 ```
 
@@ -738,13 +738,13 @@ mxresnet.body.l_0.bl_0
 
 Now lets change Resblock. NewResBlock (stiil not own name yet) is in lib from version 0.1.0
 
-```
+```python
 mxresnet.block = NewResBlock
 ```
 
 That all. Let see what we have.
 
-```
+```python
 mxresnet.body.l_1.bl_0
 ```
 
@@ -782,46 +782,46 @@ mxresnet.body.l_1.bl_0
 
 Usual way to get model - call constructor with parametrs.
 
-```
+```python
 from model_constructor.constructor import *
 ```
 
 Default is resnet18.
 
-```
+```python
 model = Net()
 ```
 
 You cant modify model after call constructor, so define model with parameters.   
 For example, resnet34:
 
-```
+```python
 resnet34 = Net(block=BasicBlock, blocks=[3, 4, 6, 3])
 ```
 
 ## Predefined Resnet models - 18, 34, 50.
 
-```
+```python
 from model_constructor.resnet import *
 ```
 
-```
+```python
 model = resnet34(num_classes=10)
 ```
 
-```
+```python
 model = resnet50(num_classes=10)
 ```
 
 ## Predefined Xresnet from fastai 1.
 
 This ie simplified version from fastai v1. I did refactoring for better understand and experiment with models. For example, it's very simple to change activation funtions, different stems, batchnorm and activation order etc. In v2 much powerfull realisation.
 
-```
+```python
 from model_constructor.xresnet import *
 ```
 
-```
+```python
 model = xresnet50()
 ```
 
@@ -836,11 +836,11 @@ Here is some examples:
 
 Stem with 3 conv layers
 
-```
+```python
 model = Net(stem=partial(Stem, stem_sizes=[32, 32]))
 ```
 
-```
+```python
 model.stem
 ```
 
@@ -869,11 +869,11 @@ model.stem
 
 
 
-```
+```python
 model = Net(stem_sizes=[32, 64])
 ```
 
-```
+```python
 model.stem
 ```
 
@@ -904,11 +904,11 @@ model.stem
 
 ### Activation function before Normalization
 
-```
+```python
 model = Net(bn_1st=False)
 ```
 
-```
+```python
 model.stem
 ```
 
@@ -930,15 +930,15 @@ model.stem
 
 ### Change activation function
 
-```
+```python
 new_act_fn = nn.LeakyReLU(inplace=True)
 ```
 
-```
+```python
 model = Net(act_fn=new_act_fn)
 ```
 
-```
+```python
 model.stem
 ```
 
@@ -957,7 +957,7 @@ model.stem
 
 
 
-```
+```python
 model.body.layer_0.block_0
 ```