Skip to content

Commit 9d9e3f9

Browse files
ayasyrevayasyrev
committed
docs after cfg
1 parent e5880b8 commit 9d9e3f9

File tree

6 files changed

+1361
-337
lines changed

6 files changed

+1361
-337
lines changed

‎Nbs/00_ModelConstructor.ipynb‎

Lines changed: 416 additions & 45 deletions
Large diffs are not rendered by default.

‎Nbs/index.ipynb‎

Lines changed: 310 additions & 102 deletions
Large diffs are not rendered by default.

‎README.md‎

Lines changed: 187 additions & 129 deletions
Large diffs are not rendered by default.

‎docs/00_ModelConstructor.md‎

Lines changed: 302 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -85,10 +85,11 @@ block
8585

8686

8787
```python
88+
8889
block = ResBlock(2,64,64,act_fn=nn.LeakyReLU(), bn_1st=False)
8990
block
9091
```
91-
???+ done "output"
92+
??? done "output"
9293
<pre>ResBlock(
9394
(convs): Sequential(
9495
(conv_0): ConvBnAct(
@@ -113,10 +114,11 @@ block
113114

114115

115116
```python
116-
block = ResBlock(2, 32, 64, dw=True)
117+
118+
lock = ResBlock(2, 32, 64, dw=True)
117119
block
118120
```
119-
???+ done "output"
121+
??? done "output"
120122
<pre>ResBlock(
121123
(convs): Sequential(
122124
(conv_0): ConvBnAct(
@@ -152,10 +154,11 @@ pool = nn.AvgPool2d(2, ceil_mode=True)
152154

153155

154156
```python
157+
155158
block = ResBlock(2, 32, 64, stride=2, dw=True, pool=pool)
156159
block
157160
```
158-
???+ done "output"
161+
??? done "output"
159162
<pre>ResBlock(
160163
(convs): Sequential(
161164
(conv_0): ConvBnAct(
@@ -216,9 +219,9 @@ block
216219
(se): SEModule(
217220
(squeeze): AdaptiveAvgPool2d(output_size=1)
218221
(excitation): Sequential(
219-
(fc_reduce): Linear(in_features=128, out_features=8, bias=True)
222+
(reduce): Linear(in_features=128, out_features=8, bias=True)
220223
(se_act): ReLU(inplace=True)
221-
(fc_expand): Linear(in_features=8, out_features=128, bias=True)
224+
(expand): Linear(in_features=8, out_features=128, bias=True)
222225
(se_gate): Sigmoid()
223226
)
224227
)
@@ -261,9 +264,9 @@ block
261264
(se): SEModule(
262265
(squeeze): AdaptiveAvgPool2d(output_size=1)
263266
(excitation): Sequential(
264-
(fc_reduce): Linear(in_features=128, out_features=8, bias=True)
267+
(reduce): Linear(in_features=128, out_features=8, bias=True)
265268
(se_act): ReLU(inplace=True)
266-
(fc_expand): Linear(in_features=8, out_features=128, bias=True)
269+
(expand): Linear(in_features=8, out_features=128, bias=True)
267270
(se_gate): Sigmoid()
268271
)
269272
)
@@ -283,12 +286,302 @@ block
283286

284287

285288

289+
## Stem, Body, Layer, Head
290+
291+
Helper functions to create stem, body and head of model from config.
292+
Returns `nn.Sequential`.
293+
294+
295+
```python
296+
from model_constructor.model_constructor import CfgMC, make_stem, make_body, make_layer, make_head
297+
from rich import print
298+
```
299+
300+
301+
```python
302+
cfg = CfgMC()
303+
print(cfg)
304+
```
305+
306+
307+
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="color: #800080; text-decoration-color: #800080; font-weight: bold">CfgMC</span><span style="font-weight: bold">(</span>
308+
<span style="color: #808000; text-decoration-color: #808000">name</span>=<span style="color: #008000; text-decoration-color: #008000">'MC'</span>,
309+
<span style="color: #808000; text-decoration-color: #808000">in_chans</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">3</span>,
310+
<span style="color: #808000; text-decoration-color: #808000">num_classes</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">1000</span>,
311+
<span style="color: #808000; text-decoration-color: #808000">block</span>=<span style="font-weight: bold">&lt;</span><span style="color: #ff00ff; text-decoration-color: #ff00ff; font-weight: bold">class</span><span style="color: #000000; text-decoration-color: #000000"> </span><span style="color: #008000; text-decoration-color: #008000">'model_constructor.model_constructor.ResBlock'</span><span style="font-weight: bold">&gt;</span>,
312+
<span style="color: #808000; text-decoration-color: #808000">conv_layer</span>=<span style="font-weight: bold">&lt;</span><span style="color: #ff00ff; text-decoration-color: #ff00ff; font-weight: bold">class</span><span style="color: #000000; text-decoration-color: #000000"> </span><span style="color: #008000; text-decoration-color: #008000">'model_constructor.layers.ConvBnAct'</span><span style="font-weight: bold">&gt;</span>,
313+
<span style="color: #808000; text-decoration-color: #808000">block_sizes</span>=<span style="font-weight: bold">[</span><span style="color: #008080; text-decoration-color: #008080; font-weight: bold">64</span>, <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">128</span>, <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">256</span>, <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">512</span><span style="font-weight: bold">]</span>,
314+
<span style="color: #808000; text-decoration-color: #808000">layers</span>=<span style="font-weight: bold">[</span><span style="color: #008080; text-decoration-color: #008080; font-weight: bold">2</span>, <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">2</span>, <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">2</span>, <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">2</span><span style="font-weight: bold">]</span>,
315+
<span style="color: #808000; text-decoration-color: #808000">norm</span>=<span style="font-weight: bold">&lt;</span><span style="color: #ff00ff; text-decoration-color: #ff00ff; font-weight: bold">class</span><span style="color: #000000; text-decoration-color: #000000"> </span><span style="color: #008000; text-decoration-color: #008000">'torch.nn.modules.batchnorm.BatchNorm2d'</span><span style="font-weight: bold">&gt;</span>,
316+
<span style="color: #808000; text-decoration-color: #808000">act_fn</span>=<span style="color: #800080; text-decoration-color: #800080; font-weight: bold">ReLU</span><span style="font-weight: bold">(</span><span style="color: #808000; text-decoration-color: #808000">inplace</span>=<span style="color: #00ff00; text-decoration-color: #00ff00; font-style: italic">True</span><span style="font-weight: bold">)</span>,
317+
<span style="color: #808000; text-decoration-color: #808000">pool</span>=<span style="color: #800080; text-decoration-color: #800080; font-weight: bold">AvgPool2d</span><span style="font-weight: bold">(</span><span style="color: #808000; text-decoration-color: #808000">kernel_size</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">2</span>, <span style="color: #808000; text-decoration-color: #808000">stride</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">2</span>, <span style="color: #808000; text-decoration-color: #808000">padding</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0</span><span style="font-weight: bold">)</span>,
318+
<span style="color: #808000; text-decoration-color: #808000">expansion</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">1</span>,
319+
<span style="color: #808000; text-decoration-color: #808000">groups</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">1</span>,
320+
<span style="color: #808000; text-decoration-color: #808000">dw</span>=<span style="color: #ff0000; text-decoration-color: #ff0000; font-style: italic">False</span>,
321+
<span style="color: #808000; text-decoration-color: #808000">div_groups</span>=<span style="color: #800080; text-decoration-color: #800080; font-style: italic">None</span>,
322+
<span style="color: #808000; text-decoration-color: #808000">sa</span>=<span style="color: #ff0000; text-decoration-color: #ff0000; font-style: italic">False</span>,
323+
<span style="color: #808000; text-decoration-color: #808000">se</span>=<span style="color: #ff0000; text-decoration-color: #ff0000; font-style: italic">False</span>,
324+
<span style="color: #808000; text-decoration-color: #808000">bn_1st</span>=<span style="color: #00ff00; text-decoration-color: #00ff00; font-style: italic">True</span>,
325+
<span style="color: #808000; text-decoration-color: #808000">zero_bn</span>=<span style="color: #00ff00; text-decoration-color: #00ff00; font-style: italic">True</span>,
326+
<span style="color: #808000; text-decoration-color: #808000">stem_stride_on</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">0</span>,
327+
<span style="color: #808000; text-decoration-color: #808000">stem_sizes</span>=<span style="font-weight: bold">[</span><span style="color: #008080; text-decoration-color: #008080; font-weight: bold">32</span>, <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">32</span>, <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">64</span><span style="font-weight: bold">]</span>,
328+
<span style="color: #808000; text-decoration-color: #808000">stem_pool</span>=<span style="color: #800080; text-decoration-color: #800080; font-weight: bold">MaxPool2d</span><span style="font-weight: bold">(</span><span style="color: #808000; text-decoration-color: #808000">kernel_size</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">3</span>, <span style="color: #808000; text-decoration-color: #808000">stride</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">2</span>, <span style="color: #808000; text-decoration-color: #808000">padding</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">1</span>, <span style="color: #808000; text-decoration-color: #808000">dilation</span>=<span style="color: #008080; text-decoration-color: #008080; font-weight: bold">1</span>, <span style="color: #808000; text-decoration-color: #808000">ceil_mode</span>=<span style="color: #ff0000; text-decoration-color: #ff0000; font-style: italic">False</span><span style="font-weight: bold">)</span>,
329+
<span style="color: #808000; text-decoration-color: #808000">stem_bn_end</span>=<span style="color: #ff0000; text-decoration-color: #ff0000; font-style: italic">False</span>
330+
<span style="font-weight: bold">)</span>
331+
</pre>
332+
333+
334+
335+
### Stem
336+
337+
338+
```python
339+
340+
stem = make_stem(cfg)
341+
stem
342+
```
343+
??? done "output"
344+
<pre>Sequential(
345+
(conv_0): ConvBnAct(
346+
(conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
347+
(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
348+
(act_fn): ReLU(inplace=True)
349+
)
350+
(conv_1): ConvBnAct(
351+
(conv): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
352+
(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
353+
(act_fn): ReLU(inplace=True)
354+
)
355+
(stem_pool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
356+
)
357+
358+
359+
360+
### Layer
361+
362+
`make_layer` need `layer_num` argument - number of layer.
363+
364+
365+
```python
366+
367+
layer = make_layer(cfg, layer_num=0)
368+
layer
369+
```
370+
??? done "output"
371+
<pre>Sequential(
372+
(bl_0): ResBlock(
373+
(convs): Sequential(
374+
(conv_0): ConvBnAct(
375+
(conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
376+
(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
377+
(act_fn): ReLU(inplace=True)
378+
)
379+
(conv_1): ConvBnAct(
380+
(conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
381+
(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
382+
)
383+
)
384+
(act_fn): ReLU(inplace=True)
385+
)
386+
(bl_1): ResBlock(
387+
(convs): Sequential(
388+
(conv_0): ConvBnAct(
389+
(conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
390+
(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
391+
(act_fn): ReLU(inplace=True)
392+
)
393+
(conv_1): ConvBnAct(
394+
(conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
395+
(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
396+
)
397+
)
398+
(act_fn): ReLU(inplace=True)
399+
)
400+
)
401+
402+
403+
404+
### Body
405+
406+
`make_body` needs `cfg._make_layer` initialized. As default - `make_layer`.
407+
408+
409+
```python
410+
411+
cfg._make_layer = make_layer
412+
body = make_body(cfg)
413+
body
414+
```
415+
??? done "output"
416+
<pre>Sequential(
417+
(l_0): Sequential(
418+
(bl_0): ResBlock(
419+
(convs): Sequential(
420+
(conv_0): ConvBnAct(
421+
(conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
422+
(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
423+
(act_fn): ReLU(inplace=True)
424+
)
425+
(conv_1): ConvBnAct(
426+
(conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
427+
(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
428+
)
429+
)
430+
(act_fn): ReLU(inplace=True)
431+
)
432+
(bl_1): ResBlock(
433+
(convs): Sequential(
434+
(conv_0): ConvBnAct(
435+
(conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
436+
(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
437+
(act_fn): ReLU(inplace=True)
438+
)
439+
(conv_1): ConvBnAct(
440+
(conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
441+
(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
442+
)
443+
)
444+
(act_fn): ReLU(inplace=True)
445+
)
446+
)
447+
(l_1): Sequential(
448+
(bl_0): ResBlock(
449+
(convs): Sequential(
450+
(conv_0): ConvBnAct(
451+
(conv): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
452+
(bn): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
453+
(act_fn): ReLU(inplace=True)
454+
)
455+
(conv_1): ConvBnAct(
456+
(conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
457+
(bn): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
458+
)
459+
)
460+
(id_conv): Sequential(
461+
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
462+
(id_conv): ConvBnAct(
463+
(conv): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
464+
(bn): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
465+
)
466+
)
467+
(act_fn): ReLU(inplace=True)
468+
)
469+
(bl_1): ResBlock(
470+
(convs): Sequential(
471+
(conv_0): ConvBnAct(
472+
(conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
473+
(bn): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
474+
(act_fn): ReLU(inplace=True)
475+
)
476+
(conv_1): ConvBnAct(
477+
(conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
478+
(bn): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
479+
)
480+
)
481+
(act_fn): ReLU(inplace=True)
482+
)
483+
)
484+
(l_2): Sequential(
485+
(bl_0): ResBlock(
486+
(convs): Sequential(
487+
(conv_0): ConvBnAct(
488+
(conv): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
489+
(bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
490+
(act_fn): ReLU(inplace=True)
491+
)
492+
(conv_1): ConvBnAct(
493+
(conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
494+
(bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
495+
)
496+
)
497+
(id_conv): Sequential(
498+
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
499+
(id_conv): ConvBnAct(
500+
(conv): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
501+
(bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
502+
)
503+
)
504+
(act_fn): ReLU(inplace=True)
505+
)
506+
(bl_1): ResBlock(
507+
(convs): Sequential(
508+
(conv_0): ConvBnAct(
509+
(conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
510+
(bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
511+
(act_fn): ReLU(inplace=True)
512+
)
513+
(conv_1): ConvBnAct(
514+
(conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
515+
(bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
516+
)
517+
)
518+
(act_fn): ReLU(inplace=True)
519+
)
520+
)
521+
(l_3): Sequential(
522+
(bl_0): ResBlock(
523+
(convs): Sequential(
524+
(conv_0): ConvBnAct(
525+
(conv): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
526+
(bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
527+
(act_fn): ReLU(inplace=True)
528+
)
529+
(conv_1): ConvBnAct(
530+
(conv): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
531+
(bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
532+
)
533+
)
534+
(id_conv): Sequential(
535+
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
536+
(id_conv): ConvBnAct(
537+
(conv): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
538+
(bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
539+
)
540+
)
541+
(act_fn): ReLU(inplace=True)
542+
)
543+
(bl_1): ResBlock(
544+
(convs): Sequential(
545+
(conv_0): ConvBnAct(
546+
(conv): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
547+
(bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
548+
(act_fn): ReLU(inplace=True)
549+
)
550+
(conv_1): ConvBnAct(
551+
(conv): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
552+
(bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
553+
)
554+
)
555+
(act_fn): ReLU(inplace=True)
556+
)
557+
)
558+
)
559+
560+
561+
562+
## Head
563+
564+
565+
```python
566+
567+
head = make_head(cfg)
568+
head
569+
```
570+
??? done "output"
571+
<pre>Sequential(
572+
(pool): AdaptiveAvgPool2d(output_size=1)
573+
(flat): Flatten(start_dim=1, end_dim=-1)
574+
(fc): Linear(in_features=512, out_features=1000, bias=True)
575+
)
576+
577+
578+
286579
## Model Constructor.
287580

288581

289582
```python
290583
mc = ModelConstructor()
291-
mc
584+
mc.print_cfg()
292585
```
293586
???+ done "output"
294587
<pre>MC constructor
@@ -301,7 +594,6 @@ mc
301594

302595

303596

304-
305597
```python
306598

307599
mc.stem

0 commit comments

Comments
 (0)