Commit 7249dd7
layernorm: enlarge the range for 2-pass reduction (#2282)
From the OOB models, there are some shapes still below the performance
expectation with large M but small N.
Simple shapes:
[128, 197, 384]
[64,784, 256]
[(64, 28, 28, 256]
[256, 197, 256]
[128, 196, 384]
After enlarging the range for 2-pass reduction, these models can benefit
an average of 10-20ms model execution time and optimize the geomean
performance of eager training in timm models from 0.835 to 0.842.
---------
Co-authored-by: Eikan Wang <eikan.wang@intel.com>1 parent a5f45df commit 7249dd7
1 file changed
+4
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1063 | 1063 | | |
1064 | 1064 | | |
1065 | 1065 | | |
1066 | | - | |
1067 | | - | |
| 1066 | + | |
| 1067 | + | |
| 1068 | + | |
| 1069 | + | |
1068 | 1070 | | |
1069 | 1071 | | |
1070 | 1072 | | |
| |||
0 commit comments