|
60 | 60 | }, |
61 | 61 | { |
62 | 62 | "cell_type": "code", |
63 | | - "execution_count": 71, |
| 63 | + "execution_count": null, |
64 | 64 | "metadata": {}, |
65 | 65 | "outputs": [], |
66 | 66 | "source": [ |
|
70 | 70 | }, |
71 | 71 | { |
72 | 72 | "cell_type": "code", |
73 | | - "execution_count": 72, |
| 73 | + "execution_count": null, |
74 | 74 | "metadata": {}, |
75 | 75 | "outputs": [], |
76 | 76 | "source": [ |
|
86 | 86 | }, |
87 | 87 | { |
88 | 88 | "cell_type": "code", |
89 | | - "execution_count": 73, |
| 89 | + "execution_count": null, |
90 | 90 | "metadata": {}, |
91 | 91 | "outputs": [], |
92 | 92 | "source": [ |
93 | 93 | "### Generate some data\n", |
94 | 94 | "torch.manual_seed(7) # Set the random seed so things are predictable\n", |
95 | 95 | "\n", |
96 | | - "# Features are 5 random normal variables\n", |
| 96 | + "# Features are 3 random normal variables\n", |
97 | 97 | "features = torch.randn((1, 5))\n", |
98 | 98 | "# True weights for our data, random normal variables again\n", |
99 | 99 | "weights = torch.randn_like(features)\n", |
|
119 | 119 | }, |
120 | 120 | { |
121 | 121 | "cell_type": "code", |
122 | | - "execution_count": 74, |
| 122 | + "execution_count": null, |
123 | 123 | "metadata": {}, |
124 | | - "outputs": [ |
125 | | - { |
126 | | - "output_type": "execute_result", |
127 | | - "data": { |
128 | | - "text/plain": "tensor([[0.1595]])" |
129 | | - }, |
130 | | - "metadata": {}, |
131 | | - "execution_count": 74 |
132 | | - } |
133 | | - ], |
| 124 | + "outputs": [], |
134 | 125 | "source": [ |
135 | | - "## Calculate the output of this network using the weights and bias tensors\n", |
136 | | - "output = activation(torch.sum(features*weights) + bias)\n", |
137 | | - "output" |
| 126 | + "## Calculate the output of this network using the weights and bias tensors" |
138 | 127 | ] |
139 | 128 | }, |
140 | 129 | { |
|
156 | 145 | "RuntimeError: size mismatch, m1: [1 x 5], m2: [1 x 5] at /Users/soumith/minicondabuild3/conda-bld/pytorch_1524590658547/work/aten/src/TH/generic/THTensorMath.c:2033\n", |
157 | 146 | "```\n", |
158 | 147 | "\n", |
159 | | - "As you're building neural networks in any framework, you'll see this often. Really often. What's happening here is our tensors aren't the correct shapes to perform a matrix multiplication. Remember that for matrix multiplications, the number of columns in the first tensor must equal to the number of rows in the second tensor. Both `features` and `weights` have the same shape, `(1, 5)`. This means we need to change the shape of `weights` to get the matrix multiplication to work.\n", |
| 148 | + "As you're building neural networks in any framework, you'll see this often. Really often. What's happening here is our tensors aren't the correct shapes to perform a matrix multiplication. Remember that for matrix multiplications, the number of columns in the first tensor must equal to the number of rows in the second column. Both `features` and `weights` have the same shape, `(1, 5)`. This means we need to change the shape of `weights` to get the matrix multiplication to work.\n", |
160 | 149 | "\n", |
161 | 150 | "**Note:** To see the shape of a tensor called `tensor`, use `tensor.shape`. If you're building neural networks, you'll be using this method often.\n", |
162 | 151 | "\n", |
163 | | - "There are a few options here: [`weights.reshape()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.reshape), [`weights.resize_()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.resize_), [`weights.view()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.view) and [`torch.transpose(weights,0,1)`](https://pytorch.org/docs/master/generated/torch.transpose.html).\n", |
| 152 | + "There are a few options here: [`weights.reshape()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.reshape), [`weights.resize_()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.resize_), and [`weights.view()`](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.view).\n", |
164 | 153 | "\n", |
165 | 154 | "* `weights.reshape(a, b)` will return a new tensor with the same data as `weights` with size `(a, b)` sometimes, and sometimes a clone, as in it copies the data to another part of memory.\n", |
166 | 155 | "* `weights.resize_(a, b)` returns the same tensor with a different shape. However, if the new shape results in fewer elements than the original tensor, some elements will be removed from the tensor (but not from memory). If the new shape results in more elements than the original tensor, new elements will be uninitialized in memory. Here I should note that the underscore at the end of the method denotes that this method is performed **in-place**. Here is a great forum thread to [read more about in-place operations](https://discuss.pytorch.org/t/what-is-in-place-operation/16244) in PyTorch.\n", |
167 | 156 | "* `weights.view(a, b)` will return a new tensor with the same data as `weights` with size `(a, b)`.\n", |
168 | | - "* `torch.transpose(weights,0,1)` will return transposed weights tensor. This returns transposed version of inpjut tensor along dim 0 and dim 1. This is efficient since we do not specify to actual dimesions of weights.\n", |
169 | 157 | "\n", |
170 | 158 | "I usually use `.view()`, but any of the three methods will work for this. So, now we can reshape `weights` to have five rows and one column with something like `weights.view(5, 1)`.\n", |
171 | 159 | "\n", |
172 | | - "One more approach is to use `.t()` to transpose vector of weights, in our case from (1,5) to (5,1) shape.\n", |
173 | | - |
174 | 160 | "> **Exercise**: Calculate the output of our little network using matrix multiplication." |
175 | 161 | ] |
176 | 162 | }, |
177 | 163 | { |
178 | 164 | "cell_type": "code", |
179 | | - "execution_count": 75, |
| 165 | + "execution_count": null, |
180 | 166 | "metadata": {}, |
181 | | - "outputs": [ |
182 | | - { |
183 | | - "output_type": "execute_result", |
184 | | - "data": { |
185 | | - "text/plain": "tensor([[0.1595]])" |
186 | | - }, |
187 | | - "metadata": {}, |
188 | | - "execution_count": 75 |
189 | | - } |
190 | | - ], |
| 167 | + "outputs": [], |
191 | 168 | "source": [ |
192 | | - "## Calculate the output of this network using matrix multiplication\n", |
193 | | - "output = activation(torch.matmul(features,torch.transpose(weights,0,1)) + bias)\n", |
194 | | - "output" |
| 169 | + "## Calculate the output of this network using matrix multiplication" |
195 | 170 | ] |
196 | 171 | }, |
197 | 172 | { |
|
229 | 204 | }, |
230 | 205 | { |
231 | 206 | "cell_type": "code", |
232 | | - "execution_count": 76, |
| 207 | + "execution_count": null, |
233 | 208 | "metadata": {}, |
234 | 209 | "outputs": [], |
235 | 210 | "source": [ |
|
263 | 238 | }, |
264 | 239 | { |
265 | 240 | "cell_type": "code", |
266 | | - "execution_count": 77, |
| 241 | + "execution_count": null, |
267 | 242 | "metadata": {}, |
268 | | - "outputs": [ |
269 | | - { |
270 | | - "output_type": "execute_result", |
271 | | - "data": { |
272 | | - "text/plain": "tensor([[0.3171]])" |
273 | | - }, |
274 | | - "metadata": {}, |
275 | | - "execution_count": 77 |
276 | | - } |
277 | | - ], |
| 243 | + "outputs": [], |
278 | 244 | "source": [ |
279 | | - "## Your solution here\n", |
280 | | - "h = activation(torch.matmul(features,W1).add_(B1))\n", |
281 | | - "output = activation(torch.matmul(h,W2).add_(B2))\n", |
282 | | - "output" |
| 245 | + "## Your solution here" |
283 | 246 | ] |
284 | 247 | }, |
285 | 248 | { |
|
288 | 251 | "source": [ |
289 | 252 | "If you did this correctly, you should see the output `tensor([[ 0.3171]])`.\n", |
290 | 253 | "\n", |
291 | | - "The number of hidden units are a parameter of the network, often called a **hyperparameter** to differentiate it from the weights and biases parameters. As you'll see later when we discuss training a neural network, the more hidden units a network has, and the more layers, the better able it is to learn from data and make accurate predictions." |
| 254 | + "The number of hidden units a parameter of the network, often called a **hyperparameter** to differentiate it from the weights and biases parameters. As you'll see later when we discuss training a neural network, the more hidden units a network has, and the more layers, the better able it is to learn from data and make accurate predictions." |
292 | 255 | ] |
293 | 256 | }, |
294 | 257 | { |
|
302 | 265 | }, |
303 | 266 | { |
304 | 267 | "cell_type": "code", |
305 | | - "execution_count": 78, |
| 268 | + "execution_count": null, |
306 | 269 | "metadata": {}, |
307 | | - "outputs": [ |
308 | | - { |
309 | | - "output_type": "execute_result", |
310 | | - "data": { |
311 | | - "text/plain": "array([[0.01999898, 0.8199435 , 0.49156905],\n [0.41055049, 0.77689295, 0.34885976],\n [0.18349863, 0.75363566, 0.92894509],\n [0.55251871, 0.60749635, 0.21301188]])" |
312 | | - }, |
313 | | - "metadata": {}, |
314 | | - "execution_count": 78 |
315 | | - } |
316 | | - ], |
| 270 | + "outputs": [], |
317 | 271 | "source": [ |
318 | 272 | "import numpy as np\n", |
319 | 273 | "np.set_printoptions(precision=8)\n", |
|
323 | 277 | }, |
324 | 278 | { |
325 | 279 | "cell_type": "code", |
326 | | - "execution_count": 79, |
| 280 | + "execution_count": null, |
327 | 281 | "metadata": {}, |
328 | | - "outputs": [ |
329 | | - { |
330 | | - "output_type": "execute_result", |
331 | | - "data": { |
332 | | - "text/plain": "tensor([[0.0200, 0.8199, 0.4916],\n [0.4106, 0.7769, 0.3489],\n [0.1835, 0.7536, 0.9289],\n [0.5525, 0.6075, 0.2130]], dtype=torch.float64)" |
333 | | - }, |
334 | | - "metadata": {}, |
335 | | - "execution_count": 79 |
336 | | - } |
337 | | - ], |
| 282 | + "outputs": [], |
338 | 283 | "source": [ |
339 | 284 | "torch.set_printoptions(precision=8)\n", |
340 | 285 | "b = torch.from_numpy(a)\n", |
|
343 | 288 | }, |
344 | 289 | { |
345 | 290 | "cell_type": "code", |
346 | | - "execution_count": 80, |
| 291 | + "execution_count": null, |
347 | 292 | "metadata": {}, |
348 | | - "outputs": [ |
349 | | - { |
350 | | - "output_type": "execute_result", |
351 | | - "data": { |
352 | | - "text/plain": "array([[0.01999898, 0.8199435 , 0.49156905],\n [0.41055049, 0.77689295, 0.34885976],\n [0.18349863, 0.75363566, 0.92894509],\n [0.55251871, 0.60749635, 0.21301188]])" |
353 | | - }, |
354 | | - "metadata": {}, |
355 | | - "execution_count": 80 |
356 | | - } |
357 | | - ], |
| 293 | + "outputs": [], |
358 | 294 | "source": [ |
359 | 295 | "b.numpy()" |
360 | 296 | ] |
|
368 | 304 | }, |
369 | 305 | { |
370 | 306 | "cell_type": "code", |
371 | | - "execution_count": 81, |
| 307 | + "execution_count": null, |
372 | 308 | "metadata": {}, |
373 | | - "outputs": [ |
374 | | - { |
375 | | - "output_type": "execute_result", |
376 | | - "data": { |
377 | | - "text/plain": "tensor([[0.0400, 1.6399, 0.9831],\n [0.8211, 1.5538, 0.6977],\n [0.3670, 1.5073, 1.8579],\n [1.1050, 1.2150, 0.4260]], dtype=torch.float64)" |
378 | | - }, |
379 | | - "metadata": {}, |
380 | | - "execution_count": 81 |
381 | | - } |
382 | | - ], |
| 309 | + "outputs": [], |
383 | 310 | "source": [ |
384 | 311 | "# Multiply PyTorch Tensor by 2, in place\n", |
385 | 312 | "b.mul_(2)" |
386 | 313 | ] |
387 | 314 | }, |
388 | 315 | { |
389 | 316 | "cell_type": "code", |
390 | | - "execution_count": 82, |
| 317 | + "execution_count": null, |
391 | 318 | "metadata": {}, |
392 | | - "outputs": [ |
393 | | - { |
394 | | - "output_type": "execute_result", |
395 | | - "data": { |
396 | | - "text/plain": "array([[0.03999795, 1.639887 , 0.9831381 ],\n [0.82110098, 1.55378589, 0.69771953],\n [0.36699725, 1.50727133, 1.85789017],\n [1.10503742, 1.2149927 , 0.42602377]])" |
397 | | - }, |
398 | | - "metadata": {}, |
399 | | - "execution_count": 82 |
400 | | - } |
401 | | - ], |
| 319 | + "outputs": [], |
402 | 320 | "source": [ |
403 | 321 | "# Numpy array matches new values from Tensor\n", |
404 | 322 | "a" |
|
407 | 325 | ], |
408 | 326 | "metadata": { |
409 | 327 | "kernelspec": { |
410 | | - "display_name": "Python 3.7.7 64-bit ('envTorch': conda)", |
| 328 | + "display_name": "Python 3", |
411 | 329 | "language": "python", |
412 | | - "name": "python37764bitenvtorchcondaf1e8697b5c364e0493551efccbc5e8bb" |
| 330 | + "name": "python3" |
413 | 331 | }, |
414 | 332 | "language_info": { |
415 | 333 | "codemirror_mode": { |
|
421 | 339 | "name": "python", |
422 | 340 | "nbconvert_exporter": "python", |
423 | 341 | "pygments_lexer": "ipython3", |
424 | | - "version": "3.7.7-final" |
| 342 | + "version": "3.6.6" |
425 | 343 | } |
426 | 344 | }, |
427 | 345 | "nbformat": 4, |
|
0 commit comments