@@ -29,7 +29,6 @@ for data in train_set
2929end
3030```
3131
32- It is important that every ` update! ` step receives a newly gradient computed gradient.
3332This loop can also be written using the function [ ` train! ` ] (@ref Flux.Train.train!),
3433but it's helpful to undersand the pieces first:
3534
4342
4443Fist recall from the section on [ taking gradients] (@ref man-taking-gradients) that
4544` Flux.gradient(f, a, b) ` always calls ` f(a, b) ` , and returns a tuple ` (∂f_∂a, ∂f_∂b) ` .
46- In the code above, the function ` f ` is an anonymous function with one argument,
47- created by the ` do ` block, hence ` grads ` is a tuple with one element.
45+ In the code above, the function ` f ` passed to ` gradient ` is an anonymous function with
46+ one argument, created by the ` do ` block, hence ` grads ` is a tuple with one element.
4847Instead of a ` do ` block, we could have written:
4948
5049``` julia
@@ -58,6 +57,9 @@ structures are what Zygote calls "explicit" gradients.
5857It is important that the execution of the model takes place inside the call to ` gradient ` ,
5958in order for the influence of the model's parameters to be observed by Zygote.
6059
60+ It is also important that every ` update! ` step receives a newly gradient computed gradient,
61+ as this will be change whenever the model's parameters are changed, and for each new data point.
62+
6163!!! compat "Explicit vs implicit gradients"
6264 Flux ≤ 0.13 used Zygote's "implicit" mode, in which ` gradient ` takes a zero-argument function.
6365 It looks like this:
0 commit comments