@@ -75,7 +75,7 @@ Possible interpretations of the graph include
7575
7676* Minimum cost for supplier to reach a destination.
7777* Routing of packets on the internet (minimize time).
78- * Etc ., etc.
78+ * etc ., etc.
7979
8080For this simple graph, a quick scan of the edges shows that the optimal paths are
8181
@@ -97,7 +97,7 @@ For large graphs, we need a systematic solution.
9797
9898Let $J(v)$ denote the minimum cost-to-go from node $v$, understood as the total cost from $v$ if we take the best route.
9999
100- Suppose that we know $J(v)$ for each node $v$, as shown below for the graph from the preceding example
100+ Suppose that we know $J(v)$ for each node $v$, as shown below for the graph from the preceding example.
101101
102102``` {figure} /_static/lecture_specific/short_path/graph2.png
103103
@@ -134,7 +134,7 @@ the function $J$ satisfies
134134J(v) = \min_{w \in F_v} \{ c(v, w) + J(w) \}
135135```
136136
137- This is known as the * Bellman equation* , after the mathematician Richard Bellman.
137+ This is known as the ** Bellman equation** , after the mathematician [ Richard Bellman] ( https://en.wikipedia.org/wiki/Richard_E._Bellman ) .
138138
139139The Bellman equation can be thought of as a restriction that $J$ must
140140satisfy.
@@ -215,7 +215,7 @@ Q = np.array([[inf, 1, 5, 3, inf, inf, inf],
215215
216216Notice that the cost of staying still (on the principle diagonal) is set to
217217
218- * np.inf for non-destination nodes --- moving on is required.
218+ * ` np.inf ` for non-destination nodes --- moving on is required.
219219* 0 for the destination node --- here is where we stop.
220220
221221For the sequence of approximations $\{ J_n\} $ of the cost-to-go functions, we can use NumPy arrays.
@@ -232,18 +232,14 @@ i = 0
232232
233233while i < max_iter:
234234 for v in nodes:
235- # minimize Q[v, w] + J[w] over all choices of w
236- lowest_cost = inf
237- for w in nodes:
238- cost = Q[v, w] + J[w]
239- if cost < lowest_cost:
240- lowest_cost = cost
241- next_J[v] = lowest_cost
242- if np.equal(next_J, J).all():
235+ # Minimize Q[v, w] + J[w] over all choices of w
236+ next_J[v] = np.min(Q[v, :] + J)
237+
238+ if np.array_equal(next_J, J):
243239 break
244- else:
245- J[:] = next_J # Copy contents of next_J to J
246- i += 1
240+
241+ J[:] = next_J # Copy contents of next_J to J
242+ i += 1
247243
248244print("The cost-to-go function is", J)
249245```
@@ -426,11 +422,7 @@ The minimization step is vectorized to make it faster.
426422
427423``` {code-cell} python3
428424def bellman(J, Q):
429- num_nodes = Q.shape[0]
430- next_J = np.empty_like(J)
431- for v in range(num_nodes):
432- next_J[v] = np.min(Q[v, :] + J)
433- return next_J
425+ return np.min(Q + J, axis=1)
434426
435427
436428def compute_cost_to_go(Q):
0 commit comments