Skip to content

Commit 6a2108c

Browse files
Shivaji Kantkuba-moo
authored andcommitted
net: devmem: refresh devmem TX dst in case of route invalidation
The zero-copy Device Memory (Devmem) transmit path relies on the socket's route cache (`dst_entry`) to validate that the packet is being sent via the network device to which the DMA buffer was bound. However, this check incorrectly fails and returns `-ENODEV` if the socket's route cache entry (`dst`) is merely missing or expired (`dst == NULL`). This scenario is observed during network events, such as when flow steering rules are deleted, leading to a temporary route cache invalidation. This patch fixes -ENODEV error for `net_devmem_get_binding()` by doing the following: 1. It attempts to rebuild the route via `rebuild_header()` if the route is initially missing (`dst == NULL`). This allows the TCP/IP stack to recover from transient route cache misses. 2. It uses `rcu_read_lock()` and `dst_dev_rcu()` to safely access the network device pointer (`dst_dev`) from the route, preventing use-after-free conditions if the device is concurrently removed. 3. It maintains the critical safety check by validating that the retrieved destination device (`dst_dev`) is exactly the device registered in the Devmem binding (`binding->dev`). These changes prevent unnecessary ENODEV failures while maintaining the critical safety requirement that the Devmem resources are only used on the bound network device. Reviewed-by: Bobby Eshleman <bobbyeshleman@meta.com> Reported-by: Eric Dumazet <edumazet@google.com> Reported-by: Vedant Mathur <vedantmathur@google.com> Suggested-by: Eric Dumazet <edumazet@google.com> Fixes: bd61848 ("net: devmem: Implement TX path") Signed-off-by: Shivaji Kant <shivajikant@google.com> Link: https://patch.msgid.link/20251029065420.3489943-1-shivajikant@google.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
1 parent a38eeec commit 6a2108c

File tree

1 file changed

+24
-3
lines changed

1 file changed

+24
-3
lines changed

net/core/devmem.c

Lines changed: 24 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
#include <net/page_pool/helpers.h>
1818
#include <net/page_pool/memory_provider.h>
1919
#include <net/sock.h>
20+
#include <net/tcp.h>
2021
#include <trace/events/page_pool.h>
2122

2223
#include "devmem.h"
@@ -357,7 +358,8 @@ struct net_devmem_dmabuf_binding *net_devmem_get_binding(struct sock *sk,
357358
unsigned int dmabuf_id)
358359
{
359360
struct net_devmem_dmabuf_binding *binding;
360-
struct dst_entry *dst = __sk_dst_get(sk);
361+
struct net_device *dst_dev;
362+
struct dst_entry *dst;
361363
int err = 0;
362364

363365
binding = net_devmem_lookup_dmabuf(dmabuf_id);
@@ -366,16 +368,35 @@ struct net_devmem_dmabuf_binding *net_devmem_get_binding(struct sock *sk,
366368
goto out_err;
367369
}
368370

371+
rcu_read_lock();
372+
dst = __sk_dst_get(sk);
373+
/* If dst is NULL (route expired), attempt to rebuild it. */
374+
if (unlikely(!dst)) {
375+
if (inet_csk(sk)->icsk_af_ops->rebuild_header(sk)) {
376+
err = -EHOSTUNREACH;
377+
goto out_unlock;
378+
}
379+
dst = __sk_dst_get(sk);
380+
if (unlikely(!dst)) {
381+
err = -ENODEV;
382+
goto out_unlock;
383+
}
384+
}
385+
369386
/* The dma-addrs in this binding are only reachable to the corresponding
370387
* net_device.
371388
*/
372-
if (!dst || !dst->dev || dst->dev->ifindex != binding->dev->ifindex) {
389+
dst_dev = dst_dev_rcu(dst);
390+
if (unlikely(!dst_dev) || unlikely(dst_dev != binding->dev)) {
373391
err = -ENODEV;
374-
goto out_err;
392+
goto out_unlock;
375393
}
376394

395+
rcu_read_unlock();
377396
return binding;
378397

398+
out_unlock:
399+
rcu_read_unlock();
379400
out_err:
380401
if (binding)
381402
net_devmem_dmabuf_binding_put(binding);

0 commit comments

Comments
 (0)