|
| 1 | +udp: Fix memory accounting leak. |
| 2 | + |
| 3 | +jira LE-4018 |
| 4 | +cve CVE-2025-22058 |
| 5 | +Rebuild_History Non-Buildable kernel-5.14.0-570.37.1.el9_6 |
| 6 | +commit-author Kuniyuki Iwashima <kuniyu@amazon.com> |
| 7 | +commit df207de9d9e7a4d92f8567e2c539d9c8c12fd99d |
| 8 | +Empty-Commit: Cherry-Pick Conflicts during history rebuild. |
| 9 | +Will be included in final tarball splat. Ref for failed cherry-pick at: |
| 10 | +ciq/ciq_backports/kernel-5.14.0-570.37.1.el9_6/df207de9.failed |
| 11 | + |
| 12 | +Matt Dowling reported a weird UDP memory usage issue. |
| 13 | + |
| 14 | +Under normal operation, the UDP memory usage reported in /proc/net/sockstat |
| 15 | +remains close to zero. However, it occasionally spiked to 524,288 pages |
| 16 | +and never dropped. Moreover, the value doubled when the application was |
| 17 | +terminated. Finally, it caused intermittent packet drops. |
| 18 | + |
| 19 | +We can reproduce the issue with the script below [0]: |
| 20 | + |
| 21 | + 1. /proc/net/sockstat reports 0 pages |
| 22 | + |
| 23 | + # cat /proc/net/sockstat | grep UDP: |
| 24 | + UDP: inuse 1 mem 0 |
| 25 | + |
| 26 | + 2. Run the script till the report reaches 524,288 |
| 27 | + |
| 28 | + # python3 test.py & sleep 5 |
| 29 | + # cat /proc/net/sockstat | grep UDP: |
| 30 | + UDP: inuse 3 mem 524288 <-- (INT_MAX + 1) >> PAGE_SHIFT |
| 31 | + |
| 32 | + 3. Kill the socket and confirm the number never drops |
| 33 | + |
| 34 | + # pkill python3 && sleep 5 |
| 35 | + # cat /proc/net/sockstat | grep UDP: |
| 36 | + UDP: inuse 1 mem 524288 |
| 37 | + |
| 38 | + 4. (necessary since v6.0) Trigger proto_memory_pcpu_drain() |
| 39 | + |
| 40 | + # python3 test.py & sleep 1 && pkill python3 |
| 41 | + |
| 42 | + 5. The number doubles |
| 43 | + |
| 44 | + # cat /proc/net/sockstat | grep UDP: |
| 45 | + UDP: inuse 1 mem 1048577 |
| 46 | + |
| 47 | +The application set INT_MAX to SO_RCVBUF, which triggered an integer |
| 48 | +overflow in udp_rmem_release(). |
| 49 | + |
| 50 | +When a socket is close()d, udp_destruct_common() purges its receive |
| 51 | +queue and sums up skb->truesize in the queue. This total is calculated |
| 52 | +and stored in a local unsigned integer variable. |
| 53 | + |
| 54 | +The total size is then passed to udp_rmem_release() to adjust memory |
| 55 | +accounting. However, because the function takes a signed integer |
| 56 | +argument, the total size can wrap around, causing an overflow. |
| 57 | + |
| 58 | +Then, the released amount is calculated as follows: |
| 59 | + |
| 60 | + 1) Add size to sk->sk_forward_alloc. |
| 61 | + 2) Round down sk->sk_forward_alloc to the nearest lower multiple of |
| 62 | + PAGE_SIZE and assign it to amount. |
| 63 | + 3) Subtract amount from sk->sk_forward_alloc. |
| 64 | + 4) Pass amount >> PAGE_SHIFT to __sk_mem_reduce_allocated(). |
| 65 | + |
| 66 | +When the issue occurred, the total in udp_destruct_common() was 2147484480 |
| 67 | +(INT_MAX + 833), which was cast to -2147482816 in udp_rmem_release(). |
| 68 | + |
| 69 | +At 1) sk->sk_forward_alloc is changed from 3264 to -2147479552, and |
| 70 | +2) sets -2147479552 to amount. 3) reverts the wraparound, so we don't |
| 71 | +see a warning in inet_sock_destruct(). However, udp_memory_allocated |
| 72 | +ends up doubling at 4). |
| 73 | + |
| 74 | +Since commit 3cd3399dd7a8 ("net: implement per-cpu reserves for |
| 75 | +memory_allocated"), memory usage no longer doubles immediately after |
| 76 | +a socket is close()d because __sk_mem_reduce_allocated() caches the |
| 77 | +amount in udp_memory_per_cpu_fw_alloc. However, the next time a UDP |
| 78 | +socket receives a packet, the subtraction takes effect, causing UDP |
| 79 | +memory usage to double. |
| 80 | + |
| 81 | +This issue makes further memory allocation fail once the socket's |
| 82 | +sk->sk_rmem_alloc exceeds net.ipv4.udp_rmem_min, resulting in packet |
| 83 | +drops. |
| 84 | + |
| 85 | +To prevent this issue, let's use unsigned int for the calculation and |
| 86 | +call sk_forward_alloc_add() only once for the small delta. |
| 87 | + |
| 88 | +Note that first_packet_length() also potentially has the same problem. |
| 89 | + |
| 90 | +[0]: |
| 91 | +from socket import * |
| 92 | + |
| 93 | +SO_RCVBUFFORCE = 33 |
| 94 | +INT_MAX = (2 ** 31) - 1 |
| 95 | + |
| 96 | +s = socket(AF_INET, SOCK_DGRAM) |
| 97 | +s.bind(('', 0)) |
| 98 | +s.setsockopt(SOL_SOCKET, SO_RCVBUFFORCE, INT_MAX) |
| 99 | + |
| 100 | +c = socket(AF_INET, SOCK_DGRAM) |
| 101 | +c.connect(s.getsockname()) |
| 102 | + |
| 103 | +data = b'a' * 100 |
| 104 | + |
| 105 | +while True: |
| 106 | + c.send(data) |
| 107 | + |
| 108 | +Fixes: f970bd9e3a06 ("udp: implement memory accounting helpers") |
| 109 | + Reported-by: Matt Dowling <madowlin@amazon.com> |
| 110 | + Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com> |
| 111 | + Reviewed-by: Willem de Bruijn <willemb@google.com> |
| 112 | +Link: https://patch.msgid.link/20250401184501.67377-3-kuniyu@amazon.com |
| 113 | + Signed-off-by: Jakub Kicinski <kuba@kernel.org> |
| 114 | +(cherry picked from commit df207de9d9e7a4d92f8567e2c539d9c8c12fd99d) |
| 115 | + Signed-off-by: Jonathan Maple <jmaple@ciq.com> |
| 116 | + |
| 117 | +# Conflicts: |
| 118 | +# net/ipv4/udp.c |
| 119 | +diff --cc net/ipv4/udp.c |
| 120 | +index a783cc8e339f,2742cc7602bb..000000000000 |
| 121 | +--- a/net/ipv4/udp.c |
| 122 | ++++ b/net/ipv4/udp.c |
| 123 | +@@@ -1487,10 -1650,8 +1487,15 @@@ static void udp_rmem_release(struct soc |
| 124 | + if (!rx_queue_lock_held) |
| 125 | + spin_lock(&sk_queue->lock); |
| 126 | + |
| 127 | +++<<<<<<< HEAD |
| 128 | + + |
| 129 | + + sk->sk_forward_alloc += size; |
| 130 | + + amt = (sk->sk_forward_alloc - partial) & ~(PAGE_SIZE - 1); |
| 131 | + + sk->sk_forward_alloc -= amt; |
| 132 | +++======= |
| 133 | ++ amt = (size + sk->sk_forward_alloc - partial) & ~(PAGE_SIZE - 1); |
| 134 | ++ sk_forward_alloc_add(sk, size - amt); |
| 135 | +++>>>>>>> df207de9d9e7 (udp: Fix memory accounting leak.) |
| 136 | + |
| 137 | + if (amt) |
| 138 | + __sk_mem_reduce_allocated(sk, amt >> PAGE_SHIFT); |
| 139 | +* Unmerged path net/ipv4/udp.c |
0 commit comments