CVE-2024-50095: RDMA/mad: Improve handling of timed out WRs of mad agent
In the Linux kernel, the following vulnerability has been resolved:
RDMA/mad: Improve handling of timed out WRs of mad agent
Current timeout handler of mad agent acquires/releases madagentpriv lock for every timed out WRs. This causes heavy locking contention when higher no. of WRs are to be handled inside timeout handler.
This leads to softlockup with below trace in some use cases where rdma-cm path is used to establish connection between peer nodes
Trace: ----- BUG: soft lockup - CPU#4 stuck for 26s! [kworker/u128:3:19767] CPU: 4 PID: 19767 Comm: kworker/u128:3 Kdump: loaded Tainted: G OE ------- --- 5.14.0-427.13.1.el94.x8664 #1 Hardware name: Dell Inc. PowerEdge R740/01YM03, BIOS 2.4.8 11/26/2019 Workqueue: ibmad1 timeoutsends [ibcore] RIP: 0010:dosoftirq+0x78/0x2ac RSP: 0018:ffffb253449e4f98 EFLAGS: 00000246 RAX: 00000000ffffffff RBX: 0000000000000000 RCX: 000000000000001f RDX: 000000000000001d RSI: 000000003d1879ab RDI: fff363b66fd3a86b RBP: ffffb253604cbcd8 R08: 0000009065635f3b R09: 0000000000000000 R10: 0000000000000040 R11: ffffb253449e4ff8 R12: 0000000000000000 R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000040 FS: 0000000000000000(0000) GS:ffff8caa1fc80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fd9ec9db900 CR3: 0000000891934006 CR4: 00000000007706e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 PKRU: 55555554 Call Trace: <IRQ> ? showtraceloglvl+0x1c4/0x2df ? showtraceloglvl+0x1c4/0x2df ? irqexitrcu+0xa1/0xc0 ? watchdogtimerfn+0x1b2/0x210 ? pfxwatchdogtimerfn+0x10/0x10 ? hrtimerrunqueues+0x127/0x2c0 ? hrtimerinterrupt+0xfc/0x210 ? sysvecapictimerinterrupt+0x5c/0x110 ? sysvecapictimerinterrupt+0x37/0x90 ? asmsysvecapictimerinterrupt+0x16/0x20 ? dosoftirq+0x78/0x2ac ? dosoftirq+0x60/0x2ac irqexitrcu+0xa1/0xc0 sysveccallfunctionsingle+0x72/0x90 </IRQ> <TASK> asmsysveccallfunctionsingle+0x16/0x20 RIP: 0010:rawspinunlockirq+0x14/0x30 RSP: 0018:ffffb253604cbd88 EFLAGS: 00000247 RAX: 000000000001960d RBX: 0000000000000002 RCX: ffff8cad2a064800 RDX: 000000008020001b RSI: 0000000000000001 RDI: ffff8cad5d39f66c RBP: ffff8cad5d39f600 R08: 0000000000000001 R09: 0000000000000000 R10: ffff8caa443e0c00 R11: ffffb253604cbcd8 R12: ffff8cacb8682538 R13: 0000000000000005 R14: ffffb253604cbd90 R15: ffff8cad5d39f66c cmprocesssenderror+0x122/0x1d0 [ibcm] timeoutsends+0x1dd/0x270 [ibcore] processonework+0x1e2/0x3b0 ? pfxworkerthread+0x10/0x10 workerthread+0x50/0x3a0 ? pfxworkerthread+0x10/0x10 kthread+0xdd/0x100 ? pfxkthread+0x10/0x10 retfromfork+0x29/0x50 </TASK>
Simplified timeout handler by creating local list of timed out WRs and invoke send handler post creating the list. The new method acquires/ releases lock once to fetch the list and hence helps to reduce locking contetiong when processing higher no. of WRs
Other sources
This CVE was automatically created from a reference found in an email or other text. If you are reading this, then this CVE entry is probably erroneous, since this text should be replaced by the official CVE description automatically.
— Launchpad
Affected Software
Remediation
Recommended actions to resolve this vulnerability, in priority order.
- Upgrade
Upgrade
debian/linuxto a version that resolves this vulnerability.Fixed in 5.10.234-1Fixed in 6.1.129-1Fixed in 6.1.135-1Fixed in 6.12.25-1Fixed in 6.12.27-1 - Upgrade
Upgrade
debian/linux-6.1to a version that resolves this vulnerability.Fixed in 6.1.129-1~deb11u1 - Upgrade
Upgrade
Linux kernelto a version that resolves this vulnerability.Fixed in 5.14.0-427.13.1.el9_4.x86_64 - Compensating control
For environments using the rdma-cm path to establish connections between peer nodes, mitigate the observed soft lockup caused by heavy locking contention in RDMA/mad timeout handling (e.g., temporarily reduce connection establishment/timeout load until the kernel fix is deployed).
Event History
Frequently Asked Questions
What is the severity of CVE-2024-50095?
CVE-2024-50095 has a medium severity rating due to potential locking contention affecting performance.
How do I fix CVE-2024-50095?
To fix CVE-2024-50095, upgrade to the recommended kernel versions 6.1.123-1, 6.1.119-1, 6.12.10-1, or 6.12.11-1 from Debian.
Which systems are affected by CVE-2024-50095?
CVE-2024-50095 affects multiple versions of the Linux kernel, particularly those prior to 5.10.227 and between 5.11 and 5.15.168.
What components of the Linux kernel are impacted by CVE-2024-50095?
CVE-2024-50095 specifically impacts the RDMA/mad component of the Linux kernel.
Is CVE-2024-50095 exploitable remotely?
CVE-2024-50095 does not appear to be remotely exploitable, but it may degrade performance under heavy load.