CVE-2024-56547: rcu/nocb: Fix missed RCU barrier on deoffloading
In the Linux kernel, the following vulnerability has been resolved:
rcu/nocb: Fix missed RCU barrier on deoffloading
Currently, running rcutorture test with torturetype=rcu fwdprogress=8 nbarriercbs=8 nocbsnthreads=8 nocbstoggle=100 onoffinterval=60 testboost=2, will trigger the following warning:
WARNING: CPU: 19 PID: 100 at kernel/rcu/treenocb.h:1061 rcunocbrdpdeoffload+0x292/0x2a0 RIP: 0010:rcunocbrdpdeoffload+0x292/0x2a0 Call Trace: <TASK> ? warn+0x7e/0x120 ? rcunocbrdpdeoffload+0x292/0x2a0 ? reportbug+0x18e/0x1a0 ? handlebug+0x3d/0x70 ? excinvalidop+0x18/0x70 ? asmexcinvalidop+0x1a/0x20 ? rcunocbrdpdeoffload+0x292/0x2a0 rcunocbcpudeoffload+0x70/0xa0 rcunocbtoggle+0x136/0x1c0 ? pfxrcunocbtoggle+0x10/0x10 kthread+0xd1/0x100 ? pfxkthread+0x10/0x10 retfromfork+0x2f/0x50 ? pfxkthread+0x10/0x10 retfromforkasm+0x1a/0x30 </TASK>
CPU0 CPU2 CPU3 //rcunocbtoggle //nocbcbwait //rcutorture
// deoffload CPU1 // process CPU1's rdp rcubarrier() rcusegcblistentrain() rcusegcblistaddlen(1); // len == 2 // enqueue barrier // callback to CPU1's // rdp->cblist rcudobatch() // invoke CPU1's rdp->cblist // callback rcubarriercallback() rcubarrier() mutexlock(&rcustate.barriermutex); // still see len == 2 // enqueue barrier callback // to CPU1's rdp->cblist rcusegcblistentrain() rcusegcblistaddlen(1); // len == 3 // decrement len rcusegcblistaddlen(-2); kthreadparkme()
// CPU1's rdp->cblist len == 1 // Warn because there is // still a pending barrier // trigger warning WARNONONCE(rcusegcblistncbs(&rdp->cblist)); cpusreadunlock();
// wait CPU1 to comes online and // invoke barrier callback on // CPU1 rdp's->cblist waitforcompletion(&rcustate.barriercompletion); // deoffload CPU4 cpusreadlock() rcubarrier() mutexlock(&rcustate.barriermutex); // block on barriermutex // wait rcubarrier() on // CPU3 to unlock barriermutex // but CPU3 unlock barriermutex // need to wait CPU1 comes online // when CPU1 going online will block on cpuswritelock
The above scenario will not only trigger a WARNONONCE(), but also trigger a deadlock.
Thanks to nocb locking, a second racing rcubarrier() on an offline CPU will either observe the decremented callback counter down to 0 and spare the callback enqueue, or rcuo will observe the new callback and keep rdp->nocbcbsleep to false.
Therefore check rdp->nocbcbsleep before parking to make sure no further rcubarrier() is waiting on the rdp.