CVE-2024-56368: ring-buffer: Fix overflow in __rb_map_vma

In the Linux kernel, the following vulnerability has been resolved:

ring-buffer: Fix overflow in rbmapvma

An overflow occurred when performing the following calculation:

nrpages = ((nrsubbufs + 1) << subbuforder) - pgoff;

Add a check before the calculation to avoid this problem.

syzbot reported this as a slab-out-of-bounds in rbmapvma:

BUG: KASAN: slab-out-of-bounds in rbmapvma+0x9ab/0xae0 kernel/trace/ringbuffer.c:7058 Read of size 8 at addr ffff8880767dd2b8 by task syz-executor187/5836

CPU: 0 UID: 0 PID: 5836 Comm: syz-executor187 Not tainted 6.13.0-rc2-syzkaller-00159-gf932fb9b4074 #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/25/2024 Call Trace: <TASK> dumpstack lib/dumpstack.c:94 [inline] dumpstacklvl+0x116/0x1f0 lib/dumpstack.c:120 printaddressdescription mm/kasan/report.c:378 [inline] printreport+0xc3/0x620 mm/kasan/report.c:489 kasanreport+0xd9/0x110 mm/kasan/report.c:602 rbmapvma+0x9ab/0xae0 kernel/trace/ringbuffer.c:7058 ringbuffermap+0x56e/0x9b0 kernel/trace/ringbuffer.c:7138 tracingbuffersmmap+0xa6/0x120 kernel/trace/trace.c:8482 callmmap include/linux/fs.h:2183 [inline] mmapfile mm/internal.h:124 [inline] mmapnewfilevma mm/vma.c:2291 [inline] mmapnewvma mm/vma.c:2355 [inline] mmapregion+0x1786/0x2670 mm/vma.c:2456 mmapregion+0x127/0x320 mm/mmap.c:1348 dommap+0xc00/0xfc0 mm/mmap.c:496 vmmmappgoff+0x1ba/0x360 mm/util.c:580 ksysmmappgoff+0x32c/0x5c0 mm/mmap.c:542 dosysmmap arch/x86/kernel/sysx8664.c:89 [inline] sesysmmap arch/x86/kernel/sysx8664.c:82 [inline] x64sysmmap+0x125/0x190 arch/x86/kernel/sysx8664.c:82 dosyscallx64 arch/x86/entry/common.c:52 [inline] dosyscall64+0xcd/0x250 arch/x86/entry/common.c:83 entrySYSCALL64afterhwframe+0x77/0x7f

The reproducer for this bug is:

------------------------8<------------------------- #include <fcntl.h> #include <stdlib.h> #include <unistd.h> #include <asm/types.h> #include <sys/mman.h>

int main(int argc, char argv) { int pagesize = getpagesize(); int fd; void meta;

system("echo 1 > /sys/kernel/tracing/buffersizekb"); fd = open("/sys/kernel/tracing/percpu/cpu0/tracepiperaw", ORDONLY);

meta = mmap(NULL, pagesize, PROTREAD, MAPSHARED, fd, pagesize 5); } ------------------------>8-------------------------