Integer overflow in the rdsrdmapages function in net/rds/rdma.c in the Linux kernel allows local users to cause a denial of service (crash) and possibly execute arbitrary code via a crafted iovec struct in a Reliable Datagram Sockets (RDS) request, which triggers a buffer overflow.

Description of problem: Discovered by Ben Hawkes, this patch adds a limit for nframes as the number of frames in TXSETUP and RXSETUP are derived from a single byte multiplex value by default. Use-cases that would require to send/filter more than 256 CAN frames should be implemented in userspace for complexity reasons anyway. Additionally the assignments of unsigned values from userspace to signed values in kernelspace and vice versa are fixed by using unsigned values in kernelspace consistently.

Upstream commit: http://git.kernel.org/linus/5b75c4973ce779520b9d1e392483207d6f842cde