Description of problem: The problem was in the way the gfs2 directory code was trying to re-use sentinel directory entries.
In the failing case, gfs2's rename function was renaming a file to another name that had the same non-trivial length. The file being renamed happened to be the first directory entry on the leaf block.
First, the rename code (gfs2rename in opsinode.c) found the original directory entry and decided it could do its job by simply replacing the directory entry with another. Therefore it determined correctly that no block allocations were needed.
Next, the rename code deleted the old directory entry prior to replacing it with the new name. Therefore, the soon-to-be replaced directory entry was temporarily made into a directory entry "sentinel" or a place holder at the start of a leaf block.
Lastly, it went to re-add the replacement directory entry in that leaf block. However, when gfs2direntfindspace was looking for space in the leaf block, it used the wrong value for the sentinel. That threw off its calculations so later it decides it can't really re-use the sentinel and therefore must allocate a new leaf block. But because it previously decided to re-use the directory entry, it didn't waste the time to grab a new block allocation for the inode. Therefore, the inode's ialloc pointer was still NULL and it crashes trying to reference it.
In the case of sentinel directory entries, the entire dirent is reused, not just the "free space" portion of it, and therefore the function gfs2direntfindspace should use the value 0 rather than GFS2DIRENTSIZE(0) for the actual dirent size.
Fixing this calculation enables the reproducer programs to work properly.
Description of problem: Reported by Toshiyuki Okajima. With linux-2.6.34-rc5, findkeyringbyname() can gain the keyring which has been already freed. And then, its space (which is gained by findkeyringbyname()) is broken by accessing the freed keyring as the available keyring:
1) If the space of the freed keyring is reallocated for other purpose (ie. filp SLUB), the data of the filp object may be destroyed by the user of the freed keyring. (SLUB configuration can share the freed space with other same-size slabs.)
2) If the slab space of the freed keyring is released into the system, the system panic may happen because accessing the space of the freed keyring causes the page-fault.
Example: (we can easily confirm this problem if CONFIGSLUB is "y".) [1] with CONFIGSLUBDEBUGON While we are executing my reproducer (which is attached), we can notice that the user of the freed keyring breaks its space: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ./reproducer & ... dmesg ... ============================================================================= BUG keyjar: Poison overwritten -----------------------------------------------------------------------------
INFO: 0xffff880197a7e200-0xffff880197a7e200. First byte 0x6a instead of 0x6b INFO: Allocated in keyalloc+0x10b/0x35f age=25 cpu=1 pid=5086 INFO: Freed in keycleanup+0xd0/0xd5 age=12 cpu=1 pid=10 INFO: Slab 0xffffea000592cb90 objects=16 used=2 fp=0xffff880197a7e200 flags=0x200000000000c3 INFO: Object 0xffff880197a7e200 @offset=512 fp=0xffff880197a7e300
Bytes b4 0xffff880197a7e1f0: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ Object 0xffff880197a7e200: 6a 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b jkkkkkkkkkkkkkkk Object 0xffff880197a7e210: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk Object 0xffff880197a7e220: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk Object 0xffff880197a7e230: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk Object 0xffff880197a7e240: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk Object 0xffff880197a7e250: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk Object 0xffff880197a7e260: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk Object 0xffff880197a7e270: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk Object 0xffff880197a7e280: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk Object 0xffff880197a7e290: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk� Redzone 0xffff880197a7e298: bb bb bb bb bb bb bb bb �������� Padding 0xffff880197a7e2d8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ Padding 0xffff880197a7e2e8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ Padding 0xffff880197a7e2f8: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ Pid: 5090, comm: su Not tainted 2.6.34-rc5-nofixed #1 Call Trace: [<ffffffff810e3e9c>] ? checkbytesandreport+0xb5/0xe6 [<ffffffff81152994>] ? keyalloc+0x10b/0x35f [<ffffffff810e3f8f>] ? checkobject+0xc2/0x1a4 [<ffffffff810e5c09>] ? slaballoc+0x2ff/0x3a6 [<ffffffff81152994>] ? keyalloc+0x10b/0x35f [<ffffffff81152780>] ? keyuserlookup+0x78/0x181 [<ffffffff81152994>] ? keyalloc+0x10b/0x35f [<ffffffff810e658a>] ? kmemcachealloc+0x75/0xe9 [<ffffffff81152994>] ? keyalloc+0x10b/0x35f [<ffffffff8115360a>] ? keyringalloc+0x29/0x61 [<ffffffff81155897>] ? installuserkeyrings+0x99/0x168 [<ffffffff81155ae6>] ? lookupuserkey+0x180/0x30a [<ffffffff81153eaf>] ? keyctlgetkeyringID+0x15/0x38 [<ffffffff81008a02>] ? systemcallfastpath+0x16/0x1b FIX keyjar: Restoring 0xffff880197a7e200-0xffff880197a7e200=0x6b
FIX keyjar: Marking all objects used ============================================================================= BUG keyjar: Poison overwritten ----------------------------------------------------------------------------- ... - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[2] without CONFIGSLUBDEBUGON While we are executing my reproducer (which is attached), we may find such the back-trace at the system panic: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ls -l /sys/kernel/slab total 0 drwxr-xr-x 2 root root 0 Apr 21 00:45 :at-0000016 ... lrwxrwxrwx 1 root root 0 Apr 21 00:45 credjar -> :t-0000192 ... lrwxrwxrwx 1 root root 0 Apr 21 00:45 filp -> :t-0000192 ... lrwxrwxrwx 1 root root 0 Apr 21 00:45 keyjar -> :t-0000192 ... lrwxrwxrwx 1 root root 0 Apr 21 00:45 kmalloc-192 -> :t-0000192 ...
./reproducer.sh (PANIC) <1>BUG: unable to handle kernel NULL pointer dereference at 0000000000000001 <1>IP: [<ffffffff810e61a3>] kmemcachealloc+0x5b/0xe9 <4>PGD 6b2b4067 PUD 6a80d067 PMD 0 <0>Oops: 0000 [#1] SMP <0>last sysfs file: /sys/kernel/kexeccrashloaded <4>CPU 1 <4>Modules linked in: nfsd lockd nfsacl authrpcgss exportfs sunrpc autofs4 ipv6 ext4 jbd2 dmmirror dmregionhash dmlog dmmul tipath uinput shpchp i2ci801 ppdev parportpc e752xedac edaccore i2ccore tg3 iTCOwdt iTCOvendorsupport parport pcspkr ext3 jbd mbcache atageneric pataacpi megaraidmbox megaraidmm atapiix floppy dmmod [last unloaded: scsiwaitscan] <4>Pid: 31245, comm: su Not tainted 2.6.34-rc5-nofixed-nodebug #2 D2089/PRIMERGY <4>RIP: 0010:[<ffffffff810e61a3>] [<ffffffff810e61a3>] kmemcachealloc+0x5b/0xe9 <4>RSP: 0018:ffff88006af3bd98 EFLAGS: 00010002 <4>RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffff88007d19900b <4>RDX: 0000000100000000 RSI: 00000000000080d0 RDI: ffffffff81828430 <4>RBP: ffffffff81828430 R08: ffff88000a293750 R09: 0000000000000000 <4>R10: 0000000000000001 R11: 0000000000100000 R12: 00000000000080d0 <4>R13: 00000000000080d0 R14: 0000000000000296 R15: ffffffff810f20ce <4>FS: 00007f97116bc700(0000) GS:ffff88000a280000(0000) knlGS:0000000000000000 <4>CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4>CR2: 0000000000000001 CR3: 000000006a91c000 CR4: 00000000000006e0 <4>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 <4>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 <4>Process su (pid: 31245, threadinfo ffff88006af3a000, task ffff8800374414c0) <0>Stack: <4> 0000000512e0958e 0000000000008000 ffff880037f8d180 0000000000000001 <4><0> 0000000000000000 0000000000008001 ffff88007d199000 ffffffff810f20ce <4><0> 0000000000008000 ffff88006af3be48 0000000000000024 ffffffff810face3 <0>Call Trace: <4> [<ffffffff810f20ce>] ? getemptyfilp+0x70/0x12f <4> [<ffffffff810face3>] ? dofilpopen+0x145/0x590 <4> [<ffffffff810ce208>] ? tlbfinishmmu+0x2a/0x33 <4> [<ffffffff810ce43c>] ? unmapregion+0xd3/0xe2 <4> [<ffffffff810e4393>] ? virttoheadpage+0x9/0x2d <4> [<ffffffff81103916>] ? allocfd+0x69/0x10e <4> [<ffffffff810ef4ed>] ? dosysopen+0x56/0xfc <4> [<ffffffff81008a02>] ? systemcallfastpath+0x16/0x1b <0>Code: 0f 1f 44 00 00 49 89 c6 fa 66 0f 1f 44 00 00 65 4c 8b 04 25 60 e8 00 00 48 8b 45 00 49 01 c0 49 8b 18 48 85 db 74 0d 48 63 45 18 <48> 8b 04 03 49 89 00 eb 14 4c 89 f9 83 ca ff 44 89 e6 48 89 ef <1>RIP [<ffffffff810e61a3>] kmemcachealloc+0x5b/0xe9 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[reproducer.sh] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - #!/bin/sh
LOOP=100000 USER=dummyuser add /bin/su -c "exit;" $USER || { /usr/sbin/adduser -m $USER; add=1; }
for ((i=0; i<LOOP; i++)) do /bin/su -c "echo '$i' > /dev/null" $USER done
del (( add == 1 )) && /usr/sbin/userdel -r $USER exit - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
I created the patch which fixes this problem. (I attach it into the following mail. And I couldn't see the "Poison" lines of dmesg with the fixed kernel while I was executing my reproducer. So, this problem seems to be fixed by this patch.) This problem is serious because it may trigger the user data destructions.