CVE-2010-2798: Null Pointer Dereference

Description of problem: The problem was in the way the gfs2 directory code was trying to re-use sentinel directory entries.

In the failing case, gfs2's rename function was renaming a file to another name that had the same non-trivial length. The file being renamed happened to be the first directory entry on the leaf block.

First, the rename code (gfs2rename in opsinode.c) found the original directory entry and decided it could do its job by simply replacing the directory entry with another. Therefore it determined correctly that no block allocations were needed.

Next, the rename code deleted the old directory entry prior to replacing it with the new name. Therefore, the soon-to-be replaced directory entry was temporarily made into a directory entry "sentinel" or a place holder at the start of a leaf block.

Lastly, it went to re-add the replacement directory entry in that leaf block. However, when gfs2direntfindspace was looking for space in the leaf block, it used the wrong value for the sentinel. That threw off its calculations so later it decides it can't really re-use the sentinel and therefore must allocate a new leaf block. But because it previously decided to re-use the directory entry, it didn't waste the time to grab a new block allocation for the inode. Therefore, the inode's ialloc pointer was still NULL and it crashes trying to reference it.

In the case of sentinel directory entries, the entire dirent is reused, not just the "free space" portion of it, and therefore the function gfs2direntfindspace should use the value 0 rather than GFS2DIRENTSIZE(0) for the actual dirent size.

Fixing this calculation enables the reproducer programs to work properly.