Linux Operating System Source Code: IBM Patches

When you use fcntl locking on a filesystem that defines the lock operation (eg. GPFS) and a second thread tests a lock while the first is in the process of unlock due to the file being closed, the second thread loops with this printk from locks_conflict in the linux kernel:

    default:
        printk("locks_conflict(): impossible lock type - %d\n",
              caller_fl->fl_type);

Note that corruption of the lock list and an Oops is also possile. The unlock should complete, this printk should never occur.

When a file is closed, filp_close calls locks_remove_posix to remove locks. To remove the locks, the kernel lock is obtained and the filesystem lock operation is called with an unlock for each of the locks on the i_flock list. The problem is that the filesystem may have to wait for synchronization which implies a call to schedule() which releases the kernel lock. Once the lock is released, other threads can manipulate the list and it can become corrupted.

A secondary problem is that locks_unlock_delete doesn't allocate a file_lock structure for the unlock, but instead changes the existing lock to fl_type F_UNLCK and points to it on the filesystem call. This is assumed to be ok because the kernel lock is held by locks_remove_posix before the call is made (so no other thread will be able to observe the lock while it is in this invalid state). However, if the unlock thread waits in the filesystem, another thread may see the invalid lock (in addition to corrupting the lock list). This was obseved as a printk in locks_conflict when it detected the invalid fl_type.

Two small changes are required: locks_remove_posix must restart from the top of the i_flock list after deleting a lock from a filesystem that defined its own lock operation, and the file_lock used by locks_unlock_delete must be a COPY of the held lock (with the fl_type in the COPY changed to F_UNLCK).