[PATCH v4] block: avoid long sync_blockdev() under disk->open_mutex during parallel close

[Tetsuo Handa]

Commit b849dd84b6cc ("bdev: Reduce time holding bd_mutex in sync in
blkdev_close()") introduced a sync_blockdev() call immediately before
holding disk->open_mutex. But this check was racy and best-effort basis.

If more than one process concurrently called bdev_release() and saw
atomic_read(&bdev->bd_openers) > 1, the process which actually hit
the atomic_dec_and_test(&bdev->bd_openers) condition will call
sync_blockdev() from blkdev_flush_mapping() (which could take several
minutes) with disk->open_mutex held.

Kill this race window and make sync_blockdev() always be called before
holding disk->open_mutex, by using a marker that checks whether lockless
sync_blockdev() has raced.

Signed-off-by: Tetsuo Handa <penguin-kernel@xxxxxxxxxxxxxxxxxxx>
---
 block/bdev.c              | 37 ++++++++++++++++++++++++++++++++++---
 include/linux/blk_types.h |  1 +
 2 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/block/bdev.c b/block/bdev.c
index cd8323083740..57f0f57da8c1 100644
--- a/block/bdev.c
+++ b/block/bdev.c
@@ -1167,10 +1167,41 @@ void bdev_release(struct file *bdev_file)
 	 * of the world and we want to avoid long (could be several minute)
 	 * syncs while holding the mutex.
 	 */
-	if (atomic_read(&bdev->bd_openers) == 1)
-		sync_blockdev(bdev);
-
 	mutex_lock(&disk->open_mutex);
+ retry:
+	if (bdev_is_partition(bdev)) {
+		if (atomic_read(&bdev->bd_openers) == 1) {
+			bdev->bdev_syncer = current;
+			mutex_unlock(&disk->open_mutex);
+			sync_blockdev(bdev);
+			/*
+			 * Always sync the whole disk despite we need to sync
+			 * only when we're the last one, for we can't serialize
+			 * concurrent closing of multiple partitions on the
+			 * same disk using open_mutex.
+			 */
+			sync_blockdev(bdev_whole(bdev));
+			mutex_lock(&disk->open_mutex);
+			if (bdev->bdev_syncer != current)
+				goto retry;
+		} else {
+			/* Mark sync_blockdev() being possibly raced. */
+			bdev->bdev_syncer = NULL;
+		}
+	} else {
+		if (atomic_read(&bdev->bd_openers) == 1) {
+			bdev->bdev_syncer = current;
+			mutex_unlock(&disk->open_mutex);
+			sync_blockdev(bdev);
+			mutex_lock(&disk->open_mutex);
+			if (bdev->bdev_syncer != current)
+				goto retry;
+		} else {
+			/* Mark sync_blockdev() being possibly raced. */
+			bdev->bdev_syncer = NULL;
+		}
+	}
+
 	bdev_yield_write_access(bdev_file);
 
 	if (holder)
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 98e21b4cbf32..d79bb4b6100f 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -74,6 +74,7 @@ struct block_device {
 #ifdef CONFIG_SECURITY
 	void			*bd_security;
 #endif
+	struct task_struct	*bdev_syncer; /* Not refcounted. */
 	/*
 	 * keep this out-of-line as it's both big and not needed in the fast
 	 * path
-- 
2.56.0