[PATCH] pNFS: avoid repeatedly cancelling I/O for the same lseg

[Tim Menninger]

Commit b739a5bd9d9f ("NFSv4/flexfiles: Cancel I/O if the layout is
recalled or revoked") added I/O cancellation when an lseg is
invalidated but still has outstanding references.

If references remain after mark_lseg_invalid() clears NFS_LSEG_VALID,
the lseg stays on the layout's segment list until they drain. A later
scan can therefore find the same invalid lseg and call
pnfs_lseg_cancel_io() again, even though no new invalidation occurred.

For flexfiles, cancel_io() cancels matching RPC tasks and disconnects
the associated data server RPC clients. With RPC/RDMA, repeatedly
cancelling an invalid lseg during data server recovery can drive the
client into a reconnect loop that consumes effectively all CPU and
prevents forward progress.

Once an lseg is invalid it cannot be selected for new I/O, so the
cancel_io() callback only needs to be invoked once for that lseg.

Record whether cancellation has already been requested in the lseg
flags, and suppress subsequent calls to the layout driver's cancel_io()
callback.

This does not eliminate the RPC/RDMA reconnect activity during data
server recovery, but prevents repeated cancellation from amplifying it
into a client livelock.

Fixes: b739a5bd9d9f ("NFSv4/flexfiles: Cancel I/O if the layout is recalled or revoked")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Tim Menninger <tmenninger@xxxxxxxxxxxxxxxx>
---
 fs/nfs/pnfs.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index 70d20f779678..1266420bf491 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -44,6 +44,7 @@ enum {
 	NFS_LSEG_LAYOUTCOMMIT,	/* layoutcommit bit set for layoutcommit */
 	NFS_LSEG_LAYOUTRETURN,	/* layoutreturn bit set for layoutreturn */
 	NFS_LSEG_UNAVAILABLE,	/* unavailable bit set for temporary problem */
+	NFS_LSEG_IO_CANCELLED,	/* IO cancelled on behalf of this lseg */
 };
 
 /* Individual ip address */
@@ -692,7 +693,8 @@ pnfs_lseg_request_intersecting(struct pnfs_layout_segment *lseg, struct nfs_page
 static inline void pnfs_lseg_cancel_io(struct nfs_server *server,
 				       struct pnfs_layout_segment *lseg)
 {
-	if (server->pnfs_curr_ld->cancel_io)
+	if (server->pnfs_curr_ld->cancel_io &&
+	    !test_and_set_bit(NFS_LSEG_IO_CANCELLED, &lseg->pls_flags))
 		server->pnfs_curr_ld->cancel_io(lseg);
 }
 
-- 
2.34.1