[PATCH v3 0/2] ksmbd: fix named-pipe reply loss and operation races

[Wang Zhaolong]

This is v3 of the named-pipe fix series. v2 was sent earlier; v3 is
rebased onto current mainline and keeps the new mainline RPC pipe
ownership checks.

The first problem is that RPC IPC replies are matched only by the pipe
handle. READ, WRITE, FSCTL_PIPE_TRANSCEIVE and CLOSE on one pipe all use
that handle, and the session RPC lock is dropped before an IPC request
waits for its reply. There is no separate IPC correlation ID, so two
operations can overlap and interfere with reply delivery. CLOSE can also
release the handle while an earlier operation is still in flight.
Protecting only the handle table is not enough: an active pipe needs
per-pipe serialization and lifetime.

Patch 1 serializes operations per pipe, keeps the pipe state and handle
alive until all users finish, and rechecks cancellation, connection
release and session teardown after taking the pipe lock. It erases a
closed pipe before waiting for its active operation and releases the
handle id only after the last user. Session logoff now clears the RPC
pipe list after the other requests have drained. Failed pipe operations
use standard SMB2 error replies while preserving the selected status.
The existing foreign-handle checks and STATUS_FILE_CLOSED behavior added
by current mainline are retained.

The second problem is reply loss with small output buffers.
FSCTL_PIPE_TRANSCEIVE copies at most MaxOutputResponse bytes and returns
STATUS_BUFFER_OVERFLOW when the RPC reply is larger, but then frees the
whole IPC reply. The unread bytes are no longer available to a follow-up
READ, so the RPC call can fail even though userspace produced a complete
reply. READ also ignores the requested Length when copying an IPC reply.

Patch 2 builds on the per-pipe lifetime from patch 1 and keeps the
original reply allocation and an offset on the pipe. READ and TRANSCEIVE
consume only the bytes that fit and retain the rest. The retained reply
is freed only when drained, or when the pipe is closed, logged off or
disconnected. A failed response preparation or input request does not
discard output the client has not received. An IPC failure is reported
as an I/O error instead of an empty read or a successful write without a
reply. No kernel RPC queue is added; additional reply queueing remains
userspace's responsibility.

Changes in v3:
- Rebased from 21a7f44fd076 to 7d37252fc49a.
- Preserved current mainline's ksmbd_session_rpc_method() and
  STATUS_FILE_CLOSED checks for foreign RPC pipe handles. The new
  lifetime lookup maps its -ENOENT race result to STATUS_FILE_CLOSED.
- No other functional change from v2. The W=1 kernel-doc warning fix is
  retained.
- Revalidated both reports on current mainline.

Reproduction and validation on 7d37252fc49a:
- Short output: a 2632-byte NetrShareEnum reply with a 512-byte
  MaxOutputResponse returns 512 bytes; the base loses the remaining 2120
  bytes and the follow-up READ returns STATUS_UNEXPECTED_IO_ERROR. With
  the series, the follow-up READ returns the remaining 2120 bytes and the
  reassembled reply is the exact 2632-byte control reply.
- Same-pipe overlap: SIGSTOP ksmbd.mountd, send two TRANSCEIVE requests
  with different RPC call IDs, then SIGCONT. Five base runs all failed
  (three call-ID swaps, two empty bodies); five runs with the series all
  passed with each response carrying its own call ID and the full reply.
- Both tests used the same baseline ksmbd.mountd binary and the same
  16-share configuration. No test-only kernel delay patch was used.

The series was built and tested on x86-64 in an isolated Debian QEMU
guest. It is kernel-only and does not require a userspace protocol
change. It does not change the IPC timeout, quotas, on-disk format or
protocol ABI.

Bugzilla references:
  222196: short PIPE_TRANSCEIVE output loses unread RPC reply bytes
  222197: same-pipe RPC replies can reach the wrong request

Wang Zhaolong (2):
  ksmbd: avoid races between named-pipe operations
  ksmbd: fix named-pipe reply loss with small output buffers

 fs/smb/server/mgmt/user_session.c | 128 +++++++++-----
 fs/smb/server/mgmt/user_session.h |  18 ++
 fs/smb/server/smb2pdu.c           | 283 ++++++++++++++++++++----------
 fs/smb/server/transport_ipc.c     |  45 +++--
 fs/smb/server/transport_ipc.h     |  14 +-
 5 files changed, 325 insertions(+), 163 deletions(-)

base-commit: 7d37252fc49a939d005a7495b81d1bb337c29e45
-- 
2.56.0