Skip to content

Commit

Permalink
io_uring: optimise ltimeout for inline execution
Browse files Browse the repository at this point in the history
At one point in time we had an optimisation that would not spin up a
linked timeout timer when the master request successfully completes
inline (during the first nowait execution attempt). We somehow lost it,
so this patch restores it back.

Note, that it's fine using io_arm_ltimeout() after the io_issue_sqe()
completes the request because of delayed completion, but that that adds
unwanted overhead.

Reported-by: Christian Mazakas <christian.mazakas@gmail.com>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/8bf69c2a4beec14c565c85c86edb871ca8b8bcc8.1701390926.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
  • Loading branch information
Pavel Begunkov authored and Jens Axboe committed Dec 12, 2023
1 parent 9b43ef3 commit e0b23d9
Showing 1 changed file with 9 additions and 10 deletions.
19 changes: 9 additions & 10 deletions io_uring/io_uring.c
Original file line number Diff line number Diff line change
Expand Up @@ -1902,14 +1902,15 @@ static int io_issue_sqe(struct io_kiocb *req, unsigned int issue_flags)
return 0;
}

if (ret != IOU_ISSUE_SKIP_COMPLETE)
return ret;

/* If the op doesn't have a file, we're not polling for it */
if ((req->ctx->flags & IORING_SETUP_IOPOLL) && def->iopoll_queue)
io_iopoll_req_issued(req, issue_flags);
if (ret == IOU_ISSUE_SKIP_COMPLETE) {
ret = 0;
io_arm_ltimeout(req);

return 0;
/* If the op doesn't have a file, we're not polling for it */
if ((req->ctx->flags & IORING_SETUP_IOPOLL) && def->iopoll_queue)
io_iopoll_req_issued(req, issue_flags);
}
return ret;
}

int io_poll_issue(struct io_kiocb *req, struct io_tw_state *ts)
Expand Down Expand Up @@ -2080,9 +2081,7 @@ static inline void io_queue_sqe(struct io_kiocb *req)
* We async punt it if the file wasn't marked NOWAIT, or if the file
* doesn't support non-blocking read/write attempts
*/
if (likely(!ret))
io_arm_ltimeout(req);
else
if (unlikely(ret))
io_queue_async(req, ret);
}

Expand Down

0 comments on commit e0b23d9

Please sign in to comment.