Skip to content

Commit

Permalink
io_uring: single shot poll removal optimisation
Browse files Browse the repository at this point in the history
We don't need to poll oneshot request if we've got a desired mask in
io_poll_wake(), task_work will clean it up correctly, but as we already
hold a wq spinlock, we can remove ourselves and save on additional
spinlocking in io_poll_remove_entries().

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/ee170a344a18c9ef36b554d806c64caadfd61c31.1639605189.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
  • Loading branch information
Pavel Begunkov authored and Jens Axboe committed Dec 28, 2021
1 parent aa43477 commit eb0089d
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion fs/io_uring.c
Original file line number Diff line number Diff line change
Expand Up @@ -5568,8 +5568,14 @@ static int io_poll_wake(struct wait_queue_entry *wait, unsigned mode, int sync,
if (mask && !(mask & poll->events))
return 0;

if (io_poll_get_ownership(req))
if (io_poll_get_ownership(req)) {
/* optional, saves extra locking for removal in tw handler */
if (mask && poll->events & EPOLLONESHOT) {
list_del_init(&poll->wait.entry);
poll->head = NULL;
}
__io_poll_execute(req, mask);
}
return 1;
}

Expand Down

0 comments on commit eb0089d

Please sign in to comment.