Skip to content

Commit

Permalink
block: extend bio caching to task context
Browse files Browse the repository at this point in the history
bio_put_percpu_cache() puts all non-iopoll bios into the irq-safe list,
which entails disabling irqs. The overhead of that is not that bad when
interrupts are already off but getting worse otherwise. We can optimise
it when we're in the task context by using ->free_list directly just as
the IOPOLL path does.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/4774e1a0f905f96c63174b0f3e4f79f0d9b63246.1707314970.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
  • Loading branch information
Pavel Begunkov authored and Jens Axboe committed Feb 8, 2024
1 parent 79ae56f commit c9f5f3a
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion block/bio.c
Original file line number Diff line number Diff line change
Expand Up @@ -770,8 +770,9 @@ static inline void bio_put_percpu_cache(struct bio *bio)

bio_uninit(bio);

if ((bio->bi_opf & REQ_POLLED) && !WARN_ON_ONCE(in_interrupt())) {
if (in_task()) {
bio->bi_next = cache->free_list;
/* Not necessary but helps not to iopoll already freed bios */
bio->bi_bdev = NULL;
cache->free_list = bio;
cache->nr++;
Expand Down

0 comments on commit c9f5f3a

Please sign in to comment.