Skip to content

Commit

Permalink
nvme-pci: fix "slimmer CQ head update"
Browse files Browse the repository at this point in the history
Pre-incrementing ->cq_head can't be done in memory because OOB value
can be observed by another context.

This devalues space savings compared to original code :-\

	$ ./scripts/bloat-o-meter ../vmlinux-000 ../obj/vmlinux
	add/remove: 0/0 grow/shrink: 0/4 up/down: 0/-32 (-32)
	Function                                     old     new   delta
	nvme_poll_irqdisable                         464     456      -8
	nvme_poll                                    455     447      -8
	nvme_irq                                     388     380      -8
	nvme_dev_disable                             955     947      -8

But the code is minimal now: one read for head, one read for q_depth,
one increment, one comparison, single instruction phase bit update and
one write for new head.

Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
Reported-by: John Garry <john.garry@huawei.com>
Tested-by: John Garry <john.garry@huawei.com>
Fixes: e2a366a ("nvme-pci: slimmer CQ head update")
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
  • Loading branch information
Alexey Dobriyan authored and Jens Axboe committed May 9, 2020
1 parent 6bd87ee commit a8de663
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion drivers/nvme/host/pci.c
Original file line number Diff line number Diff line change
Expand Up @@ -973,9 +973,13 @@ static inline void nvme_handle_cqe(struct nvme_queue *nvmeq, u16 idx)

static inline void nvme_update_cq_head(struct nvme_queue *nvmeq)
{
if (++nvmeq->cq_head == nvmeq->q_depth) {
u16 tmp = nvmeq->cq_head + 1;

if (tmp == nvmeq->q_depth) {
nvmeq->cq_head = 0;
nvmeq->cq_phase ^= 1;
} else {
nvmeq->cq_head = tmp;
}
}

Expand Down

0 comments on commit a8de663

Please sign in to comment.