torvalds/linux
torvalds/linux
2
0
Fork 0
mirror of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2025-09-04 20:19:47 +08:00
Code

io_uring/rsrc: skip atomic refcount for uncloned buffers

Browse Source 
io_buffer_unmap() performs an atomic decrement of the io_mapped_ubuf's
reference count in case it has been cloned into another io_ring_ctx's
registered buffer table. This is an expensive operation and unnecessary
in the common case that the io_mapped_ubuf is only registered once.
Load the reference count first and check whether it's 1. In that case,
skip the atomic decrement and immediately free the io_mapped_ubuf.

Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Link: https://lore.kernel.org/r/20250619143435.3474028-1-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This commit is contained in:
Caleb Sander Mateos 2025-06-19 08:34:34 -06:00 committed by Jens Axboe
parent e448d57826
commit daa01d954b
1 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
io_uring/rsrc.c
View File

@ -135,8 +135,10 @@ static void io_free_imu(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)
static void io_buffer_unmap(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu) static void io_buffer_unmap(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)
{ {
if (!refcount_dec_and_test(&imu->refs)) if (unlikely(refcount_read(&imu->refs) > 1)) {
return; if (!refcount_dec_and_test(&imu->refs))
return;
}
if (imu->acct_pages) if (imu->acct_pages)
io_unaccount_mem(ctx, imu->acct_pages); io_unaccount_mem(ctx, imu->acct_pages);