Skip to content

Commit

Permalink
x86: do the user address masking outside the user access area
Browse files Browse the repository at this point in the history
In any normal situation this really shouldn't matter, but in case the
address passed in to masked_user_access_begin() were to be some complex
expression, we should evaluate it fully before doing the 'stac'
instruction.

And even without that issue (which objdump would pick up on for any
really bad case), just in general we should strive to minimize the
amount of code we run with user accesses enabled.

For example, even for the trivial pselect6() case, the code generation
(obviously with a non-debug build) just diff with this ends up being

  -	stac
 	mov    %rax,%rcx
 	sar    $0x3f,%rcx
 	or     %rax,%rcx
  +	stac
 	mov    (%rcx),%r13
 	mov    0x8(%rcx),%r14
 	clac

so the area delimeted by the 'stac / clac' pair is now literally just
the two user access instructions, and the address generation has been
moved out to before that code.

This will be much more noticeable if we end up deciding that we can go
back to just inlining "get_user()" using the new masked user access
model.  The get_user() pointers can often be more complex expressions
involving kernel memory accesses or even function calls.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
  • Loading branch information
Linus Torvalds committed Aug 19, 2024
1 parent 2865baf commit 05f4216
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion arch/x86/include/asm/uaccess_64.h
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,9 @@ static inline unsigned long __untagged_addr_remote(struct mm_struct *mm,
* for dense accesses starting at the address.
*/
#define mask_user_address(x) ((typeof(x))((long)(x)|((long)(x)>>63)))
#define masked_user_access_begin(x) ({ __uaccess_begin(); mask_user_address(x); })
#define masked_user_access_begin(x) ({ \
__auto_type __masked_ptr = mask_user_address(x); \
__uaccess_begin(); __masked_ptr; })

/*
* User pointers can have tag bits on x86-64. This scheme tolerates
Expand Down

0 comments on commit 05f4216

Please sign in to comment.