Skip to content

Commit

Permalink
[PATCH] Notify page fault call chain for x86_64
Browse files Browse the repository at this point in the history
Currently in the do_page_fault() code path, we call notify_die(DIE_PAGE_FAULT,
...) to notify the page fault.  Since notify_die() is highly overloaded, this
page fault notification is currently being sent to all the components
registered with register_die_notification() which uses the same die_chain to
loop for all the registered components which is unnecessary.

In order to optimize the do_page_fault() code path, this critical page fault
notification is now moved to different call chain and the test results showed
great improvements.

And the kprobes which is interested in this notifications, now registers onto
this new call chain only when it need to, i.e Kprobes now registers for page
fault notification only when their are an active probes and unregisters from
this page fault notification when no probes are active.

I have incorporated all the feedback given by Ananth and Keith and everyone,
and thanks for all the review feedback.

This patch:

Overloading of page fault notification with the notify_die() has performance
issues(since the only interested components for page fault is kprobes and/or
kdb) and hence this patch introduces the new notifier call chain exclusively
for page fault notifications their by avoiding notifying unnecessary
components in the do_page_fault() code path.

Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
  • Loading branch information
Anil S Keshavamurthy authored and Linus Torvalds committed Jun 26, 2006
1 parent 3672165 commit 1bd858a
Show file tree
Hide file tree
Showing 2 changed files with 39 additions and 2 deletions.
39 changes: 37 additions & 2 deletions arch/x86_64/mm/fault.c
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,41 @@
#define PF_RSVD (1<<3)
#define PF_INSTR (1<<4)

#ifdef CONFIG_KPROBES
ATOMIC_NOTIFIER_HEAD(notify_page_fault_chain);

/* Hook to register for page fault notifications */
int register_page_fault_notifier(struct notifier_block *nb)
{
vmalloc_sync_all();
return atomic_notifier_chain_register(&notify_page_fault_chain, nb);
}

int unregister_page_fault_notifier(struct notifier_block *nb)
{
return atomic_notifier_chain_unregister(&notify_page_fault_chain, nb);
}

static inline int notify_page_fault(enum die_val val, const char *str,
struct pt_regs *regs, long err, int trap, int sig)
{
struct die_args args = {
.regs = regs,
.str = str,
.err = err,
.trapnr = trap,
.signr = sig
};
return atomic_notifier_call_chain(&notify_page_fault_chain, val, &args);
}
#else
static inline int notify_page_fault(enum die_val val, const char *str,
struct pt_regs *regs, long err, int trap, int sig)
{
return NOTIFY_DONE;
}
#endif

void bust_spinlocks(int yes)
{
int loglevel_save = console_loglevel;
Expand Down Expand Up @@ -348,7 +383,7 @@ asmlinkage void __kprobes do_page_fault(struct pt_regs *regs,
if (vmalloc_fault(address) >= 0)
return;
}
if (notify_die(DIE_PAGE_FAULT, "page fault", regs, error_code, 14,
if (notify_page_fault(DIE_PAGE_FAULT, "page fault", regs, error_code, 14,
SIGSEGV) == NOTIFY_STOP)
return;
/*
Expand All @@ -358,7 +393,7 @@ asmlinkage void __kprobes do_page_fault(struct pt_regs *regs,
goto bad_area_nosemaphore;
}

if (notify_die(DIE_PAGE_FAULT, "page fault", regs, error_code, 14,
if (notify_page_fault(DIE_PAGE_FAULT, "page fault", regs, error_code, 14,
SIGSEGV) == NOTIFY_STOP)
return;

Expand Down
2 changes: 2 additions & 0 deletions include/asm-x86_64/kdebug.h
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ struct die_args {

extern int register_die_notifier(struct notifier_block *);
extern int unregister_die_notifier(struct notifier_block *);
extern int register_page_fault_notifier(struct notifier_block *);
extern int unregister_page_fault_notifier(struct notifier_block *);
extern struct atomic_notifier_head die_chain;

/* Grossly misnamed. */
Expand Down

0 comments on commit 1bd858a

Please sign in to comment.