Skip to content

Commit

Permalink
page_pool: Add API to update numa node
Browse files Browse the repository at this point in the history
Add page_pool_update_nid() to be called by page pool consumers when they
detect numa node changes.

It will update the page pool nid value to start allocating from the new
effective numa node.

This is to mitigate page pool allocating pages from a wrong numa node,
where the pool was originally allocated, and holding on to pages that
belong to a different numa node, which causes performance degradation.

For pages that are already being consumed and could be returned to the
pool by the consumer, in next patch we will add a check per page to avoid
recycling them back to the pool and return them to the page allocator.

Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Acked-by: Jonathan Lemon <jonathan.lemon@gmail.com>
Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
  • Loading branch information
Saeed Mahameed authored and David S. Miller committed Nov 20, 2019
1 parent 1f12177 commit bc83674
Show file tree
Hide file tree
Showing 3 changed files with 37 additions and 0 deletions.
7 changes: 7 additions & 0 deletions include/net/page_pool.h
Original file line number Diff line number Diff line change
Expand Up @@ -204,4 +204,11 @@ static inline bool page_pool_put(struct page_pool *pool)
return refcount_dec_and_test(&pool->user_cnt);
}

/* Caller must provide appropriate safe context, e.g. NAPI. */
void page_pool_update_nid(struct page_pool *pool, int new_nid);
static inline void page_pool_nid_changed(struct page_pool *pool, int new_nid)
{
if (unlikely(pool->p.nid != new_nid))
page_pool_update_nid(pool, new_nid);
}
#endif /* _NET_PAGE_POOL_H */
22 changes: 22 additions & 0 deletions include/trace/events/page_pool.h
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,28 @@ TRACE_EVENT(page_pool_state_hold,
__entry->pool, __entry->page, __entry->pfn, __entry->hold)
);

TRACE_EVENT(page_pool_update_nid,

TP_PROTO(const struct page_pool *pool, int new_nid),

TP_ARGS(pool, new_nid),

TP_STRUCT__entry(
__field(const struct page_pool *, pool)
__field(int, pool_nid)
__field(int, new_nid)
),

TP_fast_assign(
__entry->pool = pool;
__entry->pool_nid = pool->p.nid;
__entry->new_nid = new_nid;
),

TP_printk("page_pool=%p pool_nid=%d new_nid=%d",
__entry->pool, __entry->pool_nid, __entry->new_nid)
);

#endif /* _TRACE_PAGE_POOL_H */

/* This part must be outside protection */
Expand Down
8 changes: 8 additions & 0 deletions net/core/page_pool.c
Original file line number Diff line number Diff line change
Expand Up @@ -436,3 +436,11 @@ void page_pool_destroy(struct page_pool *pool)
schedule_delayed_work(&pool->release_dw, DEFER_TIME);
}
EXPORT_SYMBOL(page_pool_destroy);

/* Caller must provide appropriate safe context, e.g. NAPI. */
void page_pool_update_nid(struct page_pool *pool, int new_nid)
{
trace_page_pool_update_nid(pool, new_nid);
pool->p.nid = new_nid;
}
EXPORT_SYMBOL(page_pool_update_nid);

0 comments on commit bc83674

Please sign in to comment.