-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
net/mlx5: DR, Keep track of hot ICM chunks in an array instead of list
When ICM chunk is freed, it might still be accessed by HW until we do sync with HW. This sync is expensive operation, so we don't do it often. Instead, when the chunk is freed, it is moved to the buddy's "hot memory" list. Once sync is done, we traverse the hot list and finally free all the chunks. It appears that traversing a long list takes unusually long time due to cache misses on many entries, which causes a big "hiccup" during rule insertion. This patch deals with this issue the following way: - Move hot chunks list from buddy to pool, so that the pool will keep track of all its hot memory. - Replace the list with pre-allocated array on the memory pool struct, and store only the information that is needed to later free this chunk in its buddy allocator. This cost additional memory for the array that is dynamically allocated, but it allows not to save long list of hot chunks, so at peak times it actually saves memory due to the fact that each array entry is much smaller than the chunk struct. This way an overhead of traversing the long list is virtually removed: the loop of freeing hot chunks takes ~27 msec instead of ~70 msec, where most of it are the actual freeing activities. Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com> Reviewed-by: Alex Vesker <valex@nvidia.com> Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
- Loading branch information
Yevgeny Kliteynik
authored and
Saeed Mahameed
committed
Oct 27, 2022
1 parent
133ea37
commit 4519fc4
Showing
4 changed files
with
71 additions
and
22 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters