Skip to content

Commit

Permalink
bnx2x: uses build_skb() in receive path
Browse files Browse the repository at this point in the history
bnx2x uses following formula to compute its rx_buf_sz :

dev->mtu + 2*L1_CACHE_BYTES + 14 + 8 + 8 + 2

Then core network adds NET_SKB_PAD and SKB_DATA_ALIGN(sizeof(struct
skb_shared_info))

Final allocated size for skb head on x86_64 (L1_CACHE_BYTES = 64,
MTU=1500) : 2112 bytes : SLUB/SLAB round this to 4096 bytes.

Since skb truesize is then bigger than SK_MEM_QUANTUM, we have lot of
false sharing because of mem_reclaim in UDP stack.

One possible way to half truesize is to reduce the need by 64 bytes
(2112 -> 2048 bytes)

Instead of allocating a full cache line at the end of packet for
alignment, we can use the fact that skb_shared_info sits at the end of
skb->head, and we can use this room, if we convert bnx2x to new
build_skb() infrastructure.

skb_shared_info will be initialized after hardware finished its
transfert, so we can eventually overwrite the final padding.

Using build_skb() also reduces cache line misses in the driver, since we
use cache hot skb instead of cold ones. Number of in-flight sk_buff
structures is lower, they are recycled while still hot.

Performance results :

(820.000 pps on a rx UDP monothread benchmark, instead of 720.000 pps)

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
CC: Eilon Greenstein <eilong@broadcom.com>
CC: Ben Hutchings <bhutchings@solarflare.com>
CC: Tom Herbert <therbert@google.com>
CC: Jamal Hadi Salim <hadi@mojatatu.com>
CC: Stephen Hemminger <shemminger@vyatta.com>
CC: Thomas Graf <tgraf@infradead.org>
CC: Herbert Xu <herbert@gondor.apana.org.au>
CC: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Acked-by: Eilon Greenstein <eilong@broadcom.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
  • Loading branch information
Eric Dumazet authored and David S. Miller committed Nov 14, 2011
1 parent b2b5ce9 commit e52fcb2
Show file tree
Hide file tree
Showing 5 changed files with 171 additions and 169 deletions.
30 changes: 23 additions & 7 deletions drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
Original file line number Diff line number Diff line change
Expand Up @@ -293,8 +293,13 @@ enum {
#define FCOE_TXQ_IDX(bp) (MAX_ETH_TXQ_IDX(bp))

/* fast path */
/*
* This driver uses new build_skb() API :
* RX ring buffer contains pointer to kmalloc() data only,
* skb are built only after Hardware filled the frame.
*/
struct sw_rx_bd {
struct sk_buff *skb;
u8 *data;
DEFINE_DMA_UNMAP_ADDR(mapping);
};

Expand Down Expand Up @@ -424,8 +429,8 @@ union host_hc_status_block {

struct bnx2x_agg_info {
/*
* First aggregation buffer is an skb, the following - are pages.
* We will preallocate the skbs for each aggregation when
* First aggregation buffer is a data buffer, the following - are pages.
* We will preallocate the data buffer for each aggregation when
* we open the interface and will replace the BD at the consumer
* with this one when we receive the TPA_START CQE in order to
* keep the Rx BD ring consistent.
Expand All @@ -439,6 +444,7 @@ struct bnx2x_agg_info {
u16 parsing_flags;
u16 vlan_tag;
u16 len_on_bd;
u32 rxhash;
};

#define Q_STATS_OFFSET32(stat_name) \
Expand Down Expand Up @@ -1187,10 +1193,20 @@ struct bnx2x {
#define ETH_MAX_JUMBO_PACKET_SIZE 9600

/* Max supported alignment is 256 (8 shift) */
#define BNX2X_RX_ALIGN_SHIFT ((L1_CACHE_SHIFT < 8) ? \
L1_CACHE_SHIFT : 8)
/* FW use 2 Cache lines Alignment for start packet and size */
#define BNX2X_FW_RX_ALIGN (2 << BNX2X_RX_ALIGN_SHIFT)
#define BNX2X_RX_ALIGN_SHIFT min(8, L1_CACHE_SHIFT)

/* FW uses 2 Cache lines Alignment for start packet and size
*
* We assume skb_build() uses sizeof(struct skb_shared_info) bytes
* at the end of skb->data, to avoid wasting a full cache line.
* This reduces memory use (skb->truesize).
*/
#define BNX2X_FW_RX_ALIGN_START (1UL << BNX2X_RX_ALIGN_SHIFT)

#define BNX2X_FW_RX_ALIGN_END \
max(1UL << BNX2X_RX_ALIGN_SHIFT, \
SKB_DATA_ALIGN(sizeof(struct skb_shared_info)))

#define BNX2X_PXP_DRAM_ALIGN (BNX2X_RX_ALIGN_SHIFT - 5)

struct host_sp_status_block *def_status_blk;
Expand Down
Loading

0 comments on commit e52fcb2

Please sign in to comment.