Skip to content

Commit

Permalink
ice: Micro-optimize .ndo_xdp_xmit() path
Browse files Browse the repository at this point in the history
After the recent mbuf changes, ice_xmit_xdp_ring() became a 3-liner.
It makes no sense to keep it global in a different file than its caller.
Move it just next to the sole call site and mark static. Also, it
doesn't need a full xdp_convert_frame_to_buff(). Save several cycles
and fill only the fields used by __ice_xmit_xdp_ring() later on.
Finally, since it doesn't modify @xdpf anyhow, mark the argument const
to save some more (whole -11 bytes of .text! :D).

Thanks to 1 jump less and less calcs as well, this yields as many as
6.7 Mpps per queue. `xdp.data_hard_start = xdpf` is fully intentional
again (see xdp_convert_buff_to_frame()) and just works when there are
no source device's driver issues.

Signed-off-by: Alexander Lobakin <alexandr.lobakin@intel.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
Link: https://lore.kernel.org/bpf/20230210170618.1973430-7-alexandr.lobakin@intel.com
  • Loading branch information
Alexander Lobakin authored and Daniel Borkmann committed Feb 13, 2023
1 parent 055d092 commit ad07f29
Show file tree
Hide file tree
Showing 3 changed files with 20 additions and 15 deletions.
21 changes: 20 additions & 1 deletion drivers/net/ethernet/intel/ice/ice_txrx.c
Original file line number Diff line number Diff line change
Expand Up @@ -605,6 +605,25 @@ ice_run_xdp(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp,
ice_set_rx_bufs_act(xdp, rx_ring, ret);
}

/**
* ice_xmit_xdp_ring - submit frame to XDP ring for transmission
* @xdpf: XDP frame that will be converted to XDP buff
* @xdp_ring: XDP ring for transmission
*/
static int ice_xmit_xdp_ring(const struct xdp_frame *xdpf,
struct ice_tx_ring *xdp_ring)
{
struct xdp_buff xdp;

xdp.data_hard_start = (void *)xdpf;
xdp.data = xdpf->data;
xdp.data_end = xdp.data + xdpf->len;
xdp.frame_sz = xdpf->frame_sz;
xdp.flags = xdpf->flags;

return __ice_xmit_xdp_ring(&xdp, xdp_ring, true);
}

/**
* ice_xdp_xmit - submit packets to XDP ring for transmission
* @dev: netdev
Expand Down Expand Up @@ -650,7 +669,7 @@ ice_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames,

tx_buf = &xdp_ring->tx_buf[xdp_ring->next_to_use];
for (i = 0; i < n; i++) {
struct xdp_frame *xdpf = frames[i];
const struct xdp_frame *xdpf = frames[i];
int err;

err = ice_xmit_xdp_ring(xdpf, xdp_ring);
Expand Down
13 changes: 0 additions & 13 deletions drivers/net/ethernet/intel/ice/ice_txrx_lib.c
Original file line number Diff line number Diff line change
Expand Up @@ -434,19 +434,6 @@ int __ice_xmit_xdp_ring(struct xdp_buff *xdp, struct ice_tx_ring *xdp_ring,
return ICE_XDP_CONSUMED;
}

/**
* ice_xmit_xdp_ring - submit frame to XDP ring for transmission
* @xdpf: XDP frame that will be converted to XDP buff
* @xdp_ring: XDP ring for transmission
*/
int ice_xmit_xdp_ring(struct xdp_frame *xdpf, struct ice_tx_ring *xdp_ring)
{
struct xdp_buff xdp;

xdp_convert_frame_to_buff(xdpf, &xdp);
return __ice_xmit_xdp_ring(&xdp, xdp_ring, true);
}

/**
* ice_finalize_xdp_rx - Bump XDP Tx tail and/or flush redirect map
* @xdp_ring: XDP ring
Expand Down
1 change: 0 additions & 1 deletion drivers/net/ethernet/intel/ice/ice_txrx_lib.h
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,6 @@ static inline u32 ice_set_rs_bit(const struct ice_tx_ring *xdp_ring)

void ice_finalize_xdp_rx(struct ice_tx_ring *xdp_ring, unsigned int xdp_res, u32 first_idx);
int ice_xmit_xdp_buff(struct xdp_buff *xdp, struct ice_tx_ring *xdp_ring);
int ice_xmit_xdp_ring(struct xdp_frame *xdpf, struct ice_tx_ring *xdp_ring);
int __ice_xmit_xdp_ring(struct xdp_buff *xdp, struct ice_tx_ring *xdp_ring,
bool frame);
void ice_release_rx_desc(struct ice_rx_ring *rx_ring, u16 val);
Expand Down

0 comments on commit ad07f29

Please sign in to comment.