-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'tls-offload-netdev-and-mlx5-support'
Boris Pismenny says: ==================== TLS offload, netdev & MLX5 support The following series provides TLS TX inline crypto offload. v1->v2: - Added IS_ENABLED(CONFIG_TLS_DEVICE) and a STATIC_KEY for icsk_clean_acked - File license fix - Fix spelling, comment by DaveW - Move memory allocations out of tls_set_device_offload and other misc fixes, comments by Kiril. v2->v3: - Reversed xmas tree where needed and style fixes - Removed the need for skb_page_frag_refill, per Eric's comment - IPv6 dependency fixes v3->v4: - Remove "inline" from functions in C files - Make clean_acked_data_enabled a static variable and add enable/disable functions to control it. - Remove unnecessary variable initialization mentioned by ShannonN - Rebase over TLS RX - Refactor the tls_software_fallback to reduce the number of variables mentioned by KirilT v4->v5: - Add missing CONFIG_TLS_DEVICE v5->v6: - Move changes to the software implementation into a seperate patch - Fix some checkpatch warnings - GPL export the enable/disable clean_acked_data functions v6->v7: - Use the dst_entry to obtain the netdev in dev_get_by_index - Remove the IPv6 patch since it is redundent now v7->v8: - Fix a merge conflict in mlx5 header v8->v9: - Fix false -Wmaybe-uninitialized warning - Fix empty space in the end of new files v9->v10: - Remove default "n" in net/Kconfig This series adds a generic infrastructure to offload TLS crypto to a network devices. It enables the kernel TLS socket to skip encryption and authentication operations on the transmit side of the data path. Leaving those computationally expensive operations to the NIC. The NIC offload infrastructure builds TLS records and pushes them to the TCP layer just like the SW KTLS implementation and using the same API. TCP segmentation is mostly unaffected. Currently the only exception is that we prevent mixed SKBs where only part of the payload requires offload. In the future we are likely to add a similar restriction following a change cipher spec record. The notable differences between SW KTLS and NIC offloaded TLS implementations are as follows: 1. The offloaded implementation builds "plaintext TLS record", those records contain plaintext instead of ciphertext and place holder bytes instead of authentication tags. 2. The offloaded implementation maintains a mapping from TCP sequence number to TLS records. Thus given a TCP SKB sent from a NIC offloaded TLS socket, we can use the tls NIC offload infrastructure to obtain enough context to encrypt the payload of the SKB. A TLS record is released when the last byte of the record is ack'ed, this is done through the new icsk_clean_acked callback. The infrastructure should be extendable to support various NIC offload implementations. However it is currently written with the implementation below in mind: The NIC assumes that packets from each offloaded stream are sent as plaintext and in-order. It keeps track of the TLS records in the TCP stream. When a packet marked for offload is transmitted, the NIC encrypts the payload in-place and puts authentication tags in the relevant place holders. The responsibility for handling out-of-order packets (i.e. TCP retransmission, qdisc drops) falls on the netdev driver. The netdev driver keeps track of the expected TCP SN from the NIC's perspective. If the next packet to transmit matches the expected TCP SN, the driver advances the expected TCP SN, and transmits the packet with TLS offload indication. If the next packet to transmit does not match the expected TCP SN. The driver calls the TLS layer to obtain the TLS record that includes the TCP of the packet for transmission. Using this TLS record, the driver posts a work entry on the transmit queue to reconstruct the NIC TLS state required for the offload of the out-of-order packet. It updates the expected TCP SN accordingly and transmit the now in-order packet. The same queue is used for packet transmission and TLS context reconstruction to avoid the need for flushing the transmit queue before issuing the context reconstruction request. Expected TCP SN is accessed without a lock, under the assumption that TCP doesn't transmit SKBs from different TX queue concurrently. If packets are rerouted to a different netdevice, then a software fallback routine handles encryption. Paper: https://www.netdevconf.org/1.2/papers/netdevconf-TLS.pdf ==================== Signed-off-by: David S. Miller <davem@davemloft.net>
- Loading branch information
Showing
43 changed files
with
3,355 additions
and
191 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| /* | ||
| * Copyright (c) 2018 Mellanox Technologies. All rights reserved. | ||
| * | ||
| * This software is available to you under a choice of one of two | ||
| * licenses. You may choose to be licensed under the terms of the GNU | ||
| * General Public License (GPL) Version 2, available from the file | ||
| * COPYING in the main directory of this source tree, or the | ||
| * OpenIB.org BSD license below: | ||
| * | ||
| * Redistribution and use in source and binary forms, with or | ||
| * without modification, are permitted provided that the following | ||
| * conditions are met: | ||
| * | ||
| * - Redistributions of source code must retain the above | ||
| * copyright notice, this list of conditions and the following | ||
| * disclaimer. | ||
| * | ||
| * - Redistributions in binary form must reproduce the above | ||
| * copyright notice, this list of conditions and the following | ||
| * disclaimer in the documentation and/or other materials | ||
| * provided with the distribution. | ||
| * | ||
| * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, | ||
| * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF | ||
| * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND | ||
| * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS | ||
| * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN | ||
| * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN | ||
| * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| * SOFTWARE. | ||
| * | ||
| */ | ||
|
|
||
| #include <linux/mlx5/device.h> | ||
|
|
||
| #include "accel/tls.h" | ||
| #include "mlx5_core.h" | ||
| #include "fpga/tls.h" | ||
|
|
||
| int mlx5_accel_tls_add_tx_flow(struct mlx5_core_dev *mdev, void *flow, | ||
| struct tls_crypto_info *crypto_info, | ||
| u32 start_offload_tcp_sn, u32 *p_swid) | ||
| { | ||
| return mlx5_fpga_tls_add_tx_flow(mdev, flow, crypto_info, | ||
| start_offload_tcp_sn, p_swid); | ||
| } | ||
|
|
||
| void mlx5_accel_tls_del_tx_flow(struct mlx5_core_dev *mdev, u32 swid) | ||
| { | ||
| mlx5_fpga_tls_del_tx_flow(mdev, swid, GFP_KERNEL); | ||
| } | ||
|
|
||
| bool mlx5_accel_is_tls_device(struct mlx5_core_dev *mdev) | ||
| { | ||
| return mlx5_fpga_is_tls_device(mdev); | ||
| } | ||
|
|
||
| u32 mlx5_accel_tls_device_caps(struct mlx5_core_dev *mdev) | ||
| { | ||
| return mlx5_fpga_tls_device_caps(mdev); | ||
| } | ||
|
|
||
| int mlx5_accel_tls_init(struct mlx5_core_dev *mdev) | ||
| { | ||
| return mlx5_fpga_tls_init(mdev); | ||
| } | ||
|
|
||
| void mlx5_accel_tls_cleanup(struct mlx5_core_dev *mdev) | ||
| { | ||
| mlx5_fpga_tls_cleanup(mdev); | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| /* | ||
| * Copyright (c) 2018 Mellanox Technologies. All rights reserved. | ||
| * | ||
| * This software is available to you under a choice of one of two | ||
| * licenses. You may choose to be licensed under the terms of the GNU | ||
| * General Public License (GPL) Version 2, available from the file | ||
| * COPYING in the main directory of this source tree, or the | ||
| * OpenIB.org BSD license below: | ||
| * | ||
| * Redistribution and use in source and binary forms, with or | ||
| * without modification, are permitted provided that the following | ||
| * conditions are met: | ||
| * | ||
| * - Redistributions of source code must retain the above | ||
| * copyright notice, this list of conditions and the following | ||
| * disclaimer. | ||
| * | ||
| * - Redistributions in binary form must reproduce the above | ||
| * copyright notice, this list of conditions and the following | ||
| * disclaimer in the documentation and/or other materials | ||
| * provided with the distribution. | ||
| * | ||
| * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, | ||
| * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF | ||
| * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND | ||
| * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS | ||
| * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN | ||
| * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN | ||
| * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| * SOFTWARE. | ||
| * | ||
| */ | ||
|
|
||
| #ifndef __MLX5_ACCEL_TLS_H__ | ||
| #define __MLX5_ACCEL_TLS_H__ | ||
|
|
||
| #include <linux/mlx5/driver.h> | ||
| #include <linux/tls.h> | ||
|
|
||
| #ifdef CONFIG_MLX5_ACCEL | ||
|
|
||
| enum { | ||
| MLX5_ACCEL_TLS_TX = BIT(0), | ||
| MLX5_ACCEL_TLS_RX = BIT(1), | ||
| MLX5_ACCEL_TLS_V12 = BIT(2), | ||
| MLX5_ACCEL_TLS_V13 = BIT(3), | ||
| MLX5_ACCEL_TLS_LRO = BIT(4), | ||
| MLX5_ACCEL_TLS_IPV6 = BIT(5), | ||
| MLX5_ACCEL_TLS_AES_GCM128 = BIT(30), | ||
| MLX5_ACCEL_TLS_AES_GCM256 = BIT(31), | ||
| }; | ||
|
|
||
| struct mlx5_ifc_tls_flow_bits { | ||
| u8 src_port[0x10]; | ||
| u8 dst_port[0x10]; | ||
| union mlx5_ifc_ipv6_layout_ipv4_layout_auto_bits src_ipv4_src_ipv6; | ||
| union mlx5_ifc_ipv6_layout_ipv4_layout_auto_bits dst_ipv4_dst_ipv6; | ||
| u8 ipv6[0x1]; | ||
| u8 direction_sx[0x1]; | ||
| u8 reserved_at_2[0x1e]; | ||
| }; | ||
|
|
||
| int mlx5_accel_tls_add_tx_flow(struct mlx5_core_dev *mdev, void *flow, | ||
| struct tls_crypto_info *crypto_info, | ||
| u32 start_offload_tcp_sn, u32 *p_swid); | ||
| void mlx5_accel_tls_del_tx_flow(struct mlx5_core_dev *mdev, u32 swid); | ||
| bool mlx5_accel_is_tls_device(struct mlx5_core_dev *mdev); | ||
| u32 mlx5_accel_tls_device_caps(struct mlx5_core_dev *mdev); | ||
| int mlx5_accel_tls_init(struct mlx5_core_dev *mdev); | ||
| void mlx5_accel_tls_cleanup(struct mlx5_core_dev *mdev); | ||
|
|
||
| #else | ||
|
|
||
| static inline int | ||
| mlx5_accel_tls_add_tx_flow(struct mlx5_core_dev *mdev, void *flow, | ||
| struct tls_crypto_info *crypto_info, | ||
| u32 start_offload_tcp_sn, u32 *p_swid) { return 0; } | ||
| static inline void mlx5_accel_tls_del_tx_flow(struct mlx5_core_dev *mdev, u32 swid) { } | ||
| static inline bool mlx5_accel_is_tls_device(struct mlx5_core_dev *mdev) { return false; } | ||
| static inline u32 mlx5_accel_tls_device_caps(struct mlx5_core_dev *mdev) { return 0; } | ||
| static inline int mlx5_accel_tls_init(struct mlx5_core_dev *mdev) { return 0; } | ||
| static inline void mlx5_accel_tls_cleanup(struct mlx5_core_dev *mdev) { } | ||
|
|
||
| #endif | ||
|
|
||
| #endif /* __MLX5_ACCEL_TLS_H__ */ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
72 changes: 72 additions & 0 deletions
72
drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| /* | ||
| * Copyright (c) 2018 Mellanox Technologies. All rights reserved. | ||
| * | ||
| * This software is available to you under a choice of one of two | ||
| * licenses. You may choose to be licensed under the terms of the GNU | ||
| * General Public License (GPL) Version 2, available from the file | ||
| * COPYING in the main directory of this source tree, or the | ||
| * OpenIB.org BSD license below: | ||
| * | ||
| * Redistribution and use in source and binary forms, with or | ||
| * without modification, are permitted provided that the following | ||
| * conditions are met: | ||
| * | ||
| * - Redistributions of source code must retain the above | ||
| * copyright notice, this list of conditions and the following | ||
| * disclaimer. | ||
| * | ||
| * - Redistributions in binary form must reproduce the above | ||
| * copyright notice, this list of conditions and the following | ||
| * disclaimer in the documentation and/or other materials | ||
| * provided with the distribution. | ||
| * | ||
| * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, | ||
| * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF | ||
| * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND | ||
| * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS | ||
| * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN | ||
| * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN | ||
| * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| * SOFTWARE. | ||
| * | ||
| */ | ||
|
|
||
| #ifndef __MLX5E_EN_ACCEL_H__ | ||
| #define __MLX5E_EN_ACCEL_H__ | ||
|
|
||
| #ifdef CONFIG_MLX5_ACCEL | ||
|
|
||
| #include <linux/skbuff.h> | ||
| #include <linux/netdevice.h> | ||
| #include "en_accel/ipsec_rxtx.h" | ||
| #include "en_accel/tls_rxtx.h" | ||
| #include "en.h" | ||
|
|
||
| static inline struct sk_buff *mlx5e_accel_handle_tx(struct sk_buff *skb, | ||
| struct mlx5e_txqsq *sq, | ||
| struct net_device *dev, | ||
| struct mlx5e_tx_wqe **wqe, | ||
| u16 *pi) | ||
| { | ||
| #ifdef CONFIG_MLX5_EN_TLS | ||
| if (sq->state & BIT(MLX5E_SQ_STATE_TLS)) { | ||
| skb = mlx5e_tls_handle_tx_skb(dev, sq, skb, wqe, pi); | ||
| if (unlikely(!skb)) | ||
| return NULL; | ||
| } | ||
| #endif | ||
|
|
||
| #ifdef CONFIG_MLX5_EN_IPSEC | ||
| if (sq->state & BIT(MLX5E_SQ_STATE_IPSEC)) { | ||
| skb = mlx5e_ipsec_handle_tx_skb(dev, *wqe, skb); | ||
| if (unlikely(!skb)) | ||
| return NULL; | ||
| } | ||
| #endif | ||
|
|
||
| return skb; | ||
| } | ||
|
|
||
| #endif /* CONFIG_MLX5_ACCEL */ | ||
|
|
||
| #endif /* __MLX5E_EN_ACCEL_H__ */ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.