【dpdk】IntelQAT技术

参考链接

Intel® QuickAssist (QAT) Crypto Poll Mode Driver — Data Plane Development Kit 23.11.0 documentation (dpdk.org)

Intel® QuickAssist (QAT) Compression Poll Mode Driver — Data Plane Development Kit 23.11.0 documentation (dpdk.org)

QuickAssist Technology (intel.com)

Getting-started-guide

面向客户Intel® QuickAssist Technology (Intel® QAT) Software for Linux*

Data Compression with Intel® ISA-L/ Intel® IPP/ Intel® QAT Tuning…

1.QAT是什么？

Intel® QuickAssist Technology是Intel®公司提供的一种高性能数据安全和压缩的加速方案。该方案利用QAT芯片分担对称/非对称加密计算,DEFLATE无损压缩等大计算量的任务，来降低CPU使用率并提高整体平台性能。该方案可以主板芯片，独立的PCI-E加速卡或者SOC三种方式部署。

QAT支持硬件加速Deflate无损压缩算法，在处理海量数据时，QAT在不增加CPU开销的前提下，通过压缩来减少需要传输和存盘的数据量，从而减少了网络带宽和磁盘读写的开销，最终提高了整体的系统性能。 例如，在Web Serer上使用QAT硬件加速压缩处理，可将CPU从繁重的压缩计算中解放出来，以处理更多的连接请求。

2.术语

术语	解释	link
PMD	Poll Mode Driver)	14. Poll Mode Driver — Data Plane Development Kit 23.11.0 documentation (dpdk.org)

3.Architecture

Because the hardware is accessed via the Intel® QAT APIs, it is not necessary to know all of the hardware and software architecture details, but some knowledge of the underlying hardware and software is helpful for performance optimization and debug purposes.
A simplified view of the hardware/software stack is shown in the following figure.

The flow can be broken down as:

1. Application submits payloads via the Intel® QuickAssist API as part of the request. The user space library converts these requests into descriptors and places these in the Transmit (Tx) hardware-assisted queues (aka ring).
2. Firmware parses the descriptor and configures the accelerators accordingly. Upon a job completion firmware returns the processed payload (either encrypted or compressed or both) and generates a response message. This response message is inserted in the response ring.
3. A polling thread owned by the application queries the response ring via the Intel® Quick- Assist Library. If the application choses non-blocking calls the user space library will issue a callback to the application to inform that the operation is complete.

Note: The UIO (to be replaced with VFIO) layer is a framework present in both Linux Kernel and user space library libudev. This framework enables exchanging data between Kernel and user space. It offers better latency performance than IOCTL.

4.原理

A QAT device can host multiple acceleration services:

• symmetric cryptography
• data compression
• asymmetric cryptography

These services are provided to DPDK applications via PMDs which register to implement the corresponding cryptodev and compressdev APIs. The PMDs use common QAT driver code which manages the QAT PCI device. They also depend on a QAT kernel driver being installed on the platform, see Dependency on the QAT kernel driver below.

QAT compression PMD needs intermediate buffers to support Deflate compression with Dynamic Huffman encoding. RTE_PMD_QAT_COMP_IM_BUFFER_SIZE specifies the size of a single buffer, the PMD will allocate a multiple of these, plus some extra space for associated meta-data. For GEN2 devices, 20 buffers are allocated while for GEN1 devices, 12 buffers are allocated, plus 1472 bytes overhead.

Dependency on the QAT kernel driver

To use QAT an SRIOV-enabled QAT kernel driver is required. The VF devices created and initialised by this driver will be used by the QAT PMDs.

Instructions for installation are below, but first an explanation of the relationships between the PF/VF devices and the PMDs visible to DPDK applications.

Each QuickAssist PF device exposes a number of VF devices. Each VF device can enable one symmetric cryptodev PMD and/or one asymmetric cryptodev PMD and/or one compressdev PMD. These QAT PMDs share the same underlying device and pci-mgmt code, but are enumerated independently on their respective APIs and appear as independent devices to applications.

note:

Each VF can only be used by one DPDK process. It is not possible to share the same VF across multiple processes, even if these processes are using different acceleration services.

Conversely one DPDK process can use one or more QAT VFs and can expose both cryptodev and compressdev instances on each of those VFs.

Available kernel drivers

Kernel drivers for each device for each service are listed in the following table. (Scroll right to see the full table)

5.应用生态

6.应用案例

[1.data-compression-tuning-guide-on-xeon-systems](Data Compression with Intel® ISA-L/ Intel® IPP/ Intel® QAT Tuning…)

2.Intel QAT 助力 Nginx 压缩处理

3.基于第四代英特尔至强处理器的浪潮云海超融合压缩和纠删功能设计

4.TLS 加速技术：Intel QuickAssist Technology（QAT）解决方案