一文了解PCIe 6.0的基础和测试

2022年PCI-SIG组织宣布PCIe 6.0规范标准v1.0版本正式发布，宣告完工。

延续了惯例，带宽速度继续增倍，x16下可达128GB/s(单向)，由于PCIe技术允许数据全双工双向流动，因此双向总吞吐量就是256GB/s。

• Definitions/Metrics to Help Frame Server Design Challenges

• Disaggregation: dividing components and resources into subsystems

• Fabric attached: components attached in a mesh, including across network, allowing accessibility to a series of servers

• GFAM: Global Fabric Attached Memory

• IE: Fabric Attached Storage, Fabric Attached Memory

• Stranded memory: memory attached to a CPU that is accessed infrequently or not at all

• Metrics of interest contributing to server cost

• Estimated server memory cost – ~30% - 50% of total server cost

• Average utilization of DRAM (40%-50%)

If memory pooling could reduce total DRAM by 10%, total server costs could reduce by 4%-5%

Typical Applications Using PCIe and CXL Interfaces

History of PCIe Specifications

需要注意的是，此表中的PCle 3.0，相对PCle 2.0，频率只从5GT/s提高到8GT/s，增加了60%；而带宽则从16GB/s升级到32GB/s（16个Lane），这是为什么呢？

PCle 3.0编码方式从8b/10b变成了128b/130b，编码效率提高了24%，从而得到：

1.6 x 1.24 = 2 倍

的传输带宽。

受制于主板线材材质和成本的原因，这一次，PCle 6.0同样采用了类似方式，也就是仅仅通过改变编码方式到PAM-4来提高带宽。

PCIe 6.0改用PAM4脉冲调幅信令，1b/1b编码，单个信号就有能四种编码（00/01/10/11）状态，比之前翻番，允许承载最高30GHz频率。不过，由于PAM4信号比NRZ脆弱，所以必须搭配FEC前向纠错机制，纠正链路中的信号错误，保障数据完整性。

除了PAM4和FEC，PCIe 6.0额外在逻辑层使用FLIT（流量控制单元）编码。

PCIe 6.0 Goals and System Implications

• Goals of PCIe 6.0

• Double bandwidth per lane to 64 GT/sec

• Mandatory backwards-compatibility to PCIe 1.1

• Similar channel reach vs. PCIe 5.0

• Higher bandwidth efficiency

• Potential system level design changes moving to next gen PCIe

• Increased board design complexity and cost – same reach at higher data rate

• Need for new materials and/or greater manufacturing integrity

• Longer reach needed for riser cards and back planes

• Market demand to further expand disaggregation

Previous PCIe Generation(s) vs. PCIe 6.0

PAM-4介绍

PAM-4（pulse amplitude modulation，4-level）编码并不是一个新技术，在超高速以太网编码上已经广泛应用。但这不是它第一次进入板级总线PCIe6编码（还有GDDR6x）。我们知道，PCle传输信号是一对差分信号，解码的时候只有0和1两种，叫做NRZ（non-return-to-zero），而PAM-4根据则在0和1之间划分处更多的空间，一个周期可传输信号从1个bit变成传输2个bit：

System Level Considerations and Challenges

Overall system improvements are required:

• PCB and Package

• Higher layer count, improved dielectric material, tighter manufacturing tolerances

PCIe Retimer Use Models

PCIe Switch Use Models

• Connects a large number of peripheral devices to a processor system

• PCIe Switch used when connecting to JBOF (Just a Bunch of Flash) or JBOG (Just a Bunch of GPUs)

• Industry direction to fabric attached and disaggregation provides access to multiple sources of memory, storage or off load compute via CXL 3.0

PCIe 6.0 Controller IP Solution

Controller Features

• Native/AMBA interface with scalable data path width

• FLIT/non-FLIT modes

• Light weight FEC to improve BER

• Optional IDE Security Engine

• PIPE 6.1 (SerDes Mode)

• Same RTL for ASIC and FPGA implementation

• Fully parameterized through GUI configurator