CN117992374A - Storage expansion device and computing device - Google Patents
Storage expansion device and computing device Download PDFInfo
- Publication number
- CN117992374A CN117992374A CN202410129849.2A CN202410129849A CN117992374A CN 117992374 A CN117992374 A CN 117992374A CN 202410129849 A CN202410129849 A CN 202410129849A CN 117992374 A CN117992374 A CN 117992374A
- Authority
- CN
- China
- Prior art keywords
- cxl
- interface
- controller
- expansion device
- dimms
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
- G06F13/1668—Details of memory controller
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
- G06F13/1668—Details of memory controller
- G06F13/1694—Configuration of memory controller to different memory types
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/382—Information transfer, e.g. on bus using universal interface adapter
- G06F13/385—Information transfer, e.g. on bus using universal interface adapter for adaptation of a particular data processing system to different peripheral devices
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F2213/0026—PCI express
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F2213/16—Memory access
- G06F2213/1602—Memory access type
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Bus Control (AREA)
Abstract
Description
技术领域Technical Field
本发明涉及服务器技术领域,特别涉及一种存储扩展设备及计算设备。The present invention relates to the technical field of servers, and in particular to a storage expansion device and a computing device.
背景技术Background technique
随着计算机技术的发展,CPU的核心数快速增加,计算密度持续增长,受限于计算设备的物理空间和设计成本,内存性能的增长速度落后于计算密度的增长,平均到每个核心的内存性能持续下降,限制了计算设备算力的发挥。With the development of computer technology, the number of CPU cores has increased rapidly and computing density has continued to grow. However, limited by the physical space and design cost of computing devices, the growth rate of memory performance has lagged behind the growth of computing density. The average memory performance per core has continued to decline, limiting the computing power of computing devices.
计算快速链路(Compute Express Link,CXL)作为一种开放的行业标准,在数据中心的专用计算、内存、I/O和存储元素之间提供高带宽、低延迟的连接。计算设备通过CXL总线扩展内存可以有效解决内存墙和IO墙的瓶颈。目前多种形态的硬件板卡通过CXL总线扩展内存,例如包括添加卡(Add-in-Card,AIC),E3.S形态或者定制的异形卡。Compute Express Link (CXL) is an open industry standard that provides high-bandwidth, low-latency connections between dedicated computing, memory, I/O, and storage elements in data centers. Computing devices can effectively solve the bottlenecks of memory walls and IO walls by expanding memory through the CXL bus. Currently, various forms of hardware boards can expand memory through the CXL bus, such as Add-in-Card (AIC), E3.S form, or customized special-shaped cards.
目前,单个AIC卡支持两个双倍速率(DDR,Double Data Rate)通道,单个DDR通道可支持每通道双列的直插式存储模组(DPC,DIMM Per Channel),因此,单个AIC卡可以连接4根双列直插式存储模组(Dual-Inline-Memory-Modules,DIMM)。Currently, a single AIC card supports two double data rate (DDR) channels, and a single DDR channel can support dual-rank inline memory modules per channel (DPC, DIMM Per Channel). Therefore, a single AIC card can connect to four dual-inline-memory-modules (DIMMs).
因此,现有技术中通过单个AIC卡的内存访问端口仅可以访问CXL控制器连接的一部分DIMM,访问空间有限。Therefore, in the prior art, only a portion of the DIMMs connected to the CXL controller can be accessed through the memory access port of a single AIC card, and the access space is limited.
发明内容Summary of the invention
本发明实施例提供一种存储扩展设备及计算设备,能够提高存储扩展设备的单个内存访问端口的访问空间。The embodiments of the present invention provide a storage expansion device and a computing device, which can increase the access space of a single memory access port of the storage expansion device.
本申请实施例提供一种存储扩展设备,包括:基板和设置在基板上的线缆接口、PCIE接口、第一CXL控制器、第二CXL控制器及多根双列直插式存储模组DIMM;第一CXL控制器与多根DIMM中的第一部分DIMM连接,第二CXL控制器与多根DIMM中的第二部分DIMM连接;第一CXL控制器的CXL接口分为第一部分CXL接口和第二部分CXL接口,第一CXL控制器的第一部分CXL接口用于连接线缆接口,第一CXL控制器的第二部分CXL接口用于连接PCIE接口;第一CXL控制器的第一部分CXL接口和第二部分CXL接口均能访问第一CXL控制器的连接的所有DIMM的内存空间;第二CXL控制器的CXL接口分为第一部分CXL接口和第二部分CXL接口,第二CXL控制器的第一部分CXL接口用于连接线缆接口,第二CXL控制器的第二部分CXL接口用于连接PCIE接口;第二CXL控制器的第一部分CXL接口和第二部分CXL接口均能访问第二CXL控制器连接的所有DIMM的内存空间;线缆接口,用于访问第一CXL控制器和第二CXL控制器连接的所有内存空间;PCIE接口,用于访问第一CXL控制器和第二CXL控制器连接的所有内存空间。The embodiment of the present application provides a storage expansion device, comprising: a substrate and a cable interface, a PCIE interface, a first CXL controller, a second CXL controller and a plurality of dual in-line memory modules DIMMs arranged on the substrate; the first CXL controller is connected to a first part of the DIMMs in the plurality of DIMMs, and the second CXL controller is connected to a second part of the DIMMs in the plurality of DIMMs; a CXL interface of the first CXL controller is divided into a first part of the CXL interface and a second part of the CXL interface, the first part of the CXL interface of the first CXL controller is used to connect to the cable interface, and the second part of the CXL interface of the first CXL controller is used to connect to the PCIE interface; the first part of the CXL interface of the first CXL controller and the second part of the CXL interface of the first CXL controller are connected to the first part of the CXL interface of the first CXL controller. The interfaces can access the memory space of all DIMMs connected to the first CXL controller; the CXL interface of the second CXL controller is divided into a first part CXL interface and a second part CXL interface, the first part CXL interface of the second CXL controller is used to connect the cable interface, and the second part CXL interface of the second CXL controller is used to connect the PCIE interface; the first part CXL interface and the second part CXL interface of the second CXL controller can both access the memory space of all DIMMs connected to the second CXL controller; the cable interface is used to access all memory spaces connected to the first CXL controller and the second CXL controller; the PCIE interface is used to access all memory spaces connected to the first CXL controller and the second CXL controller.
本申请实施例提供的存储扩展设备,包括至少两个CXL控制器,每个CXL控制器的CXL接口分为两部分,每部分CXL接口都可以访问CXL控制器连接的DIMM的所有内存空间。这样可以实现服务器连接CXL控制器的任意一个CXL接口,都可以访问CXL控制器扩展的所有DIMM。例如内存访问端口可以包括PCIE接口和线缆接口,通过PCIE接口或线缆接口都能访问到CXL控制器连接的所有DIMM的内存空间。CXL控制器支持分叉功能,分叉是指CXL控制器的CXL接口可以被分为多个部分,例如被为两部分,或被为更多部分,可以被为四部分。当然,CXL接口可以以平分的方式分为多个部分,每个部分的引脚数量相同,也可以以非平分的方式分为多个部分,多个部分中的任意两个部分的引脚数量可以不同。The storage expansion device provided in the embodiment of the present application includes at least two CXL controllers, and the CXL interface of each CXL controller is divided into two parts, and each part of the CXL interface can access all the memory spaces of the DIMMs connected to the CXL controller. In this way, any CXL interface of the server connected to the CXL controller can access all the DIMMs extended by the CXL controller. For example, the memory access port may include a PCIE interface and a cable interface, and the memory space of all DIMMs connected to the CXL controller can be accessed through the PCIE interface or the cable interface. The CXL controller supports a bifurcation function, and bifurcation means that the CXL interface of the CXL controller can be divided into multiple parts, for example, two parts, or more parts, and can be four parts. Of course, the CXL interface can be divided into multiple parts in an equal manner, and the number of pins of each part is the same, or it can be divided into multiple parts in a non-equal manner, and the number of pins of any two parts in the multiple parts can be different.
本实施例中的存储扩展设备可以通过线缆接口和PCIE接口分别连接不同的两个计算设备,从而两个计算设备可以共享存储扩展设备的内存空间。The storage expansion device in this embodiment can be connected to two different computing devices through a cable interface and a PCIE interface respectively, so that the two computing devices can share the memory space of the storage expansion device.
一种可能的实现方式,第一CXL控制器的CXL接口为X16接口,第一CXL控制器的第一部分CXL接口和第二部分CXL接口均为X8接口;第一CXL控制器的第一部分CXL接口连接线缆接口的第一X8接口,第二CXL控制器的第一部分CXL接口连接线缆接口的第二X8接口;线缆接口用于访问第一部分DIMM的所有内存空间,以及访问第二部分DIMM的所有内存空间。In a possible implementation, the CXL interface of the first CXL controller is an X16 interface, and the first part CXL interface and the second part CXL interface of the first CXL controller are both X8 interfaces; the first part CXL interface of the first CXL controller is connected to the first X8 interface of the cable interface, and the first part CXL interface of the second CXL controller is connected to the second X8 interface of the cable interface; the cable interface is used to access all memory spaces of the first part DIMMs and access all memory spaces of the second part DIMMs.
本申请实施例不具体限定第一CXL控制器和第二CXL控制器包括的CXL接口的数量,以上仅是一个为例,以一个X16的CXL接口平分为两部分,即第一CXL控制器和第二CXL控制器均包括两个X8接口。本申请实施例提供的存储扩展设备,通过线缆接口可以访问第一CXL控制器连接的所有DIMM的内存空间,而且也可以通过线缆接口访问第二CXL控制器连接的所有DIMM的内存空间。The embodiment of the present application does not specifically limit the number of CXL interfaces included in the first CXL controller and the second CXL controller. The above is just an example, taking an X16 CXL interface as two parts, that is, the first CXL controller and the second CXL controller each include two X8 interfaces. The storage expansion device provided in the embodiment of the present application can access the memory space of all DIMMs connected to the first CXL controller through the cable interface, and can also access the memory space of all DIMMs connected to the second CXL controller through the cable interface.
一种可能的实现方式,第二CXL控制器的CXL接口为X16接口,第二CXL控制器的第一部分CXL接口和第二部分CXL接口均为X8接口;PCIE接口分为第一X8接口和第二X8接口;第一CXL控制器的第二部分CXL接口连接PCIE接口的第一X8接口,第二CXL控制器的第二部分CXL接口连接PCIE接口的第二X8接口;PCIE接口用于访问第一部分DIMM的所有内存空间,以及访问第二部分DIMM的所有内存空间。In a possible implementation manner, the CXL interface of the second CXL controller is an X16 interface, and the first part CXL interface and the second part CXL interface of the second CXL controller are both X8 interfaces; the PCIE interface is divided into a first X8 interface and a second X8 interface; the second part CXL interface of the first CXL controller is connected to the first X8 interface of the PCIE interface, and the second part CXL interface of the second CXL controller is connected to the second X8 interface of the PCIE interface; the PCIE interface is used to access all memory spaces of the first part DIMMs and access all memory spaces of the second part DIMMs.
本申请实施例提供的存储扩展设备,通过PCIE接口可以访问第一CXL控制器连接的所有DIMM的内存空间,而且也可以通过PCIE接口访问第二CXL控制器连接的所有DIMM的内存空间。The storage expansion device provided in the embodiment of the present application can access the memory space of all DIMMs connected to the first CXL controller through the PCIE interface, and can also access the memory space of all DIMMs connected to the second CXL controller through the PCIE interface.
一种可能的实现方式,第一CXL控制器存储有第一映射表和第二映射表,第一映射表与第一CXL控制器的第一部分CXL接口对应,第二映射表与第一CXL控制器的第二部分CXL接口对应,第一映射表实现第一CXL控制器的第一部分CXL接口连接的服务器的内存物理地址到第一CXL控制器的所有DIMM内存物理地址的映射转换;第二映射表实现第一CXL控制器的第二部分CXL接口连接的服务器的内存物理地址到第一CXL控制器的所有DIMM内存物理地址的映射转换。In a possible implementation, the first CXL controller stores a first mapping table and a second mapping table, the first mapping table corresponds to the first part of the CXL interface of the first CXL controller, and the second mapping table corresponds to the second part of the CXL interface of the first CXL controller, the first mapping table implements mapping conversion of the memory physical address of the server connected to the first part of the CXL interface of the first CXL controller to the memory physical addresses of all DIMMs of the first CXL controller; the second mapping table implements mapping conversion of the memory physical address of the server connected to the second part of the CXL interface of the first CXL controller to the memory physical addresses of all DIMMs of the first CXL controller.
本申请实施例提供的存储扩展设备,在第一CXL控制器内部设置两个映射表,分别对应两部分CXL接口,进而可以使连接两部分CXL接口的内存访问接口,可以通过分别对应的映射表访问第一CXL控制器连接的所有DIMM的内存空间。The storage expansion device provided in the embodiment of the present application sets two mapping tables inside the first CXL controller, which correspond to the two parts of CXL interfaces respectively, so that the memory access interface connected to the two parts of CXL interfaces can access the memory space of all DIMMs connected to the first CXL controller through the corresponding mapping tables.
一种可能的实现方式,第二CXL控制器存储有第三映射表和第四映射表,第三映射表与第二CXL控制器的第一部分CXL接口对应,第三映射表与第一CXL控制器的第二部分CXL接口对应,第三映射表实现第二CXL控制器的第一部分CXL接口连接的服务器的内存物理地址到第二CXL控制器的所有DIMM内存物理地址的映射转换;第四映射表实现第二CXL控制器的第二部分CXL接口连接的服务器的内存物理地址到第二CXL控制器的所有DIMM内存物理地址的映射转换。In a possible implementation, the second CXL controller stores a third mapping table and a fourth mapping table, the third mapping table corresponds to the first part of the CXL interface of the second CXL controller, and the third mapping table corresponds to the second part of the CXL interface of the first CXL controller, the third mapping table implements the mapping conversion of the memory physical address of the server connected to the first part of the CXL interface of the second CXL controller to the memory physical addresses of all DIMMs of the second CXL controller; the fourth mapping table implements the mapping conversion of the memory physical address of the server connected to the second part of the CXL interface of the second CXL controller to the memory physical addresses of all DIMMs of the second CXL controller.
本申请实施例提供的存储扩展设备,在第二CXL控制器内部设置两个映射表,分别对应两部分CXL接口,进而可以使连接两部分CXL接口的内存访问接口,可以通过分别对应的映射表访问第一CXL控制器连接的所有DIMM的内存空间。The storage expansion device provided in the embodiment of the present application sets two mapping tables inside the second CXL controller, which correspond to the two parts of CXL interfaces respectively, so that the memory access interface connected to the two parts of CXL interfaces can access the memory space of all DIMMs connected to the first CXL controller through the corresponding mapping tables.
一种可能的实现方式,还包括设置在基板上的传感器和系统管理总线SMbus开关接口;传感器用于检测存储扩展设备的温度;SMbus开关接口的第一端连接第一CXL控制器和第二CXL控制器;SMbus开关接口的第二端连接传感器;SMbus开关接口的第三端用于线缆接口或PCIE接口。A possible implementation method also includes a sensor and a system management bus (SMbus) switch interface arranged on the substrate; the sensor is used to detect the temperature of the storage expansion device; the first end of the SMbus switch interface is connected to the first CXL controller and the second CXL controller; the second end of the SMbus switch interface is connected to the sensor; and the third end of the SMbus switch interface is used for a cable interface or a PCIE interface.
服务器通过连接SMbus开关接口,通过SMbus开关接口访问传感器、第一CXL控制器和第二CXL控制器中的至少一项。另外,服务器可以通过SMbus开关接口同时访问传感器、第一CXL控制器和第二CXL控制器。服务器访问第一CXL控制器和第二CXL控制器,可以访问第一CXL控制器和第二CXL控制器扩展的DIMM中的数据。The server accesses at least one of the sensor, the first CXL controller and the second CXL controller through the SMbus switch interface by connecting to the SMbus switch interface. In addition, the server can access the sensor, the first CXL controller and the second CXL controller at the same time through the SMbus switch interface. The server accesses the first CXL controller and the second CXL controller, and can access data in the DIMMs expanded by the first CXL controller and the second CXL controller.
一种可能的实现方式,线缆接口和PCIE接口均提供时钟引脚、复位引脚、SMbus引脚、在位引脚和电源引脚,SMbus开关接口的第三端与线缆接口和PCIE接口的SMbus引脚连接。In a possible implementation, both the cable interface and the PCIE interface provide a clock pin, a reset pin, an SMbus pin, an in-position pin and a power pin, and the third end of the SMbus switch interface is connected to the SMbus pins of the cable interface and the PCIE interface.
一种可能的实现方式,存储扩展设备为PCIE标卡形态,PCIE接口设于基板的长边,用于垂直插接于计算设备中的背板,或者,存储扩展设备为插卡形态,PCIE接口设于基板的短边,用于水平插接于计算设备中的背板。In one possible implementation, the storage expansion device is in the form of a PCIE standard card, and the PCIE interface is arranged on the long side of the substrate for vertically plugging into the backplane of the computing device; or, the storage expansion device is in the form of a plug-in card, and the PCIE interface is arranged on the short side of the substrate for horizontally plugging into the backplane of the computing device.
本申请实施例提供的存储扩展设备,不具体限定PCIE接口设置的位置,可以根据需要来自由设计,方便连接计算设备中的背板即可。The storage expansion device provided in the embodiment of the present application does not specifically limit the location of the PCIE interface setting and can be freely designed according to needs to facilitate connection to the backplane in the computing device.
一种可能的实现方式,本申请实施例提供的存储扩展设备,还包括设置在基板上的第一全双工同步串行总线SPI闪存和第二SPI闪存;第一CXL控制器连接第一SPI闪存;第二CXL控制器连接第二SPI闪存。In a possible implementation, the storage expansion device provided in an embodiment of the present application further includes a first full-duplex synchronous serial bus SPI flash memory and a second SPI flash memory arranged on the substrate; the first CXL controller is connected to the first SPI flash memory; and the second CXL controller is connected to the second SPI flash memory.
第一SPI闪存C和第二SPI闪存D均为电子式可清除程序化只读存储器的形式,允许SPI数据的多次擦或写。The first SPI flash memory C and the second SPI flash memory D are both in the form of electronically erasable programmable read-only memory, allowing multiple erasing or writing of SPI data.
一种可能的实现方式,本申请实施例提供的计算设备,还包括:调试Debug接口;第一CXL控制器和第二CXL控制器均连接Debug接口。In a possible implementation, the computing device provided in the embodiment of the present application further includes: a debugging Debug interface; the first CXL controller and the second CXL controller are both connected to the Debug interface.
Debug接口可以连接外部设备,以实现对存储扩展设备的第一CXL控制器和第二CXL控制器测试或者调试。The debug interface can be connected to an external device to test or debug the first CXL controller and the second CXL controller of the storage expansion device.
一种可能的实现方式,本申请实施例提供的存储扩展设备,还包括支撑结构。In a possible implementation manner, the storage expansion device provided in the embodiment of the present application further includes a supporting structure.
支撑结构可以将存储扩展设备与支撑板相对固定,例如支撑板可以为金属板,存储扩展设备通过固定支撑的结构固定在金属板上,起到强度支撑作用。本申请实施例不具体限定支撑结构的类型,例如可以为螺钉或铆钉,也不具体限定支撑结果的数量,可以根据实际需要来设置。The support structure can fix the storage expansion device and the support plate relative to each other. For example, the support plate can be a metal plate, and the storage expansion device is fixed to the metal plate by the fixed support structure to play a role in strength support. The embodiment of the present application does not specifically limit the type of the support structure, for example, it can be screws or rivets, nor does it specifically limit the number of support structures, which can be set according to actual needs.
第二方面,本申请实施例还提供一种存储扩展设备,包括:基板和设置在基板上的线缆接口、PCIE接口、CXL控制器及多根双列直插式存储模组DIMM;CXL控制器与多根DIMM连接,CXL控制器的CXL接口分为第一部分CXL接口和第二部分CXL接口,第一部分CXL接口连接线缆接口,第二部分CXL接口连接PCIE接口;第一部分CXL接口和第二部分CXL接口均能访问CXL控制器连接的所有DIMM的内存空间;线缆接口,用于访问CXL控制器的所有内存空间;PCIE接口,用于访问CXL控制器的所有内存空间。In a second aspect, an embodiment of the present application further provides a storage expansion device, including: a substrate and a cable interface, a PCIE interface, a CXL controller and a plurality of dual in-line memory modules DIMMs arranged on the substrate; the CXL controller is connected to the plurality of DIMMs, and the CXL interface of the CXL controller is divided into a first part CXL interface and a second part CXL interface, the first part CXL interface is connected to the cable interface, and the second part CXL interface is connected to the PCIE interface; both the first part CXL interface and the second part CXL interface can access the memory space of all DIMMs connected to the CXL controller; the cable interface is used to access all memory spaces of the CXL controller; the PCIE interface is used to access all memory spaces of the CXL controller.
第三方面,本申请实施例还提供一种计算设备,包括:背板和以上介绍的存储扩展设备;存储扩展设备连接背板。In a third aspect, an embodiment of the present application further provides a computing device, including: a backplane and the storage expansion device introduced above; the storage expansion device is connected to the backplane.
本申请实施例提供的计算设备,通过PCIE接口即金手指可以访问所有的内存,通过线缆接口也可以访问所有的内存,实现了较少的CXL总线接口资源访问高密度内存的需求,例如可以利用X16接口扩展8根DIMM的内存密度,实现利用较少的接口可以访问更多的内存,从而节省接口数量,并且提高单个存储扩展设备的内存密度,进而提升计算设备的内存密度。The computing device provided in the embodiment of the present application can access all memories through the PCIE interface, i.e., the gold finger, and can also access all memories through the cable interface, thereby realizing the need to access high-density memory with fewer CXL bus interface resources. For example, the memory density of 8 DIMMs can be expanded using the X16 interface, so that more memory can be accessed using fewer interfaces, thereby saving the number of interfaces and improving the memory density of a single storage expansion device, thereby improving the memory density of the computing device.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1为本申请实施例提供的一种存储扩展设备的示意图;FIG1 is a schematic diagram of a storage expansion device provided in an embodiment of the present application;
图2为本申请实施例提供的另一种存储扩展设备的示意图;FIG2 is a schematic diagram of another storage expansion device provided in an embodiment of the present application;
图3为本申请实施例提供的又一种存储扩展设备的示意图;FIG3 is a schematic diagram of another storage expansion device provided in an embodiment of the present application;
图4A为本申请实施例提供的再一种存储扩展设备的示意图;FIG4A is a schematic diagram of another storage expansion device provided in an embodiment of the present application;
图4B为本申请实施例提供的又一种存储扩展设备的示意图;FIG4B is a schematic diagram of another storage expansion device provided in an embodiment of the present application;
图5为本申请实施例提供的另一种存储扩展设备的示意图;FIG5 is a schematic diagram of another storage expansion device provided in an embodiment of the present application;
图6为本申请实施例提供的又一种存储扩展设备的示意图;FIG6 is a schematic diagram of another storage expansion device provided in an embodiment of the present application;
图7为本申请实施例提供的再一种存储扩展设备的示意图;FIG7 is a schematic diagram of another storage expansion device provided in an embodiment of the present application;
图8为本申请实施例提供的另一种存储扩展设备的示意图;FIG8 is a schematic diagram of another storage expansion device provided in an embodiment of the present application;
图9为本申请实施例提供的再一种存储扩展设备的示意图;FIG9 is a schematic diagram of another storage expansion device provided in an embodiment of the present application;
图10为本申请实施例提供的一种计算设备的示意图。FIG. 10 is a schematic diagram of a computing device provided in an embodiment of the present application.
具体实施方式Detailed ways
本申请实施例提供的计算设备不具体限定应用的场景,例如,计算设备以服务器为例进行介绍,具体也不限定服务器的类型,例如计算设备可以为机架服务器或边缘服务器。服务器可以位于数据中心,也可以位于其他区域,本申请实施例不做具体限定。The computing device provided in the embodiment of the present application does not specifically limit the application scenario. For example, the computing device is introduced by taking a server as an example, and the type of the server is not specifically limited. For example, the computing device can be a rack server or an edge server. The server can be located in a data center or in other areas, and the embodiment of the present application does not make specific limitations.
服务器,属于计算设备的一种类型,服务器比普通计算机运行更快、负载更高。服务器在网络中为其它客户机提供计算或者应用服务。例如个人电脑(Personal Computer,PC)或智能手机等设备。服务器具有高速的CPU运算能力、长时间的可靠运行、强大的外部数据吞吐能力以及更好的扩展性。服务器从外形分为机架式、刀片式、塔式和机柜式。Servers are a type of computing device that runs faster and has a higher load than ordinary computers. Servers provide computing or application services to other clients in the network. For example, personal computers (PCs) or smart phones. Servers have high-speed CPU computing power, long-term reliable operation, strong external data throughput and better scalability. Servers are divided into rack-mounted, blade-mounted, tower-mounted and cabinet-mounted types.
服务器一般包括主板和供电电源,供电电源用于给主板的各个负载进行供电。本申请实施例不具体限定供电电源提供给主板的电压等级,例如以直流12V为例进行介绍。The server generally includes a mainboard and a power supply, and the power supply is used to supply power to various loads of the mainboard. The embodiment of the present application does not specifically limit the voltage level provided by the power supply to the mainboard, for example, DC 12V is used as an example for introduction.
主板,服务器中的一种重要电路板,主板包括基板和设置在基板上的基板管理控制器(baseboard manager controller,BMC)、中央处理器CPU、控制器、存储器、连接器等部件,另外,主板可以包括一个CPU,也可以包括多个CPU。控制器接口有限,因此可以通过连接器来扩展接口,以便于连接外围设备,例如扩展USB口,连接鼠标键盘等设备;扩展串行数据接口,连接显卡等设备。控制器可以是微控制单元(micro controller unit,MCU)、复杂可编程逻辑器件(complex Programming logic device,CPLD)、现场可编程逻辑门阵列(Field Programmable Gate Array,FPGA)中的一种或多种。The motherboard is an important circuit board in the server. The motherboard includes a substrate and a baseboard manager controller (BMC) arranged on the substrate, a central processing unit (CPU), a controller, a memory, a connector and other components. In addition, the motherboard can include one CPU or multiple CPUs. The controller interface is limited, so the interface can be expanded through a connector to facilitate the connection of peripheral devices, such as expanding the USB port to connect devices such as a mouse and keyboard; expanding the serial data interface to connect devices such as a graphics card. The controller can be one or more of a micro controller unit (MCU), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA).
本申请实施例不具体限定存储器的具体类型,例如存储器包括但不限定以下类型:双列直插式存储模块(DIMM,Dual-Inline-Memory-Modules)、机械硬盘(Hard DiskDrive,HDD)等。The embodiments of the present application do not specifically limit the specific type of memory. For example, the memory includes but is not limited to the following types: dual-inline-memory-modules (DIMM), hard disk drive (HDD), etc.
高速串行计算机扩展总线标准(Peripheral Component Interconnect Express,PCIE)卡,可以连接服务器中的连接器,方便为服务器的控制器扩展外围设备,例如连接显卡或存储卡等。PCIE卡属于高速串行点对点双通道高带宽传输,一般采用差分信号通道,可以为连接的设备分配独享的通道带宽,不共享总线带宽,主要支持主动电源管理,错误报告,端对端的可靠性传输,热插拔以及服务质量等功能。The high-speed serial computer expansion bus standard (Peripheral Component Interconnect Express, PCIE) card can be connected to the connector in the server to facilitate the expansion of peripheral devices for the server controller, such as connecting a graphics card or a memory card. The PCIE card belongs to high-speed serial point-to-point dual-channel high-bandwidth transmission, generally using differential signal channels, which can allocate exclusive channel bandwidth for connected devices and do not share bus bandwidth. It mainly supports active power management, error reporting, end-to-end reliability transmission, hot plugging, and quality of service functions.
串行外围设备接口(Serial Peripheral Interface,SPI),SPI是一种高速的、全双工、同步的通信总线,并且在芯片的管脚上只占用四根线。Serial Peripheral Interface (SPI), SPI is a high-speed, full-duplex, synchronous communication bus that only occupies four wires on the chip pins.
系统管理总线(System Management Bus,SMBus)是1995年由Intel提出的,应用于移动PC和桌面PC系统中的低速率通讯。SMBus为功能强大的总线,通过两条线来控制主板上的设备并收集相应的信息。The System Management Bus (SMBus) was proposed by Intel in 1995 and is used for low-speed communication in mobile PCs and desktop PC systems. SMBus is a powerful bus that controls devices on the motherboard and collects corresponding information through two lines.
为了使本领域技术人员更好地理解本申请实施例提供的技术方案,下面结合附图进行详细介绍。In order to enable those skilled in the art to better understand the technical solution provided by the embodiments of the present application, a detailed description is given below in conjunction with the accompanying drawings.
参见图1,该图为本申请实施例提供的一种存储扩展设备的示意图。Refer to Figure 1, which is a schematic diagram of a storage expansion device provided in an embodiment of the present application.
本申请实施例提供的存储扩展设备,包括:基板1000和设置在基板1000上的线缆接口200或PCIE接口100中的至少一项,以及至少一个CXL控制器;图1中以存储扩展设备的基板上设置包括第一CXL控制器A和第二CXL控制器B为例进行介绍。第一CXL控制器A用于连接第一组4根DIMM。第二CXL控制器B用于连接第二组4根DIMM。The storage expansion device provided in the embodiment of the present application includes: a base plate 1000 and at least one of a cable interface 200 or a PCIE interface 100 provided on the base plate 1000, and at least one CXL controller; FIG1 takes the base plate of the storage expansion device as an example for introduction, in which the first CXL controller A and the second CXL controller B are provided. The first CXL controller A is used to connect a first group of 4 DIMMs. The second CXL controller B is used to connect a second group of 4 DIMMs.
示例的,基板上设有多个PCIe插槽,DIMM插设于对应的PCIe插槽中。For example, a plurality of PCIe slots are provided on the substrate, and the DIMMs are inserted into corresponding PCIe slots.
应该理解,CXL控制器包括CXL接口,CXL接口可以包括X8接口或X16接口。X8接口包括8对差分信号对引脚,X16接口包括16对差分信号对引脚。It should be understood that the CXL controller includes a CXL interface, and the CXL interface may include an X8 interface or an X16 interface. The X8 interface includes 8 pairs of differential signal pins, and the X16 interface includes 16 pairs of differential signal pins.
需要说明的是,本实施例中的存储扩展设备的内存访问端口可以同时设置线缆接口200和PCIE接口100,存储扩展设备也可以仅设置线缆接口200,或仅设置PCIE接口100。与存储扩展设备的线路接口连接的设备或器件通过线缆接口200可以访问存储扩展设备上的所有DIMM,与存储扩展设备的PCIE接口连接的设备或器件通过PCIE接口100也可以访问存储扩展设备上的所有DIMM。本实施例下文以存储扩展设备同时包括线缆接口200和PCIE接口100进行说明。It should be noted that the memory access port of the storage expansion device in this embodiment can be provided with both the cable interface 200 and the PCIE interface 100, and the storage expansion device can also be provided with only the cable interface 200, or only the PCIE interface 100. The device or component connected to the line interface of the storage expansion device can access all DIMMs on the storage expansion device through the cable interface 200, and the device or component connected to the PCIE interface of the storage expansion device can also access all DIMMs on the storage expansion device through the PCIE interface 100. This embodiment is described below with the storage expansion device including both the cable interface 200 and the PCIE interface 100.
本申请实施例不具体限定存储扩展设备上设置的CXL控制器的数量,可以为一个,也可以为多个。但是,本申请实施例提供的存储扩展设备上的每个CXL控制器可以连接4根DIMM。The embodiment of the present application does not specifically limit the number of CXL controllers provided on the storage expansion device, which may be one or more. However, each CXL controller on the storage expansion device provided by the embodiment of the present application may be connected to four DIMMs.
第一CXL控制器A支持分叉功能(Bifurcation),分叉是指CXL控制器的CXL接口可以被分为多个部分,例如被为两部分,或被为更多部分,可以被为四部分。当然,CXL接口可以以平分的方式分为多个部分,每个部分的引脚数量相同,也可以以非平分的方式分为多个部分,多个部分中的任意两个部分的引脚数量可以不同。本申请实施例中以CXL控制器的CXL接口被平分为两部分为例进行介绍。The first CXL controller A supports a bifurcation function. Bifurcation means that the CXL interface of the CXL controller can be divided into multiple parts, for example, two parts, or more parts, such as four parts. Of course, the CXL interface can be divided into multiple parts in an equal manner, and the number of pins in each part is the same, or it can be divided into multiple parts in a non-equal manner, and the number of pins in any two parts of the multiple parts can be different. In the embodiment of the present application, the CXL interface of the CXL controller is divided into two parts as an example for introduction.
以CXL控制器的CXL接口包括X16接口为例,X16接口分为两个X8接口。对应地,线缆接口200包括X16接口,即包括2个X8接口。同理,PCIE接口100也包括2个X8接口。For example, the CXL interface of the CXL controller includes an X16 interface, which is divided into two X8 interfaces. Correspondingly, the cable interface 200 includes an X16 interface, that is, includes two X8 interfaces. Similarly, the PCIE interface 100 also includes two X8 interfaces.
第一CXL控制器A的CXL接口包括第一部分CXL接口和第二部分CXL接口,第一CXL控制器A的第一部分CXL接口连接线缆接口200的第一X8接口,第一CXL控制器A的第二部分CXL接口连接PCIE接口100的第一X8接口;例如,第一CXL控制器A的CXL接口为X16接口,则X16接口平分为两个X8接口。例如,其中一个X8接口用于连接线缆接口200,另一个X8接口用于连接PCIE接口100。应该理解,PCIE接口100为金手指接口,例如,存储扩展设备通过金手指连接服务器中的背板,为服务器提供内存访问通道。The CXL interface of the first CXL controller A includes a first part CXL interface and a second part CXL interface, the first part CXL interface of the first CXL controller A is connected to the first X8 interface of the cable interface 200, and the second part CXL interface of the first CXL controller A is connected to the first X8 interface of the PCIE interface 100; for example, if the CXL interface of the first CXL controller A is an X16 interface, the X16 interface is divided into two X8 interfaces. For example, one of the X8 interfaces is used to connect to the cable interface 200, and the other X8 interface is used to connect to the PCIE interface 100. It should be understood that the PCIE interface 100 is a gold finger interface, for example, a storage expansion device is connected to the backplane in the server through a gold finger to provide a memory access channel for the server.
本申请实施例中以X16接口分为2个X8接口为例,其他实现方式中,也可以X16接口分为4个X4接口等,本申请实施例不做具体限定。In the embodiment of the present application, the X16 interface is divided into two X8 interfaces as an example. In other implementations, the X16 interface can also be divided into four X4 interfaces, etc., which is not specifically limited in the embodiment of the present application.
X16接口分为2个X8接口可以理解为,X16接口为一个接口实体,X16接口的一半引脚划分为一个虚拟的X8接口,X16接口的另一半引脚划分为另一个虚拟的X8接口,也就是说,两个X8接口是通过将一个X16接口进行引脚划分得到的。当然,在其他实施例中,CXL控制器也可以包括两个实体的X8接口。The X16 interface is divided into two X8 interfaces, which can be understood as follows: the X16 interface is an interface entity, half of the pins of the X16 interface are divided into a virtual X8 interface, and the other half of the pins of the X16 interface are divided into another virtual X8 interface, that is, the two X8 interfaces are obtained by dividing the pins of one X16 interface. Of course, in other embodiments, the CXL controller may also include two physical X8 interfaces.
第二CXL控制器B支持分叉功能,第二CXL控制器B的CXL接口包括第一部分CXL接口和第二部分CXL接口,第二CXL控制器B的第一部分CXL接口连接线缆接口200的第二x8接口,第二CXL控制器B的第二部分CXL接口连接PCIE接口100的第二X8接口;例如,第二CXL控制器B的CXL接口为X16接口,则X16接口被平分为两个X8接口。其中一个X8接口用于连接线缆接口200,另一个X8接口用于连接PCIE接口100。The second CXL controller B supports a bifurcation function. The CXL interface of the second CXL controller B includes a first CXL interface and a second CXL interface. The first CXL interface of the second CXL controller B is connected to the second x8 interface of the cable interface 200, and the second CXL interface of the second CXL controller B is connected to the second x8 interface of the PCIE interface 100. For example, if the CXL interface of the second CXL controller B is an x16 interface, the x16 interface is equally divided into two x8 interfaces. One of the x8 interfaces is used to connect to the cable interface 200, and the other x8 interface is used to connect to the PCIE interface 100.
服务器可以通过线缆接口200访问对应的第一CXL控制器A扩展的所有内存空间,以及访问对应的第二CXL控制器B扩展的所有内存空间。即,当存储扩展设备包括两个CXL控制器时,服务器可以通过线缆接口200访问的第一CXL控制器A扩展的第一组4根DIMM以及第二CXL控制器B扩展的第二组4根DIMM,即服务器可以通过线缆接口200访问存储扩展设备上的8根DIMM。The server can access all memory spaces of the corresponding first CXL controller A expansion and all memory spaces of the corresponding second CXL controller B expansion through the cable interface 200. That is, when the storage expansion device includes two CXL controllers, the server can access the first group of 4 DIMMs of the first CXL controller A expansion and the second group of 4 DIMMs of the second CXL controller B expansion through the cable interface 200, that is, the server can access 8 DIMMs on the storage expansion device through the cable interface 200.
例如,第一CXL控制器的X8接口连接线缆接口200,第二CXL控制器的X8接口也连接线缆接口200。线缆接口200包括X16接口,其中,服务器通过线缆接口200的X8接口可以访问第一CXL控制器扩展的所有DIMM,服务器通过线缆接口200的另外X8接口可以访问第二CXL控制器扩展的所有DIMM。应该理解,第一CXL控制器连接4根DIMM,第二CXL控制器连接4根DIMM,即服务器通过线缆接口200总共可以访问存储扩展设备上的8根DIMM。For example, the X8 interface of the first CXL controller is connected to the cable interface 200, and the X8 interface of the second CXL controller is also connected to the cable interface 200. The cable interface 200 includes an X16 interface, wherein the server can access all DIMMs expanded by the first CXL controller through the X8 interface of the cable interface 200, and the server can access all DIMMs expanded by the second CXL controller through another X8 interface of the cable interface 200. It should be understood that the first CXL controller is connected to 4 DIMMs, and the second CXL controller is connected to 4 DIMMs, that is, the server can access a total of 8 DIMMs on the storage expansion device through the cable interface 200.
图1的PCIE接口100设置在基板的长边,图1的线缆接口200设置在基板的短边。而且每个CXL控制器连接的4根DIMM平行于基板的长边设置,应该理解,图1中各个器件的位置关系仅是示意,本申请实施例中不做具体限定。The PCIE interface 100 of FIG1 is arranged on the long side of the substrate, and the cable interface 200 of FIG1 is arranged on the short side of the substrate. Moreover, the four DIMMs connected to each CXL controller are arranged parallel to the long side of the substrate. It should be understood that the positional relationship of each device in FIG1 is only for illustration and is not specifically limited in the embodiments of the present application.
本申请实施例不具体限定线缆接口的具体实现形式,例如存储扩展设备可以包括一个16Lane CXL线缆接口200,也可以包括两个8Lane CXL线缆接口200。The embodiment of the present application does not specifically limit the specific implementation form of the cable interface. For example, the storage expansion device may include one 16Lane CXL cable interface 200 or two 8Lane CXL cable interfaces 200.
本申请实施例具体不限定线缆接口200的具体类型,例如可以为光电转换接口,即将CXL控制器的电信号转换为光信号向外传输,也可以将外部的光信号转换为电信号向CXL控制器传输。CXL控制器可以访问DIMM中的数据,可以读数据,也可以写数据。The embodiment of the present application does not specifically limit the specific type of the cable interface 200, for example, it can be an optical-electric conversion interface, that is, converting the electrical signal of the CXL controller into an optical signal for external transmission, or converting the external optical signal into an electrical signal for transmission to the CXL controller. The CXL controller can access the data in the DIMM, and can read data or write data.
服务器可以通过PCIE接口100访问对应的第一CXL控制器A扩展的所有内存空间。即,当存储扩展设备包括两个CXL控制器时,服务器可以通过PCIE接口100访问的第一CXL控制器A扩展的4根DIMM,以及第二CXL控制器B扩展的4根DIMM,即服务器可以通过PCIE接口100访问存储扩展设备上的8根DIMM。The server can access all memory spaces expanded by the corresponding first CXL controller A through the PCIE interface 100. That is, when the storage expansion device includes two CXL controllers, the server can access the 4 DIMMs expanded by the first CXL controller A and the 4 DIMMs expanded by the second CXL controller B through the PCIE interface 100, that is, the server can access the 8 DIMMs on the storage expansion device through the PCIE interface 100.
例如,第一CXL控制器的X8接口连接PCIE接口100,第二CXL控制器的X8接口也连接PCIE接口100。PCIE接口100包括X16接口,其中,服务器通过PCIE接口100的X8接口可以访问第一CXL控制器连接的所有DIMM,服务器通过PCIE接口100的另外X8接口可以访问第二CXL控制器连接的所有DIMM。应该理解,第一CXL控制器连接4根DIMM,第二CXL控制器连接4根DIMM,即服务器通过PCIE接口100总共可以访问存储扩展设备上的8根DIMM。For example, the X8 interface of the first CXL controller is connected to the PCIE interface 100, and the X8 interface of the second CXL controller is also connected to the PCIE interface 100. The PCIE interface 100 includes an X16 interface, wherein the server can access all DIMMs connected to the first CXL controller through the X8 interface of the PCIE interface 100, and the server can access all DIMMs connected to the second CXL controller through another X8 interface of the PCIE interface 100. It should be understood that the first CXL controller is connected to 4 DIMMs, and the second CXL controller is connected to 4 DIMMs, that is, the server can access a total of 8 DIMMs on the storage expansion device through the PCIE interface 100.
可以理解的,PCIE接口的两个X8接口分别对应PCIE接口的一半引脚,也就是说,两个X8接口是通过将PCIE接口进行引脚划分得到的。当然,在其他实施例中,存储扩展设备可以包括两个实体的X8的PCIE接口。It is understandable that the two X8 interfaces of the PCIE interface correspond to half of the pins of the PCIE interface, that is, the two X8 interfaces are obtained by dividing the pins of the PCIE interface. Of course, in other embodiments, the storage expansion device may include two physical X8 PCIE interfaces.
本申请实施例提供的存储扩展设备包括支持分叉功能的CXL控制器,该CXL控制器支持至少2个DDR通道,每个通道至少支持2DPC,即单个CXL控制器可以支持4根DDR DIMM。CXL控制器的一半CXL接口连接到线缆接口,CXL控制器的另一半CXL接口连接到金手指,对外进行数据交互。其中金手指可以是以PCIE标卡的形式外出,即可以垂直插接在背板上。The storage expansion device provided in the embodiment of the present application includes a CXL controller supporting a bifurcation function, the CXL controller supports at least 2 DDR channels, each channel supports at least 2DPC, that is, a single CXL controller can support 4 DDR DIMMs. Half of the CXL interface of the CXL controller is connected to the cable interface, and the other half of the CXL interface of the CXL controller is connected to the gold finger for external data exchange. The gold finger can be in the form of a PCIE standard card, that is, it can be vertically plugged into the backplane.
本实施例中的存储扩展设备可以通过线缆接口和PCIE接口分别连接不同的两个计算设备,从而两个计算设备可以共享存储扩展设备的内存空间。The storage expansion device in this embodiment can be connected to two different computing devices through a cable interface and a PCIE interface respectively, so that the two computing devices can share the memory space of the storage expansion device.
本申请实施例提供的存储扩展设备,服务器通过PCIE接口即金手指可以访问CXL控制器连接的所有DIMM的内存,通过线缆接口也可以访问CXL控制器连接的所有DIMM所有的内存,实现了较少的接口资源访问高密度内存的需求,例如可以利用X16接口扩展8根DIMM的内存密度,实现利用较少的接口可以访问更多的内存,从而节省接口数量,并且提高单个存储扩展设备的内存密度。The storage expansion device provided in the embodiment of the present application is a server that can access the memory of all DIMMs connected to the CXL controller through the PCIE interface, i.e., the gold finger, and can also access all the memory of all DIMMs connected to the CXL controller through the cable interface, thereby realizing the need to access high-density memory with fewer interface resources. For example, the memory density of 8 DIMMs can be expanded using the X16 interface, so that more memory can be accessed using fewer interfaces, thereby saving the number of interfaces and improving the memory density of a single storage expansion device.
图1是以一个存储扩展设备包括两个CXL控制器为例进行的介绍,应该理解,存储扩展设备也可以仅包括一个CXL控制器,一个CXL控制器连接4根DIMM,通过金手指或线缆接口可以访问4根DIMM,金手指或线缆接口访问4根DIMM仅需要X8接口即可,与传统技术相比,访问相同数量的DIMM,接口的引脚数量可以减少一半。FIG1 is an introduction using an example of a storage expansion device including two CXL controllers. It should be understood that the storage expansion device may also include only one CXL controller. One CXL controller is connected to four DIMMs. The four DIMMs can be accessed through a gold finger or cable interface. The gold finger or cable interface only requires an X8 interface to access the four DIMMs. Compared with traditional technologies, the number of interface pins can be reduced by half to access the same number of DIMMs.
一种可能的实现方式,本申请实施例CXL控制器通过地址映射实现外部设备(例如服务器)通过线缆接口或金手指均能够访问CXL控制器扩展的所有内存空间。即CXL控制器,用于将扩展的所有内存空间的地址均映射到第一部分CXL接口和第二部分CXL接口。In a possible implementation, the CXL controller of the embodiment of the present application enables external devices (such as servers) to access all memory spaces expanded by the CXL controller through cable interfaces or gold fingers through address mapping. That is, the CXL controller is used to map the addresses of all expanded memory spaces to the first part of the CXL interface and the second part of the CXL interface.
一种可能的实现方式,CXL控制器中设置地址映射表,地址映射表可以映射出CXL控制器扩展的所有内存空间。地址映射表可以包括两个偏移量,例如,通过第一部分CXL接口访问CXL控制器扩展的所有内存空间时,利用第一偏移量来计算地址映射关系。通过第二部分CXL接口访问CXL控制器扩展的所有内存空间时,利用第二偏移量来计算地址映射关系。In a possible implementation, an address mapping table is set in the CXL controller, and the address mapping table can map all memory spaces extended by the CXL controller. The address mapping table can include two offsets. For example, when accessing all memory spaces extended by the CXL controller through the first part of the CXL interface, the first offset is used to calculate the address mapping relationship. When accessing all memory spaces extended by the CXL controller through the second part of the CXL interface, the second offset is used to calculate the address mapping relationship.
CXL控制器内部可以完成扩展的所有内存空间与服务器(计算设备)的CPU之间的地址映射,CXL控制器内部负责计算设备的物理地址到存储扩展设备的物理地址的映射,例如计算设备为服务器,此处计算设备的物理地址可以为主板CPU的物理地址,例如存储扩展设备的存储空间为64G,则存储扩展设备相对应CPU读取的物理地址为201G-264G对应的地址,即CXL控制器内部进行物理地址的偏移,使服务器的CPU读取存储扩展设备时进行物理地址的偏移。The CXL controller can complete the address mapping between all the expanded memory spaces and the CPU of the server (computing device). The CXL controller is responsible for mapping the physical address of the computing device to the physical address of the storage expansion device. For example, if the computing device is a server, the physical address of the computing device here can be the physical address of the motherboard CPU. For example, if the storage space of the storage expansion device is 64G, then the physical address read by the storage expansion device corresponding to the CPU is the address corresponding to 201G-264G, that is, the physical address is offset inside the CXL controller, so that the server CPU performs a physical address offset when reading the storage expansion device.
例如,第一CXL控制器存储有第一映射表和第二映射表,第一映射表与第一CXL控制器的第一部分CXL接口对应,第二映射表与第一CXL控制器的第二部分CXL接口对应,第一映射可以实现第一部分CXL接口接收的第一服务器(例如连接线缆接口的服务器)内存物理地址到第一CXL控制器的所有DIMM(例如1-4号DIMM)内存物理地址的映射转换,该内存物理地址的映射转换可以遵循CXL spec或自定义的方式,例如:通过地址偏移进行计算。For example, the first CXL controller stores a first mapping table and a second mapping table, the first mapping table corresponds to a first part of the CXL interface of the first CXL controller, and the second mapping table corresponds to a second part of the CXL interface of the first CXL controller. The first mapping can implement a mapping conversion of a memory physical address of a first server (e.g., a server connected to a cable interface) received by the first part of the CXL interface to a memory physical address of all DIMMs (e.g., DIMMs 1 to 4) of the first CXL controller. The mapping conversion of the memory physical address can comply with the CXL spec or a custom method, for example, by calculating through an address offset.
第二映射表可以实现第二部分CXL接口接收的第二服务器(例如连接PCIE接口的服务器)内存物理地址到第一CXL控制器的所有DIMM(例如1-4号DIMM)内存物理地址的映射转换,该内存物理地址的映射转换可以遵循CXL协议或自定义的方式,例如:通过地址偏移进行计算。The second mapping table can implement the mapping conversion of the memory physical address of the second server (for example, a server connected to the PCIE interface) received by the second part CXL interface to the memory physical addresses of all DIMMs (for example, DIMMs 1-4) of the first CXL controller. The mapping conversion of the memory physical address can follow the CXL protocol or a custom method, for example: calculation by address offset.
鉴于第一部分CXL接口和第二部分CXL接口可能连接到不同的服务器,而服务器有不同的系统地址空间,因此第一部分CXL接口和第二部分CXL接口接收到的内存物理地址可能不同,但是需要转化出来的物理地址是相同的。同时,在CXL控制器能够保证两个映射表转换出来的地址可以访问到CXL控制器连接的所有DIMM,并由CXL控制器保证正确的访问顺序。Since the first CXL interface and the second CXL interface may be connected to different servers, and the servers have different system address spaces, the memory physical addresses received by the first CXL interface and the second CXL interface may be different, but the physical addresses to be converted are the same. At the same time, the CXL controller can ensure that the addresses converted from the two mapping tables can access all DIMMs connected to the CXL controller, and the CXL controller guarantees the correct access order.
同理,第二CXL控制器存储第三映射表和第四映射表,第三映射表与第二CXL控制器的第一部分CXL接口对应,第四映射表与第二CXL控制器的第二部分CXL接口对应,第三映射表可以实现第一部分CXL接口接收到的第一服务器(例如连接线缆接口的服务器)内存物理地址到第二CXL控制器上所有DIMM(例如5-8号DIMM)内存物理地址的映射转换,该内存物理地址的映射转换可以遵循CXL spec或自定义的方式,例如:通过地址偏移进行计算。Similarly, the second CXL controller stores a third mapping table and a fourth mapping table. The third mapping table corresponds to the first part of the CXL interface of the second CXL controller, and the fourth mapping table corresponds to the second part of the CXL interface of the second CXL controller. The third mapping table can realize the mapping conversion of the memory physical address of the first server (for example, the server connected to the cable interface) received by the first part of the CXL interface to the memory physical address of all DIMMs (for example, DIMMs 5-8) on the second CXL controller. The mapping conversion of the memory physical address can follow the CXL spec or a custom method, for example: calculated by address offset.
第四映射表可以实现第二部分CXL接口接收的第二服务器(例如连接PCIE接口的服务器)内存物理地址到第二CXL控制器上所有DIMM(例如5-8号DIMM)内存物理地址的映射转换,该内存物理地址的映射转换可以遵循CXL协议或自定义的方式,例如:通过地址偏移进行计算。The fourth mapping table can implement the mapping conversion of the memory physical address of the second server (for example, a server connected to the PCIE interface) received by the second part CXL interface to the memory physical addresses of all DIMMs (for example, DIMMs 5-8) on the second CXL controller. The mapping conversion of the memory physical address can follow the CXL protocol or a custom method, for example: calculation by address offset.
鉴于第一部分CXL接口和第二部分CXL接口可能连接到不同的服务器,而服务器有不同的系统地址空间,因此第一部分CXL接口和第二部分CXL接口接收的内存物理地址可能不同,但是需要转化出来的物理地址是相同的。同时,在CXL控制器能够保证两个映射表转换出来的地址可以访问到CXL控制器连接的所有,并由CXL控制器保证正确的访问顺序。需要说明的是,当第一服务器通过第一CXL控制器的第一部分CXL接口向第一CXL控制器发送读写操作指令时,第一CXL控制器基于第一映射表确认与读写操作指令包括的目的地址,并对于目的地址进行相应的读写操作。当第一服务器通过第二CXL控制器的第一部分CXL接口向第二CXL控制器发送读写操作指令时,第二CXL控制器基于第三映射表确认与读写操作指令包括的目的地址,并对于目的地址进行相应的读写操作。In view that the first part of the CXL interface and the second part of the CXL interface may be connected to different servers, and the servers have different system address spaces, the memory physical addresses received by the first part of the CXL interface and the second part of the CXL interface may be different, but the physical addresses that need to be converted are the same. At the same time, the CXL controller can ensure that the addresses converted by the two mapping tables can access all the addresses connected to the CXL controller, and the CXL controller guarantees the correct access order. It should be noted that when the first server sends a read-write operation instruction to the first CXL controller through the first part of the CXL interface of the first CXL controller, the first CXL controller confirms the destination address included in the read-write operation instruction based on the first mapping table, and performs corresponding read-write operations on the destination address. When the first server sends a read-write operation instruction to the second CXL controller through the first part of the CXL interface of the second CXL controller, the second CXL controller confirms the destination address included in the read-write operation instruction based on the third mapping table, and performs corresponding read-write operations on the destination address.
当第二服务器通过第一CXL控制器的第二部分CXL接口向第一CXL控制器发送读写操作指令时,第一CXL控制器基于第二映射表确认与读写操作指令包括的目的地址,并对于目的地址进行相应的读写操作。当第二服务器通过第二CXL控制器的第二部分CXL接口向第二CXL控制器发送读写操作指令时,第二CXL控制器基于第四映射表确认与读写操作指令包括的目的地址,并对于目的地址进行相应的读写操作。When the second server sends a read/write operation instruction to the first CXL controller through the second part of the CXL interface of the first CXL controller, the first CXL controller confirms the destination address included in the read/write operation instruction based on the second mapping table, and performs the corresponding read/write operation on the destination address. When the second server sends a read/write operation instruction to the second CXL controller through the second part of the CXL interface of the second CXL controller, the second CXL controller confirms the destination address included in the read/write operation instruction based on the fourth mapping table, and performs the corresponding read/write operation on the destination address.
由于CXL控制器的CXL接口包括第一部分CXL接口和第二部分CXL接口,每个CXL接口可以访问与CXL控制器连接的所有内存,每个接口都可以通过对应的地址映射表实现与CXL控制器连接的所有内存空间的访问,因为CXL控制器中的地址映射可以将所有的扩展内存映射到CXL控制器的任一部分CXL接口,即每部分CXL接口可以访问所有的CXL内存空间,即所有的DIMM,从而与线缆接口连接的服务器可以通过第一CXL控制器的一部分接口和第二CXL控制器的一部分接口访问存储扩展设备的所有内存空间,与PCIE接口连接的服务器可以通过第一CXL控制器的另一部分接口和第二CXL控制器的另一部分接口访问存储扩展设备的所有内存空间Since the CXL interface of the CXL controller includes a first part of the CXL interface and a second part of the CXL interface, each CXL interface can access all memories connected to the CXL controller, and each interface can access all memory spaces connected to the CXL controller through a corresponding address mapping table, because the address mapping in the CXL controller can map all extended memories to any part of the CXL interface of the CXL controller, that is, each part of the CXL interface can access all CXL memory spaces, that is, all DIMMs, so that a server connected to the cable interface can access all memory spaces of the storage expansion device through a part of the interface of the first CXL controller and a part of the interface of the second CXL controller, and a server connected to the PCIE interface can access all memory spaces of the storage expansion device through another part of the interface of the first CXL controller and another part of the interface of the second CXL controller
所述CXL接口为X16接口;所述CXL控制器的X16接口被平分为的第一部分CXL接口和第二部分CXL接口,第一部分CXL接口和第二部分CXL接口均为X8接口。The CXL interface is an X16 interface; the X16 interface of the CXL controller is equally divided into a first part CXL interface and a second part CXL interface, and both the first part CXL interface and the second part CXL interface are X8 interfaces.
所述第一部分CXL接口连接所述线缆接口。The first portion CXL interface is connected to the cable interface.
所述第二部分CXL接口连接所述PCIE接口。The second part CXL interface is connected to the PCIE interface.
图1中是以存储扩展设备包括线缆接口和PCIE接口,下面介绍存储扩展设备仅包括PCIE接口的情况。In FIG1 , the storage expansion device includes a cable interface and a PCIE interface. The following describes a case where the storage expansion device only includes a PCIE interface.
参见图2,该图为本申请实施例提供的另一种存储扩展设备的示意图。Refer to Figure 2, which is a schematic diagram of another storage expansion device provided in an embodiment of the present application.
图2所示的存储扩展设备与图1所示的存储扩展设备相同的是,均包括两个CXL控制器,每个CXL控制器连接对应的四根DIMM。而且图1和图2中的PCIE接口100均设置在基板的长边,相同部分不再赘述。The storage expansion device shown in Figure 2 is the same as the storage expansion device shown in Figure 1 in that both include two CXL controllers, each of which is connected to four corresponding DIMMs. Moreover, the PCIE interface 100 in Figures 1 and 2 is arranged on the long side of the substrate, and the same parts are not repeated here.
图2所示的存储扩展设备与图1所示的存储扩展设备不同的是,图2所示的存储扩展设备仅包括PCIE接口100,可以不包括线缆接口。并且图2所示的存储扩展设备,PCIE接口100设置为标准的PCIE AIC形态,即金手指可以是以PCIE标卡的形式外出,垂直插接在计算设备的背板上。The storage expansion device shown in FIG2 is different from the storage expansion device shown in FIG1 in that the storage expansion device shown in FIG2 only includes a PCIE interface 100 and may not include a cable interface. In addition, the storage expansion device shown in FIG2 has a PCIE interface 100 configured as a standard PCIE AIC form, that is, the gold finger may be externally provided in the form of a PCIE standard card and vertically plugged into the backplane of the computing device.
参见图3,该图为本申请实施例提供的又一种存储扩展设备的示意图。Refer to Figure 3, which is a schematic diagram of another storage expansion device provided in an embodiment of the present application.
图3所示的存储扩展设备与图1所示的存储扩展设备相同的是,均包括两个CXL控制器,每个CXL控制器连接对应的四根DIMM。而且图1和图3中的线缆接口200均设置在基板的短边,相同部分不再赘述。The storage expansion device shown in Figure 3 is the same as the storage expansion device shown in Figure 1 in that both include two CXL controllers, each CXL controller is connected to four corresponding DIMMs. Moreover, the cable interface 200 in Figures 1 and 3 is arranged on the short side of the substrate, and the same parts are not repeated here.
图3所示的存储扩展设备与图1所示的存储扩展设备不同的是,图3所示的存储扩展设备仅包括线缆接口200,可以不包括PCIE接口。图3所示的存储扩展设备的线缆接口200可以通过外部线缆连接服务器,例如存储扩展设备的线缆接口200可以通过光纤连接服务器等。The storage expansion device shown in FIG3 is different from the storage expansion device shown in FIG1 in that the storage expansion device shown in FIG3 only includes a cable interface 200 and may not include a PCIE interface. The cable interface 200 of the storage expansion device shown in FIG3 may be connected to a server via an external cable, for example, the cable interface 200 of the storage expansion device may be connected to a server via an optical fiber, etc.
下面介绍本申请实施例提供的存储扩展设备的PCIE接口还可以设置为插卡形式,即整个存储扩展设备可以做抽屉式的拉出和推入,例如可以插入服务器的背板上。下面结合附图进行详细介绍。The PCIE interface of the storage expansion device provided in the embodiment of the present application can also be set as a plug-in card, that is, the entire storage expansion device can be pulled out and pushed in like a drawer, for example, it can be inserted into the backplane of the server.
参见图4A,该图为本申请实施例提供的再一种存储扩展设备的示意图。See Figure 4A, which is a schematic diagram of another storage expansion device provided in an embodiment of the present application.
图4A所示的存储扩展设备与图1所示的存储扩展设备相同的是,均包括两个CXL控制器,每个CXL控制器连接对应的四根DIMM,并且均包括线缆接口200和PCIE接口100,相同部分不再赘述。The storage expansion device shown in FIG. 4A is the same as the storage expansion device shown in FIG. 1 in that both include two CXL controllers, each CXL controller is connected to corresponding four DIMMs, and both include a cable interface 200 and a PCIE interface 100 , and the same parts are not repeated here.
图4A所示的存储扩展设备与图1所示的存储扩展设备不同的是,图4A中是以存储扩展设备设置为插卡形式,主要体现在PCIE接口100设置为插卡形式外出,可以插接服务器的背板上。而且图1的PCIE接口100设置在基板的长边,图4A的PCIE接口100设置在基板的短边。The storage expansion device shown in FIG4A is different from the storage expansion device shown in FIG1 in that the storage expansion device in FIG4A is configured as a plug-in card, which is mainly reflected in that the PCIE interface 100 is configured as a plug-in card and can be plugged into the backplane of the server. Moreover, the PCIE interface 100 of FIG1 is arranged on the long side of the substrate, while the PCIE interface 100 of FIG4A is arranged on the short side of the substrate.
每个CXL控制器连接的4根DIMM平行于基板的长边设置,应该理解,图4A中各个器件的位置关系仅是示意,本申请实施例中不做具体限定。The four DIMMs connected to each CXL controller are arranged parallel to the long side of the substrate. It should be understood that the positional relationship of the various components in FIG. 4A is only for illustration and is not specifically limited in the embodiments of the present application.
由于PCIE接口100设置在基板的短边。当存储扩展设备设置为插卡形式时,存储扩展设备可能出现晃动,不够稳定,因此,存储扩展设备上设置固定支撑的结构,以稳定其的状态,不出现晃动。例如,存储扩展设备可以设置多个固定结构,分部在存储扩展设备的不同位置。例如一种可能的实现方式,具体可以参见图4B所示,该图为本申请实施例提供的又一种存储扩展设备的示意图。Since the PCIE interface 100 is arranged on the short side of the substrate, when the storage expansion device is set to a plug-in card form, the storage expansion device may shake and be unstable. Therefore, a fixed support structure is arranged on the storage expansion device to stabilize its state and prevent shaking. For example, the storage expansion device may be provided with a plurality of fixed structures, which are distributed at different positions of the storage expansion device. For example, a possible implementation method can be specifically shown in FIG. 4B, which is a schematic diagram of another storage expansion device provided in an embodiment of the present application.
本申请实施例提供的计算设备还包括支撑板M,M上设置三个固定结构,对应内存扩展设备三个固定结构形成三角形的三个顶点进行布局,三个固定位置分别为O、P和Q为例进行介绍,具体实现时,可以利用螺钉或铆钉,将存储扩展设备和支撑板固定在一起,支撑板可以位于基板背向DIMM的一侧,以使支撑板固定支撑存储扩展设备。例如支撑板为金属板,存储扩展设备的基板通过固定支撑的结构固定在金属板上,起到强度支撑作用。本申请实施例不具体限定固定结构的数量,可以根据实际需要来设置。The computing device provided in the embodiment of the present application also includes a support plate M, on which three fixed structures are arranged corresponding to the three vertices of a triangle formed by the three fixed structures of the memory expansion device. The three fixed positions are respectively O, P and Q for example. In specific implementation, the storage expansion device and the support plate can be fixed together by screws or rivets. The support plate can be located on the side of the substrate facing away from the DIMM so that the support plate can fix and support the storage expansion device. For example, the support plate is a metal plate, and the substrate of the storage expansion device is fixed on the metal plate by a fixed support structure to play a role in strength support. The embodiment of the present application does not specifically limit the number of fixed structures, and can be set according to actual needs.
参见图5,该图为本申请实施例提供的另一种存储扩展设备的示意图。See FIG. 5 , which is a schematic diagram of another storage expansion device provided in an embodiment of the present application.
图5所示的存储扩展设备与图4A所示的存储扩展设备相同的是,均包括两个CXL控制器,每个CXL控制器连接对应的四根DIMM。PCIE接口100设置在基板的短边,相同部分不再赘述。The storage expansion device shown in Fig. 5 is the same as the storage expansion device shown in Fig. 4A in that both include two CXL controllers, each of which is connected to four corresponding DIMMs. The PCIE interface 100 is arranged on the short side of the substrate, and the same parts are not repeated here.
图4所示的存储扩展设备与图4A所示的存储扩展设备不同的是,图5所示的存储扩展设备仅包括PCIE接口100,并且,PCIE接口100设置为插卡形态,即金手指可以是以插卡形态水平插接在服务器的背板上。The storage expansion device shown in FIG. 4 is different from the storage expansion device shown in FIG. 4A in that the storage expansion device shown in FIG. 5 only includes a PCIE interface 100, and the PCIE interface 100 is configured as a plug-in card, that is, the gold finger can be horizontally plugged into the backplane of the server in the form of a plug-in card.
下面结合附图详细介绍本申请实施例提供的存储扩展设备的实现形式。The implementation form of the storage expansion device provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.
参见图6,该图为本申请实施例提供的又一种存储扩展设备的示意图。See FIG. 6 , which is a schematic diagram of another storage expansion device provided in an embodiment of the present application.
本申请实施例提供的存储扩展设备的形态为PCIE标卡形态,全高全长。该存储扩展设备可以提供直流12V电源接口。与图1相同的是,该存储扩展设备包括第一CXL控制器A和第二CXL控制器B,第一CXL控制器A连接对应的4根DIMM,第二CXL控制器B连接对应的4根DIMM。整个存储扩展设备可以扩展8根DIMM,相同部分不再赘述。The storage expansion device provided in the embodiment of the present application is in the form of a PCIE standard card, full height and full length. The storage expansion device can provide a DC 12V power supply interface. Similar to Figure 1, the storage expansion device includes a first CXL controller A and a second CXL controller B, the first CXL controller A is connected to the corresponding 4 DIMMs, and the second CXL controller B is connected to the corresponding 4 DIMMs. The entire storage expansion device can expand 8 DIMMs, and the same parts will not be repeated.
存储扩展设备的基板侧面设置有金手指接口,金手指接口为X16接口,符合PCIE标准规范。A gold finger interface is provided on the side of the baseboard of the storage expansion device. The gold finger interface is an X16 interface and complies with the PCIE standard specification.
本申请实施例提供的存储扩展设备以包括两个X8的线缆接口为例,即第一线缆接口201为X8接口,第二线缆接口202为X8接口。第一线缆接口201连接第一CXL控制器A,第二线缆接口202连接第二CXL控制器B。第一线缆接口201访问第一CXL控制器A连接的4根DIMM,第二线缆接口202访问第二CXL控制器B连接的4根DIMM。应该理解,图6中也可以仅包括一个线缆接口,该线缆接口为X16接口。The storage expansion device provided in the embodiment of the present application takes two X8 cable interfaces as an example, that is, the first cable interface 201 is an X8 interface, and the second cable interface 202 is an X8 interface. The first cable interface 201 is connected to the first CXL controller A, and the second cable interface 202 is connected to the second CXL controller B. The first cable interface 201 accesses the 4 DIMMs connected to the first CXL controller A, and the second cable interface 202 accesses the 4 DIMMs connected to the second CXL controller B. It should be understood that FIG. 6 may also include only one cable interface, which is an X16 interface.
本申请实施例提供的存储扩展设备,还包括设置在基板上的第一SPI闪存C和第二SPI闪存D。The storage expansion device provided in the embodiment of the present application also includes a first SPI flash memory C and a second SPI flash memory D arranged on the substrate.
第一CXL控制器A连接第一SPI闪存C。The first CXL controller A is connected to the first SPI flash memory C.
第二CXL控制器B连接第二SPI闪存D。The second CXL controller B is connected to the second SPI flash memory D.
第一SPI闪存C和第二SPI闪存D均为电子式可清除程序化只读存储器的形式,允许SPI数据的多次擦或写,由于可以擦写,因此可以重复利用。The first SPI flash memory C and the second SPI flash memory D are both in the form of electronic erasable programmable read-only memory, allowing multiple erasure or writing of SPI data. Since they can be erased and written, they can be reused.
本申请实施例提供的存储扩展设备,还包括设置在基板上的调试Debug接口;第一CXL控制器A和第二CXL控制器B均连接Debug接口。Debug接口可以连接外部设备,以实现对存储扩展设备的第一CXL控制器A和第二CXL控制器B测试或者调试。应该理解,第一CXL控制器A和第二CXL控制器B均可以通过联合测试工作组(Joint Test Action Group,JTAG)接口或通用异步收发传输器(Universal Asynchronous Receiver/Transmitter,UART)接口连接Debug接口。The storage expansion device provided in the embodiment of the present application further includes a debugging Debug interface provided on the substrate; the first CXL controller A and the second CXL controller B are both connected to the Debug interface. The Debug interface can be connected to an external device to implement testing or debugging of the first CXL controller A and the second CXL controller B of the storage expansion device. It should be understood that the first CXL controller A and the second CXL controller B can both be connected to the Debug interface via a Joint Test Action Group (JTAG) interface or a Universal Asynchronous Receiver/Transmitter (UART) interface.
本申请实施例提供的存储扩展设备,还包括设置在基板上的系统管理总线SMbus开关接口F和传感器E。传感器E用于检测存储扩展设备的温度。The storage expansion device provided in the embodiment of the present application further includes a system management bus SMbus switch interface F and a sensor E arranged on the substrate. The sensor E is used to detect the temperature of the storage expansion device.
SMbus开关接口F的第一端连接第一CXL控制器A和第二CXL控制器B;A first end of the SMbus switch interface F is connected to the first CXL controller A and the second CXL controller B;
SMbus开关接口F的第二端连接传感器E;SMbus开关接口F的第三端用于连接服务器。例如,SMbus开关接口F可以用于连接服务器。服务器可以通过SMbus开关接口F实现对存储扩展设备上的CXL控制器的管理。The second end of the SMbus switch interface F is connected to the sensor E. The third end of the SMbus switch interface F is used to connect to the server. For example, the SMbus switch interface F can be used to connect to the server. The server can manage the CXL controller on the storage expansion device through the SMbus switch interface F.
一种可能的实现方式,服务器通过连接SMbus开关接口,通过SMbus开关接口F访问传感器E、第一CXL控制器A和第二CXL控制器B中的至少一项。另外,服务器可以通过SMbus开关接口F同时访问传感器E、第一CXL控制器A和第二CXL控制器B。服务器访问第一CXL控制器A和第二CXL控制器B,可以访问第一CXL控制器A和第二CXL控制器B扩展的DIMM中的数据。In a possible implementation, the server is connected to the SMbus switch interface and accesses at least one of the sensor E, the first CXL controller A and the second CXL controller B through the SMbus switch interface F. In addition, the server can simultaneously access the sensor E, the first CXL controller A and the second CXL controller B through the SMbus switch interface F. The server accesses the first CXL controller A and the second CXL controller B, and can access data in the DIMMs expanded by the first CXL controller A and the second CXL controller B.
第一线缆接口201、第二线缆接口202和PCIE接口100提供的接口除了包括CXL数据读写引脚以外,CXL接口还包括时钟引脚CLK、复位引脚RST、SMbus引脚、在位引脚PRSNT和电源引脚。也就是说,SMbus开关接口F的第三端可以与线缆接口连接,或PCIE接口连接。本申请实施例中以电源引脚的电压为3.3V为例进行说明。In addition to the CXL data read and write pins, the interfaces provided by the first cable interface 201, the second cable interface 202 and the PCIE interface 100 also include a clock pin CLK, a reset pin RST, an SMbus pin, a position pin PRSNT and a power pin. In other words, the third end of the SMbus switch interface F can be connected to the cable interface or the PCIE interface. In the embodiment of the present application, the voltage of the power pin is 3.3V as an example for explanation.
第一CXL控制器A通过DDR控制器连接对应的4根DIMM,并通过集成电路互连通信电路(Inter-Integrated Circuit,I2C)或改进型I2C总线接口(Improved Inter IntegratedCircuit,I3C)获取对应的4根DIMM的(Serial Presence Detect,SPD)信息。SPD信息包括DIMM的内存大小、带宽等,便于根据内存的大小进行物理地址的映射。同理,第二CXL控制器B通过DDR控制器连接对应的4根DIMM,并通过I2C/I3C获取对应的4根DIMM的串行存在检测SPD信息。The first CXL controller A is connected to the corresponding four DIMMs through the DDR controller, and obtains the (Serial Presence Detect, SPD) information of the corresponding four DIMMs through the integrated circuit interconnect communication circuit (Inter-Integrated Circuit, I2C) or the improved I2C bus interface (Improved Inter Integrated Circuit, I3C). The SPD information includes the memory size and bandwidth of the DIMM, which is convenient for mapping the physical address according to the memory size. Similarly, the second CXL controller B is connected to the corresponding four DIMMs through the DDR controller, and obtains the serial presence detection SPD information of the corresponding four DIMMs through I2C/I3C.
本申请实施例提供的存储扩展设备可以通过服务器刷新第一CXL控制器A的固件Firmware和第二CXL控制器B的Firmware,修改其内部地址映射单元,使得第一CXL控制器A和第二CXL控制器B的CXL接口平分后的所有接口均可以访问与对应CXL控制连接的所有内存空间,从而通过线缆接口和PCIe接口均可以整个存储扩展设备的CXL内存空间,即访问所有DIMM。The storage expansion device provided in the embodiment of the present application can refresh the firmware of the first CXL controller A and the firmware of the second CXL controller B through a server, and modify the internal address mapping unit thereof, so that all interfaces after the CXL interfaces of the first CXL controller A and the second CXL controller B are equally divided can access all memory spaces connected to the corresponding CXL controllers, so that the entire CXL memory space of the storage expansion device, that is, all DIMMs, can be accessed through the cable interface and the PCIe interface.
图6所示的存储扩展设备设置第一线缆接口201、第二线缆接口202和PCIE接口100,其中,PCIE接口100设置在基板的长边,第一线缆接口201和第二线缆接口202设置在基板的短边。每个CXL控制器连接对应的四根DIMM。The storage expansion device shown in Fig. 6 is provided with a first cable interface 201, a second cable interface 202 and a PCIE interface 100, wherein the PCIE interface 100 is provided on the long side of the substrate, and the first cable interface 201 and the second cable interface 202 are provided on the short side of the substrate. Each CXL controller is connected to four corresponding DIMMs.
另一种实现方式,本申请实施例提供的存储扩展设备可以仅包括PCIE接口,不包括线缆接口,下面结合附图继续详细介绍。In another implementation, the storage expansion device provided in the embodiment of the present application may only include a PCIE interface, but not a cable interface, which will be described in detail below in conjunction with the accompanying drawings.
参见图7,该图为本申请实施例提供的又一种存储扩展设备的示意图。See FIG. 7 , which is a schematic diagram of another storage expansion device provided in an embodiment of the present application.
图7所示的存储扩展设备与图6所示的存储扩展设备相同的是,均包括两个CXL控制器,每个CXL控制器连接对应的四根DIMM。PCIE接口100设置在基板的长边,相同部分不再赘述。The storage expansion device shown in Fig. 7 is the same as the storage expansion device shown in Fig. 6 in that both include two CXL controllers, each of which is connected to four corresponding DIMMs. The PCIE interface 100 is arranged on the long side of the substrate, and the same parts are not repeated here.
图7所示的存储扩展设备与图6所示的存储扩展设备不同的是,本申请实施例提供的存储扩展设备包括PCIE接口100,不包括线缆接口。该PCIE接口100作为PCIE标卡形态,可以垂直插接在服务器的背板上。The storage expansion device shown in Figure 7 is different from the storage expansion device shown in Figure 6 in that the storage expansion device provided in the embodiment of the present application includes a PCIE interface 100, but does not include a cable interface. The PCIE interface 100 is in the form of a PCIE standard card and can be vertically plugged into the backplane of the server.
第一CXL控制器A的8Lane CXL总线连接PCIE接口100,第二CXL控制器B的8LaneCXL总线连接PCIE接口100。因此,通过PCIE接口100可以访问第一CXL控制器A扩展的4根DIMM,又可以访问第二CXL控制器B扩展的4根DIMM。The 8Lane CXL bus of the first CXL controller A is connected to the PCIE interface 100, and the 8Lane CXL bus of the second CXL controller B is connected to the PCIE interface 100. Therefore, the 4 DIMMs expanded by the first CXL controller A and the 4 DIMMs expanded by the second CXL controller B can be accessed through the PCIE interface 100.
图7所示的存储扩展设备包括PCIE接口,另外,本申请实施例提供的存储扩展设备可以仅包括线缆接口,不包括PCIE接口,在此不再赘述。The storage expansion device shown in FIG. 7 includes a PCIE interface. In addition, the storage expansion device provided in the embodiment of the present application may only include a cable interface without a PCIE interface, which will not be described in detail here.
下面结合附图详细介绍当存储扩展设备包括PCIE接口,但是PCIE接口作为插卡形态的实现方式。The following describes in detail with reference to the accompanying drawings how the storage expansion device includes a PCIE interface, but the PCIE interface is implemented as a plug-in card.
参见图8,该图为本申请实施例提供的另一种存储扩展设备的示意图。See FIG. 8 , which is a schematic diagram of another storage expansion device provided in an embodiment of the present application.
图8所示的存储扩展设备与图6所示的存储扩展设备相同的是,均包括两个CXL控制器,每个CXL控制器连接对应的四根DIMM,相同部分不再赘述。The storage expansion device shown in FIG. 8 is the same as the storage expansion device shown in FIG. 6 in that both include two CXL controllers, each CXL controller is connected to corresponding four DIMMs, and the same parts are not repeated here.
比较图8与图6可以看出,两者的区别是,图8中PCIE接口100设置在基板的短边。图8中的PCIE接口100设置为插卡形态,即金手指可以是以插卡形态水平插接在服务器的背板上。Comparing FIG8 with FIG6 , it can be seen that the difference between the two is that the PCIE interface 100 in FIG8 is arranged on the short side of the substrate. The PCIE interface 100 in FIG8 is arranged in a plug-in card form, that is, the gold finger can be horizontally plugged into the backplane of the server in a plug-in card form.
同时,图8提供的存储扩展设备包括第一线缆接口201和第二线缆接口202,具体描述可以参见图6对应的描述,在此不再赘述。Meanwhile, the storage expansion device provided in FIG. 8 includes a first cable interface 201 and a second cable interface 202 . For detailed description, please refer to the corresponding description in FIG. 6 , which will not be repeated here.
图8所示的存储扩展设备包括线缆接口和PCIE接口,另外,本申请实施例提供的存储扩展设备可以仅包括PCIE接口,不包括线缆接口,下面结合附图继续详细介绍。The storage expansion device shown in Figure 8 includes a cable interface and a PCIE interface. In addition, the storage expansion device provided in the embodiment of the present application may only include a PCIE interface without a cable interface, which will be described in detail below in conjunction with the accompanying drawings.
参见图9,该图为本申请实施例提供的再一种存储扩展设备的示意图。See Figure 9, which is a schematic diagram of another storage expansion device provided in an embodiment of the present application.
图9提供的存储扩展设备与图8提供的存储扩展设备的区别是,图9中提供的存储扩展设备不包括线缆接口,仅包括PCIE接口100。The difference between the storage expansion device provided in FIG. 9 and the storage expansion device provided in FIG. 8 is that the storage expansion device provided in FIG. 9 does not include a cable interface but only includes a PCIE interface 100 .
本申请实施例提供的存储扩展设备包括PCIE接口100,该PCIE接口100设置为插卡形态,即金手指可以是以插卡形态水平插接在服务器的背板上。The storage expansion device provided in the embodiment of the present application includes a PCIE interface 100, which is configured as a plug-in card. That is, the gold finger can be horizontally plugged into the backplane of the server in the form of a plug-in card.
第一CXL控制器A的X8接口连接PCIE接口100,第二CXL控制器B的X8接口连接PCIE接口100。因此,通过PCIE接口100可以访问第一CXL控制器A扩展的4根DIMM,又可以访问第二CXL控制器B扩展的4根DIMM。The X8 interface of the first CXL controller A is connected to the PCIE interface 100, and the X8 interface of the second CXL controller B is connected to the PCIE interface 100. Therefore, the four DIMMs expanded by the first CXL controller A and the four DIMMs expanded by the second CXL controller B can be accessed through the PCIE interface 100.
本申请实施例提供的存储扩展设备上的CXL控制器支持Bifurcation,并且通过地址映射单元可以实现线缆接口或PCIE接口访问存储扩展设备上的所有内存空间。例如,可以实现X16的接口连接8个DIMM的密度内存提升,并且支持金手指或线缆接口,服务器只需连接金手指或线缆接口中一个接口便可以访问所有的内存空间。The CXL controller on the storage expansion device provided in the embodiment of the present application supports Bifurcation, and the cable interface or PCIE interface can access all memory spaces on the storage expansion device through the address mapping unit. For example, the density memory improvement of connecting 8 DIMMs with the X16 interface can be realized, and the gold finger or cable interface is supported. The server only needs to connect one of the gold finger or cable interfaces to access all memory spaces.
并且,本申请实施例提供的存储扩展设备可以实现在计算设备的机箱内进行插拔,使得CXL内存卡易于维护,且使得厂商不受限于E3.S形态的内存颗粒供应。Furthermore, the storage expansion device provided in the embodiment of the present application can be plugged in and out of the chassis of the computing device, making the CXL memory card easy to maintain and allowing manufacturers to not be restricted to the supply of E3.S-shaped memory particles.
以上实施例仅是以CXL控制器连接4根DIMM为例进行介绍,应该理解,CXL控制器也可以扩展更多数量的DIMM,无论一个CXL控制器扩展的DIMM数量多少,使用本申请实施例提供的技术方案,线缆接口或PCIE接口均可以访问每个CXL控制器扩展的所有DIMM,实现通过存储扩展设备的一个接口便可以访问存储扩展设备上所有CXL内存的作用,从而提高了内存密度,节省成本。The above embodiment is only introduced by taking the CXL controller connecting 4 DIMMs as an example. It should be understood that the CXL controller can also expand a larger number of DIMMs. Regardless of the number of DIMMs expanded by a CXL controller, using the technical solution provided in the embodiment of the present application, the cable interface or the PCIE interface can access all DIMMs expanded by each CXL controller, so that all CXL memories on the storage expansion device can be accessed through one interface of the storage expansion device, thereby improving memory density and saving costs.
另外,本申请实施例具体不限定一个存储扩展设备上设置的CXL控制器的数量,以上实施例以两个CXL控制器为例进行介绍,应该理解,其他实施例中存储扩展设备上也可以集成更多数量的CXL控制器,进而一个存储扩展设备上可以扩展更多数量的DIMM,例如集成4个CXL控制器,则可以集成16根DIMM,通过线缆接口或PCIE接口可以访问16根DIMM的内存空间。以上仅是举例说明,应该理解,存储扩展设备上也可以设置奇数数量的CXL控制器,在此不做具体限定。In addition, the embodiments of the present application do not specifically limit the number of CXL controllers set on a storage expansion device. The above embodiments are introduced by taking two CXL controllers as an example. It should be understood that in other embodiments, a larger number of CXL controllers can be integrated on the storage expansion device, and then a larger number of DIMMs can be expanded on a storage expansion device. For example, if 4 CXL controllers are integrated, 16 DIMMs can be integrated, and the memory space of 16 DIMMs can be accessed through the cable interface or PCIE interface. The above is only an example, and it should be understood that an odd number of CXL controllers can also be set on the storage expansion device, which is not specifically limited here.
基于以上实施例提供的一种存储扩展设备,本申请实施例还提供一种计算设备,下面结合附图进行详细介绍。本申请实施例不具体限定计算设备的具体类型,例如计算设备可以为服务器。Based on the storage expansion device provided in the above embodiment, the present application embodiment further provides a computing device, which is described in detail below in conjunction with the accompanying drawings. The present application embodiment does not specifically limit the specific type of the computing device, for example, the computing device may be a server.
参见图10,该图为本申请实施例提供的一种计算设备的示意图。See FIG. 10 , which is a schematic diagram of a computing device provided in an embodiment of the present application.
本申请实施例提供的计算设备,包括:背板2000和以上任意一个实施例介绍的存储扩展设备3000;The computing device provided in the embodiment of the present application includes: a backplane 2000 and a storage expansion device 3000 introduced in any one of the above embodiments;
存储扩展设备3000连接背板2000。The storage expansion device 3000 is connected to the backplane 2000 .
应该理解,本申请实施例不具体限定计算设备内部包括的存储扩展设备3000的数量,可以为一个,也可以为多个,可以根据实际场景来设置。It should be understood that the embodiment of the present application does not specifically limit the number of storage expansion devices 3000 included in the computing device, which may be one or more and can be set according to actual scenarios.
而且本申请实施例提供的计算设备,不具体限定存储扩展设备3000与背板2000的连接方式,以上实施例已经介绍,存储扩展设备3000包括的PCIE接口可以设置为PCIE标卡形态,存储扩展设备可以垂直插接在背板2000上。另外,存储扩展设备3000包括的PCIE接口也可以设置为插卡形态,存储扩展设备3000可以水平插接在背板2000上。Moreover, the computing device provided in the embodiment of the present application does not specifically limit the connection mode between the storage expansion device 3000 and the backplane 2000. As described in the above embodiment, the PCIE interface included in the storage expansion device 3000 can be set to a PCIE standard card form, and the storage expansion device can be vertically plugged into the backplane 2000. In addition, the PCIE interface included in the storage expansion device 3000 can also be set to a plug-in card form, and the storage expansion device 3000 can be horizontally plugged into the backplane 2000.
应该理解,图10中以PCIE接口为PCIE标卡形态,图10所示的存储扩展设备3000为示意图,如果垂直插接在背板2000上,即存储扩展设备3000与背板2000之间互相垂直,垂直插接后存储扩展设备3000与背板2000之间有遮挡,为了理解背板2000余存储扩展设备3000的连接关系,图10仅是示意,并不代表实际的物理形态和布局。It should be understood that in FIG10 , the PCIE interface is in the form of a PCIE standard card, and the storage expansion device 3000 shown in FIG10 is a schematic diagram. If it is vertically plugged into the backplane 2000, that is, the storage expansion device 3000 and the backplane 2000 are perpendicular to each other, there is an obstruction between the storage expansion device 3000 and the backplane 2000 after vertical plugging. In order to understand the connection relationship between the backplane 2000 and the storage expansion device 3000, FIG10 is only a schematic diagram and does not represent the actual physical form and layout.
本申请实施例提供的计算设备,通过PCIE接口即金手指可以访问所有的内存,通过线缆接口也可以访问所有的内存,实现了较少的接口资源访问高密度内存的需求,例如可以利用X16接口扩展8根DIMM的内存密度,实现利用较少的接口可以访问更多的内存,从而节省接口数量,并且提高单个存储扩展设备的内存密度,进而提升计算设备的内存密度。The computing device provided in the embodiment of the present application can access all memories through the PCIE interface, i.e., the gold finger, and can also access all memories through the cable interface, thereby realizing the demand for accessing high-density memory with fewer interface resources. For example, the memory density of 8 DIMMs can be expanded using the X16 interface, so that more memory can be accessed using fewer interfaces, thereby saving the number of interfaces and improving the memory density of a single storage expansion device, thereby improving the memory density of the computing device.
以上所述,仅是本申请的较佳实施例而已,并非对本申请作任何形式上的限制。虽然本申请已以较佳实施例揭露如上,然而并非用以限定本申请。任何熟悉本领域的技术人员,在不脱离本申请技术方案范围情况下,都可利用上述揭示的方法和技术内容对本申请技术方案做出许多可能的变动和修饰,或修改为等同变化的等效实施例。因此,凡是未脱离本申请技术方案的内容,依据本申请的技术实质对以上实施例所做的任何简单修改、等同变化及修饰,均仍属于本申请技术方案保护的范围内。The above is only a preferred embodiment of the present application and does not constitute any formal limitation to the present application. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any technician familiar with the art can make many possible changes and modifications to the technical solution of the present application using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present application, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still falls within the scope of protection of the technical solution of the present application.
Claims (10)
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410129849.2A CN117992374A (en) | 2024-01-29 | 2024-01-29 | Storage expansion device and computing device |
| PCT/CN2025/074975 WO2025162322A1 (en) | 2024-01-29 | 2025-01-24 | Storage expansion device and computing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410129849.2A CN117992374A (en) | 2024-01-29 | 2024-01-29 | Storage expansion device and computing device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117992374A true CN117992374A (en) | 2024-05-07 |
Family
ID=90900888
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410129849.2A Pending CN117992374A (en) | 2024-01-29 | 2024-01-29 | Storage expansion device and computing device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN117992374A (en) |
| WO (1) | WO2025162322A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119557246A (en) * | 2025-01-26 | 2025-03-04 | 山东浪潮科学研究院有限公司 | Data transmission method, device, equipment and medium for memory expansion module |
| WO2025162322A1 (en) * | 2024-01-29 | 2025-08-07 | 超聚变数字技术有限公司 | Storage expansion device and computing device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180213669A1 (en) * | 2015-07-10 | 2018-07-26 | Prasad Lalathuputhanpura KOCHUKUNJU | Micro data center (mdc) in a box system and method thereof |
| CN112631954A (en) * | 2019-10-09 | 2021-04-09 | 联想企业解决方案(新加坡)有限公司 | Expandable dual inline memory module |
| CN114675722A (en) * | 2022-03-25 | 2022-06-28 | 苏州浪潮智能科技有限公司 | A memory expansion unit and a rack |
| CN115686153A (en) * | 2022-12-29 | 2023-02-03 | 浪潮电子信息产业股份有限公司 | Memory module and electronic equipment |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2022164414A (en) * | 2021-04-16 | 2022-10-27 | 株式会社日立製作所 | Storage device and address setting method |
| CN114020655A (en) * | 2022-01-05 | 2022-02-08 | 苏州浪潮智能科技有限公司 | A memory expansion method, device, device and storage medium |
| CN117009264B (en) * | 2023-09-13 | 2023-12-19 | 上海云豹创芯智能科技有限公司 | Method, system, chip and storage medium for realizing high-speed memory expansion in SOC |
| CN117992374A (en) * | 2024-01-29 | 2024-05-07 | 超聚变数字技术有限公司 | Storage expansion device and computing device |
-
2024
- 2024-01-29 CN CN202410129849.2A patent/CN117992374A/en active Pending
-
2025
- 2025-01-24 WO PCT/CN2025/074975 patent/WO2025162322A1/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180213669A1 (en) * | 2015-07-10 | 2018-07-26 | Prasad Lalathuputhanpura KOCHUKUNJU | Micro data center (mdc) in a box system and method thereof |
| CN112631954A (en) * | 2019-10-09 | 2021-04-09 | 联想企业解决方案(新加坡)有限公司 | Expandable dual inline memory module |
| CN114675722A (en) * | 2022-03-25 | 2022-06-28 | 苏州浪潮智能科技有限公司 | A memory expansion unit and a rack |
| CN115686153A (en) * | 2022-12-29 | 2023-02-03 | 浪潮电子信息产业股份有限公司 | Memory module and electronic equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2025162322A1 (en) * | 2024-01-29 | 2025-08-07 | 超聚变数字技术有限公司 | Storage expansion device and computing device |
| CN119557246A (en) * | 2025-01-26 | 2025-03-04 | 山东浪潮科学研究院有限公司 | Data transmission method, device, equipment and medium for memory expansion module |
| CN119557246B (en) * | 2025-01-26 | 2025-05-30 | 山东浪潮科学研究院有限公司 | Data transmission method, device, equipment and medium for memory expansion module |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2025162322A1 (en) | 2025-08-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117992374A (en) | Storage expansion device and computing device | |
| US10055379B2 (en) | Peripheral component interconnect express card | |
| US9128662B2 (en) | Solid state drive memory system | |
| TWI439843B (en) | Printed circuit assembly with automatic selection of storage configuration based on installed paddle board | |
| US8028404B2 (en) | Multi-function module | |
| US9535468B2 (en) | Server | |
| US7631134B2 (en) | Half-sized PCI central processing unit card and computer device having the capability of PCIe expansion | |
| CN107992438A (en) | A kind of server and in server flexible configuration PCIe topologys method | |
| CN106502593A (en) | A kind of mixing storage system of interconnection at a high speed | |
| EP1963977B1 (en) | Memory systems with memory chips down and up | |
| CN108090014A (en) | The storage IO casees system and its design method of a kind of compatible NVMe | |
| CN109947682A (en) | A server motherboard and server | |
| CN213276460U (en) | A dual-channel server motherboard and server | |
| CN117909273A (en) | A computing device | |
| CN213365380U (en) | Server mainboard and server | |
| CN211427338U (en) | Server mainboard based on explain majestic treaters | |
| WO2024250751A1 (en) | Memory module, electronic device, and data migration method | |
| CN104679172A (en) | Motherboard for supporting hybrid-type storage device | |
| CN117873914A (en) | Server and memory expansion assembly | |
| CN117349212A (en) | Server main board and solid state disk insertion detection method thereof | |
| CN213276462U (en) | Dual-socket server motherboard and dual-socket server | |
| CN116301572A (en) | Hard disk backboard assembling method, electronic equipment and medium | |
| CN104572518B (en) | A kind of storage device | |
| CN204189089U (en) | A kind of server | |
| CN222337623U (en) | A storage server motherboard |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |