Please note that the functionality discussed in this document is currently tagged as experimental.
Peer-2-Peer (P2P) is the concept of DMAing data directly from one PCI End Point (EP) to another without using a system memory buffer. The most obvious example of this from an SPDK perspective is using a NVMe Controller Memory Buffer (CMB) to enable direct copies of data between two NVMe SSDs.
In this section of documentation we outline how to perform P2P operations in SPDK and outline some of the issues that can occur when performing P2P operations.
The functions that provide access to the NVMe CMBs for P2P capabilities are given in the table below.
| Key Functions | Description |
|---|---|
| spdk_nvme_ctrlr_map_cmb() | Map a previously reserved controller memory buffer so that it's data is visible from the CPU. |
| spdk_nvme_ctrlr_unmap_cmb() | Free a controller memory I/O buffer. |
| spdk_nvme_ctrlr_get_regs_cmbsz() | Get the NVMe controller CMBSZ (Controller Memory Buffer Size) register. |
SPDK's identify example application displays whether a device has a controller memory buffer and which operations it supports. Run it as follows:
Run the cmb_copy example application.
This should copy a single LBA (LBA 0) from namespace 1 on the read NVMe SSD to LBA 0 on namespace 1 on the write SSD using the CMB as the DMA buffer.