|
| 1 | +.. SPDX-License-Identifier: GPL-2.0 |
| 2 | +
|
| 3 | +
|
| 4 | +=========== |
| 5 | +TPH Support |
| 6 | +=========== |
| 7 | + |
| 8 | +:Copyright: 2024 Advanced Micro Devices, Inc. |
| 9 | +:Authors: - Eric van Tassell <eric.vantassell@amd.com> |
| 10 | + - Wei Huang <wei.huang2@amd.com> |
| 11 | + |
| 12 | + |
| 13 | +Overview |
| 14 | +======== |
| 15 | + |
| 16 | +TPH (TLP Processing Hints) is a PCIe feature that allows endpoint devices |
| 17 | +to provide optimization hints for requests that target memory space. |
| 18 | +These hints, in a format called Steering Tags (STs), are embedded in the |
| 19 | +requester's TLP headers, enabling the system hardware, such as the Root |
| 20 | +Complex, to better manage platform resources for these requests. |
| 21 | + |
| 22 | +For example, on platforms with TPH-based direct data cache injection |
| 23 | +support, an endpoint device can include appropriate STs in its DMA |
| 24 | +traffic to specify which cache the data should be written to. This allows |
| 25 | +the CPU core to have a higher probability of getting data from cache, |
| 26 | +potentially improving performance and reducing latency in data |
| 27 | +processing. |
| 28 | + |
| 29 | + |
| 30 | +How to Use TPH |
| 31 | +============== |
| 32 | + |
| 33 | +TPH is presented as an optional extended capability in PCIe. The Linux |
| 34 | +kernel handles TPH discovery during boot, but it is up to the device |
| 35 | +driver to request TPH enablement if it is to be utilized. Once enabled, |
| 36 | +the driver uses the provided API to obtain the Steering Tag for the |
| 37 | +target memory and to program the ST into the device's ST table. |
| 38 | + |
| 39 | +Enable TPH support in Linux |
| 40 | +--------------------------- |
| 41 | + |
| 42 | +To support TPH, the kernel must be built with the CONFIG_PCIE_TPH option |
| 43 | +enabled. |
| 44 | + |
| 45 | +Manage TPH |
| 46 | +---------- |
| 47 | + |
| 48 | +To enable TPH for a device, use the following function:: |
| 49 | + |
| 50 | + int pcie_enable_tph(struct pci_dev *pdev, int mode); |
| 51 | + |
| 52 | +This function enables TPH support for device with a specific ST mode. |
| 53 | +Current supported modes include: |
| 54 | + |
| 55 | + * PCI_TPH_ST_NS_MODE - NO ST Mode |
| 56 | + * PCI_TPH_ST_IV_MODE - Interrupt Vector Mode |
| 57 | + * PCI_TPH_ST_DS_MODE - Device Specific Mode |
| 58 | + |
| 59 | +`pcie_enable_tph()` checks whether the requested mode is actually |
| 60 | +supported by the device before enabling. The device driver can figure out |
| 61 | +which TPH mode is supported and can be properly enabled based on the |
| 62 | +return value of `pcie_enable_tph()`. |
| 63 | + |
| 64 | +To disable TPH, use the following function:: |
| 65 | + |
| 66 | + void pcie_disable_tph(struct pci_dev *pdev); |
| 67 | + |
| 68 | +Manage ST |
| 69 | +--------- |
| 70 | + |
| 71 | +Steering Tags are platform specific. PCIe spec does not specify where STs |
| 72 | +are from. Instead PCI Firmware Specification defines an ACPI _DSM method |
| 73 | +(see the `Revised _DSM for Cache Locality TPH Features ECN |
| 74 | +<https://members.pcisig.com/wg/PCI-SIG/document/15470>`_) for retrieving |
| 75 | +STs for a target memory of various properties. This method is what is |
| 76 | +supported in this implementation. |
| 77 | + |
| 78 | +To retrieve a Steering Tag for a target memory associated with a specific |
| 79 | +CPU, use the following function:: |
| 80 | + |
| 81 | + int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type type, |
| 82 | + unsigned int cpu_uid, u16 *tag); |
| 83 | + |
| 84 | +The `type` argument is used to specify the memory type, either volatile |
| 85 | +or persistent, of the target memory. The `cpu_uid` argument specifies the |
| 86 | +CPU where the memory is associated to. |
| 87 | + |
| 88 | +After the ST value is retrieved, the device driver can use the following |
| 89 | +function to write the ST into the device:: |
| 90 | + |
| 91 | + int pcie_tph_set_st_entry(struct pci_dev *pdev, unsigned int index, |
| 92 | + u16 tag); |
| 93 | + |
| 94 | +The `index` argument is the ST table entry index the ST tag will be |
| 95 | +written into. `pcie_tph_set_st_entry()` will figure out the proper |
| 96 | +location of ST table, either in the MSI-X table or in the TPH Extended |
| 97 | +Capability space, and write the Steering Tag into the ST entry pointed by |
| 98 | +the `index` argument. |
| 99 | + |
| 100 | +It is completely up to the driver to decide how to use these TPH |
| 101 | +functions. For example a network device driver can use the TPH APIs above |
| 102 | +to update the Steering Tag when interrupt affinity of a RX/TX queue has |
| 103 | +been changed. Here is a sample code for IRQ affinity notifier: |
| 104 | + |
| 105 | +.. code-block:: c |
| 106 | +
|
| 107 | + static void irq_affinity_notified(struct irq_affinity_notify *notify, |
| 108 | + const cpumask_t *mask) |
| 109 | + { |
| 110 | + struct drv_irq *irq; |
| 111 | + unsigned int cpu_id; |
| 112 | + u16 tag; |
| 113 | +
|
| 114 | + irq = container_of(notify, struct drv_irq, affinity_notify); |
| 115 | + cpumask_copy(irq->cpu_mask, mask); |
| 116 | +
|
| 117 | + /* Pick a right CPU as the target - here is just an example */ |
| 118 | + cpu_id = cpumask_first(irq->cpu_mask); |
| 119 | +
|
| 120 | + if (pcie_tph_get_cpu_st(irq->pdev, TPH_MEM_TYPE_VM, cpu_id, |
| 121 | + &tag)) |
| 122 | + return; |
| 123 | +
|
| 124 | + if (pcie_tph_set_st_entry(irq->pdev, irq->msix_nr, tag)) |
| 125 | + return; |
| 126 | + } |
| 127 | +
|
| 128 | +Disable TPH system-wide |
| 129 | +----------------------- |
| 130 | + |
| 131 | +There is a kernel command line option available to control TPH feature: |
| 132 | + * "notph": TPH will be disabled for all endpoint devices. |
0 commit comments