lkml.org 
[lkml]   [2022]   [Mar]   [8]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
SubjectRe: [PATCH] Documentation: x86: add documenation for AMD IOMMU
From
On 2022-03-08 19:04, Alex Deucher via iommu wrote:
> Add preliminary documenation for AMD IOMMU.
>
> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
> ---
> Documentation/x86/amd-iommu.rst | 85 +++++++++++++++++++++++++++++++
> Documentation/x86/index.rst | 1 +
> Documentation/x86/intel-iommu.rst | 2 +-
> 3 files changed, 87 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/x86/amd-iommu.rst
>
> diff --git a/Documentation/x86/amd-iommu.rst b/Documentation/x86/amd-iommu.rst
> new file mode 100644
> index 000000000000..89820140fefa
> --- /dev/null
> +++ b/Documentation/x86/amd-iommu.rst
> @@ -0,0 +1,85 @@
> +=================
> +AMD IOMMU Support
> +=================
> +
> +The architecture spec can be obtained from the below location.
> +
> +https://www.amd.com/system/files/TechDocs/48882_IOMMU.pdf
> +
> +This guide gives a quick cheat sheet for some basic understanding.
> +
> +Some Keywords
> +
> +- IVRS - I/O Virtualization Reporting Structure
> +- IVDB - I/O Virtualization Definition Block
> +- IVHD - I/O Virtualization Hardware Definition
> +- IOVA - I/O Virtual Address.
> +
> +Basic stuff
> +-----------
> +
> +ACPI enumerates and lists the different DMA engines in the platform, and
> +device scope relationships between PCI devices and which DMA engine controls
> +them.

"DMA engine" typically means a dedicated device for peripheral-to-memory
or memory-to-memory transfers, or the responsible block within a general
DMA-capable endpoint. In the context of the original Intel doc from
whence I see this is copied, this probably should have said "DMAR unit"
or similar; here I'd suggest picking your favourite vendor-appropriate
term for "instance of IOMMU translation hardware". Let's not promote
confusion more than necessary.

> +
> +What is IVRS?
> +-------------
> +
> +The architecture defines an ACPI-compatible data structure called an I/O
> +Virtualization Reporting Structure (IVRS) that is used to convey information
> +related to I/O virtualization to system software. The IVRS describes the
> +configuration and capabilities of the IOMMUs contained in the platform as
> +well as information about the devices that each IOMMU virtualizes.
> +
> +The IVRS provides information about the following:
> +- IOMMUs present in the platform including their capabilities and proper configuration
> +- System I/O topology relevant to each IOMMU
> +- Peripheral devices that cannot be otherwise enumerated
> +- Memory regions used by SMI/SMM, platform firmware, and platform hardware. These are
> +generally exclusion ranges to be configured by system software.
> +
> +How is IOVA generated?
> +----------------------
> +
> +Well behaved drivers call pci_map_*() calls before sending command to device

Horribly out-of-date drivers call pci_map_*(). Modern well-behaved
drivers call dma_map_*() ;)

> +that needs to perform DMA. Once DMA is completed and mapping is no longer
> +required, device performs a pci_unmap_*() calls to unmap the region.
> +
> +The AMD IOMMU driver allocates a virtual address per domain. Each PCIE
> +device has its own domain (hence protection). Devices under p2p bridges
> +share the virtual address with all devices under the p2p bridge due to
> +transaction id aliasing for p2p bridges.
> +
> +IOVA generation is pretty generic. We used the same technique as vmalloc()
> +but these are not global address spaces, but separate for each domain.
> +Different DMA engines may support different number of domains.

I'm not sure about this whole section, really - IOVA management was
entirely decoupled from drivers some time ago. If there's value in
having some overview documentation, then it's probably worth
consolidating into a common "IOMMU API" doc that can be cross-referenced
for a summary of domains, groups, and iommu_dma_ops.

> +
> +
> +Fault reporting
> +---------------
> +When errors are reported, the DMA engine signals via an interrupt. The fault

Again, here I instinctively read "DMA engine" as being the endpoint
device *making* the DMA transaction that faulted, and indeed that might
happen to raise its own error interrupt if it gets an unexpected abort
back from the IOMMU, which is coincidental to a thoroughly misleading
degree...

Thanks,
Robin.

> +reason and device that caused it with fault reason is printed on console.
> +
> +See below for sample.
> +
> +
> +Boot Message Sample
> +-------------------
> +
> +Something like this gets printed indicating presence of the IOMMU.
> +
> + iommu: Default domain type: Translated
> + iommu: DMA domain TLB invalidation policy: lazy mode
> +
> +
> +PCI-DMA: Using AMD IOMMU
> +------------------------
> +
> +Fault reporting
> +^^^^^^^^^^^^^^^
> +
> +::
> +
> + AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0007 address=0xffffc02000 flags=0x0000]
> + AMD-Vi: Event logged [IO_PAGE_FAULT device=07:00.0 domain=0x0007 address=0xffffc02000 flags=0x0000]
> +
> diff --git a/Documentation/x86/index.rst b/Documentation/x86/index.rst
> index f498f1d36cd3..15711134eb68 100644
> --- a/Documentation/x86/index.rst
> +++ b/Documentation/x86/index.rst
> @@ -22,6 +22,7 @@ x86-specific Documentation
> mtrr
> pat
> intel-iommu
> + amd-iommu
> intel_txt
> amd-memory-encryption
> pti
> diff --git a/Documentation/x86/intel-iommu.rst b/Documentation/x86/intel-iommu.rst
> index 099f13d51d5f..4d3391c7bd3f 100644
> --- a/Documentation/x86/intel-iommu.rst
> +++ b/Documentation/x86/intel-iommu.rst
> @@ -1,5 +1,5 @@
> ===================
> -Linux IOMMU Support
> +Intel IOMMU Support
> ===================
>
> The architecture spec can be obtained from the below location.

\
 
 \ /
  Last update: 2022-03-08 21:10    [W:0.053 / U:0.040 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site