[lkml]   [2018]   [Jan]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [PATCH v4 4/8] PCI: brcmstb: Add dma-range mapping for inbound traffic
On Mon, Jan 15, 2018 at 5:28 PM, Jim Quinlan <> wrote:
> The Broadcom STB PCIe host controller is intimately related to the
> memory subsystem. This close relationship adds complexity to how cpu
> system memory is mapped to PCIe memory. Ideally, this mapping is an
> identity mapping, or an identity mapping off by a constant. Not so in
> this case.
> Consider the Broadcom reference board BCM97445LCC_4X8 which has 6 GB
> of system memory. Here is how the PCIe controller maps the
> system memory to PCIe memory:
> memc0-a@[ 0....3fffffff] <=> pci@[ 0....3fffffff]
> memc0-b@[100000000...13fffffff] <=> pci@[ 40000000....7fffffff]
> memc1-a@[ 40000000....7fffffff] <=> pci@[ 80000000....bfffffff]
> memc1-b@[300000000...33fffffff] <=> pci@[ c0000000....ffffffff]
> memc2-a@[ 80000000....bfffffff] <=> pci@[100000000...13fffffff]
> memc2-b@[c00000000...c3fffffff] <=> pci@[140000000...17fffffff]
> Although there are some "gaps" that can be added between the
> individual mappings by software, the permutation of memory regions for
> the most part is fixed by HW. The solution of having something close
> to an identity mapping is not possible.
> The idea behind this HW design is that the same PCIe module can
> act as an RC or EP, and if it acts as an EP it concatenates all
> of system memory into a BAR so anything can be accessed. Unfortunately,
> when the PCIe block is in the role of an RC it also presents this
> "BAR" to downstream PCIe devices, rather than offering an identity map
> between its system memory and PCIe space.
> Suppose that an endpoint driver allocs some DMA memory. Suppose this
> memory is located at 0x6000_0000, which is in the middle of memc1-a.
> The driver wants a dma_addr_t value that it can pass on to the EP to
> use. Without doing any custom mapping, the EP will use this value for
> DMA: the driver will get a dma_addr_t equal to 0x6000_0000. But this
> won't work; the device needs a dma_addr_t that reflects the PCIe space
> address, namely 0xa000_0000.
> So, essentially the solution to this problem must modify the
> dma_addr_t returned by the DMA routines routines. There are two
> ways (I know of) of doing this:
> (a) overriding/redefining the dma_to_phys() and phys_to_dma() calls
> that are used by the dma_ops routines. This is the approach of
> arch/mips/cavium-octeon/dma-octeon.c

MIPS is rarely an example to follow. :)

> In ARM and ARM64 these two routines are defiend in asm/dma-mapping.h
> as static inline functions.
> (b) Subscribe to a notifier that notifies when a device is added to a
> bus. When this happens, set_dma_ops() can be called for the device.
> This method is mentioned in:

Why refer to an external website when you can just refer to the source
of the project this patch applies to directly.

> where it says as a comment
> "In case if platform code need to use own special DMA
> configuration, it can use Platform bus notifier and
> handle BUS_NOTIFY_ADD_DEVICE event to fix up DMA
> configuration."

In the current tree, this comment is in drivers/of/device.c.

> Solution (b) is what this commit does. It uses its own set of
> dma_ops which are wrappers around the arch_dma_ops. The
> wrappers translate the dma addresses before/after invoking
> the arch_dma_ops, as appropriate.

 \ /
  Last update: 2018-01-18 03:16    [W:0.119 / U:0.440 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site