[lkml]   [2019]   [Jan]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [RFC PATCH] drm/ttm: force cached mappings for system RAM on ARM
Am 14.01.19 um 20:13 schrieb Will Deacon:
> On Mon, Jan 14, 2019 at 07:07:54PM +0000, Koenig, Christian wrote:
>> Am 14.01.19 um 18:32 schrieb Ard Biesheuvel:
>> - The reason remapping the CPU side as cacheable does work (which I
>> did test) is because the GPU's uncacheable accesses (which I assume
>> are made using the NoSnoop PCIe transaction attribute) are actually
>> emitted as cacheable in some cases.
>> . On my AMD Seattle, with or without SMMU (which is stage 2 only), I
>> must use cacheable accesses from the CPU side or things are broken.
>> This might be a h/w flaw, though.
>> . On systems with stage 1+2 SMMUs, the driver uses stage 1
>> translations which always override the memory attributes to cacheable
>> for DMA coherent devices. This is what is affecting the Cavium
>> ThunderX2 (although it appears the attributes emitted by the RC may be
>> incorrect as well.)
>> The latter issue is a shortcoming in the SMMU driver that we have to
>> fix, i.e., it should take care not to modify the incoming attributes
>> of DMA coherent PCIe devices for NoSnoop to be able to work.
>> So in summary, the mismatch appears to be between the CPU accessing
>> the vmap region with non-cacheable attributes and the GPU accessing
>> the same memory with cacheable attributes, resulting in a loss of
>> coherency and lots of visible corruption.
>> Actually it is the other way around. The CPU thinks some data is in the
>> cache and the GPU only updates the system memory version because the
>> snoop flag is not set.
>> That doesn't seem to be what is happening. As far as we can tell from
>> our experiments, all inbound transactions are always cacheable, and so
>> the only way to make things work is to ensure that the CPU uses the
>> same attributes.
>> Ok that doesn't make any sense. If inbound transactions are cacheable or not is
>> irrelevant when the CPU always uses uncached accesses.
>> See on the PCIe side you have the snoop bit in the read/write transactions
>> which tells the root hub if the device wants to snoop caches or not.
>> When the CPU accesses some memory as cached then devices need to snoop the
>> cache for coherent accesses.
>> When the CPU accesses some memory as uncached then devices can disable snooping
>> to improve performance, but when they don't do this it is mandated by the spec
>> that this still works.
> Which spec?

The PCIe spec. The snoop bit (or rather the NoSnoop) in the transaction
is perfectly optional IIRC.

> The Arm architecture (and others including Power afaiu) doesn't
> guarantee coherency when memory is accessed using mismatched cacheability
> attributes.

Well what exactly goes wrong on ARM?

As far as I know Power doesn't really supports un-cached memory at all,
except for a very very old and odd configuration with AGP.

I mean in theory I agree that devices should use matching cacheability
attributes, but in practice I know of quite a bunch of devices/engines
which fails to do this correctly.


> Will

 \ /
  Last update: 2019-01-14 20:22    [W:0.068 / U:5.228 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site