lkml.org 
[lkml]   [2018]   [Nov]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [RFCv3 PATCH 1/6] uacce: Add documents for WarpDrive/uacce
On Mon, Nov 19, 2018 at 11:48:54AM -0500, Jerome Glisse wrote:

> Just to comment on this, any infiniband driver which use umem and do
> not have ODP (here ODP for me means listening to mmu notifier so all
> infiniband driver except mlx5) will be affected by same issue AFAICT.
>
> AFAICT there is no special thing happening after fork() inside any of
> those driver. So if parent create a umem mr before fork() and program
> hardware with it then after fork() the parent might start using new
> page for the umem range while the old memory is use by the child. The
> reverse is also true (parent using old memory and child new memory)
> bottom line you can not predict which memory the child or the parent
> will use for the range after fork().
>
> So no matter what you consider the child or the parent, what the hw
> will use for the mr is unlikely to match what the CPU use for the
> same virtual address. In other word:
>
> Before fork:
> CPU parent: virtual addr ptr1 -> physical address = 0xCAFE
> HARDWARE: virtual addr ptr1 -> physical address = 0xCAFE
>
> Case 1:
> CPU parent: virtual addr ptr1 -> physical address = 0xCAFE
> CPU child: virtual addr ptr1 -> physical address = 0xDEAD
> HARDWARE: virtual addr ptr1 -> physical address = 0xCAFE
>
> Case 2:
> CPU parent: virtual addr ptr1 -> physical address = 0xBEEF
> CPU child: virtual addr ptr1 -> physical address = 0xCAFE
> HARDWARE: virtual addr ptr1 -> physical address = 0xCAFE

IIRC this is solved in IB by automatically calling
madvise(MADV_DONTFORK) before creating the MR.

MADV_DONTFORK
.. This is useful to prevent copy-on-write semantics from changing the
physical location of a page if the parent writes to it after a
fork(2) ..

Jason

\
 
 \ /
  Last update: 2018-11-19 19:29    [W:0.071 / U:8.396 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site