[lkml]   [2019]   [Feb]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [LSF/MM ATTEND ] memory reclaim with NUMA rebalancing
On Thu, 31 Jan 2019, Aneesh Kumar K.V wrote:

> I would be interested in this topic too. I would like to
> understand the API and how it can help exploit the different type of
> devices we have on OpenCAPI.

So am I. We may want to rethink the whole NUMA API and the way we handle
different types of memory with their divergent performance

We need some way to allow a better selection of memory from the kernel
without creating too much complexity. We have new characteristics to

1. Persistence (NVRAM) or generally a storage device that allows access to
the medium via a RAM like interface.

2. Coprocessor memory that can be shuffled back and forth to a device

3. On Device memory (important since PCIe limitations are currently a
problem and Intel is stuck on PCIe3 and devices start to bypass the
processor to gain performance)

4. High Density RAM (GDDR f.e.) with different caching behavior
and/or different cacheline sizes.

5. Modifying access characteristics by reserving slice of a cache (f.e.
L3) for a specific memory region.

6. SRAM support (high speed memory on the processor itself or by using
the processor cache to persist a cacheline)

And then the old NUMA stuff where only the latency to memory varies. But
that was a particular solution targeted at scaling SMP system through
interconnects. This was a mostly symmetric approach. The use of
accellerators etc etc and the above characteristics lead to more complex
assymmetric memory approaches that may be difficult to manage and use from
kernel space.

 \ /
  Last update: 2019-02-06 20:04    [W:0.115 / U:0.768 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site