[lkml]   [2019]   [Jan]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
Subject[LSF/MM TOPIC] Use NVDIMM as NUMA node and NUMA API
Hi folks,

I would like to attend the LSF/MM Summit 2019. I'm interested in most MM
topics, particularly the NUMA API topic proposed by Jerome since it is
related to my below proposal.

I would like to share some our usecases, needs and approaches about
using NVDIMM as a NUMA node.

We would like to provide NVDIMM to our cloud customers as some low cost
memory.  Virtual machines could run with NVDIMM as backed memory.  Then
we would like the below needs are met:

    * The ratio of DRAM vs NVDIMM is configurable per process, or even
per VMA
    * The user VMs alway get DRAM first as long as the ratio is not
    * Migrate cold data to NVDIMM and keep hot data in DRAM dynamically
and throughout the life time of VMs

To meet the needs we did some in-house implementation:
    * Provide madvise interface to configure the ratio
    * Put NVDIMM into a separate zonelist so that default allocation
can't touch it as long as it is requested explicitly
    * A kernel thread scans cold pages

We tried to just use current NUMA APIs, but we realized they can't meet
our needs.  For example, if we configure a VMA use 50% DRAM and 50%
NVDIMM, mbind() could set preferred node policy (DRAM node or NVDIMM
node) for this VMA, but it can't control how much DRAM or NVDIMM is used
by this specific VMA to satisfy the ratio.

So, IMHO we definitely need more fine-grained APIs to control the NUMA

I'd like also to discuss about this topic with:
    Dave Hansen
    Dan Williams
    Fengguang Wu

Other than the above topic, I'd also like to meet other MM developers to
discuss about some our usecases about memory cgroup (hallway
conversation may be good enough).  I had submitted some RFC patches to
the mailing list and they did incur some discussion, but we have not
reached solid conclusion yet.



 \ /
  Last update: 2019-01-30 18:29    [W:0.044 / U:9.484 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site