[lkml]   [2020]   [Jan]   [27]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [PATCH v9 0/5] Add NUMA-awareness to qspinlock
On 1/27/20 12:12 PM, Waiman Long wrote:
> On 1/27/20 10:09 AM, Paul E. McKenney wrote:
>>> Do you mean within the VM, lscpu showed that the system has one node and
>>> 12 threads per node? If that is the case, it should behave like regular
>>> qspinlock and be fair.
>> I mean that I saw this in dmesg, which I believe to be telling me the
>> same thing as lscpu saying that there is one node, but you tell me!
>> [ 0.007106] No NUMA configuration found
>> [ 0.007107] Faking a node at [mem 0x0000000000000000-0x000000001ffdefff]
>> [ 0.007111] NODE_DATA(0) allocated [mem 0x1ffdb000-0x1ffdefff]
>> [ 0.007126] Zone ranges:
>> [ 0.007127] DMA [mem 0x0000000000001000-0x0000000000ffffff]
>> [ 0.007128] DMA32 [mem 0x0000000001000000-0x000000001ffdefff]
>> [ 0.007128] Normal empty
>> [ 0.007129] Movable zone start for each node
>> [ 0.007129] Early memory node ranges
>> [ 0.007130] node 0: [mem 0x0000000000001000-0x000000000009efff]
>> [ 0.007132] node 0: [mem 0x0000000000100000-0x000000001ffdefff]
>> [ 0.007227] Zeroed struct page in unavailable ranges: 98 pages
>> [ 0.007227] Initmem setup node 0 [mem 0x0000000000001000-0x000000001ffdefff]
>> [ 0.007228] On node 0 totalpages: 130941
>> [ 0.007231] DMA zone: 64 pages used for memmap
>> [ 0.007231] DMA zone: 21 pages reserved
>> [ 0.007232] DMA zone: 3998 pages, LIFO batch:0
>> [ 0.007266] DMA32 zone: 1984 pages used for memmap
>> [ 0.007267] DMA32 zone: 126943 pages, LIFO batch:31
> That does look like just one node.
>>>> I will try reader-writer lock next.
>>>> Again, should I be using qemu's -numa command-line option to create nodes?
>>>> If so, what would be a sane configuration given 12 CPUs and 512MB of
>>>> memory for the VM? If not, what is a good way to exercise CNA's NUMA
>>>> capabilities within a guest OS?
>>> You can certainly play around with CNA in a VM. However, it is generally
>>> not recommended to use CNA in a VM unless the VM cpu topology matches
>>> the host with 1-to-1 vcpu pinning and there is no vcpu overcommit. In
>>> this case, one may see some performance improvement using CNA by using
>>> the "numa_spinlock=on" option to explicitly turn it on.
>> Sorry, but I will not be booting this on bare metal on the systems that
>> I currently have access to. No more than I run rcutorture on bare metal
>> on them, especially not with newly modified variants of RCU. ;-)
>>> Because of the shuffling of queue entries, CNA is inherently less fair
>>> than the regular qspinlock. However, a ratio of 5 seems excessive to me.
>>> vcpu preemption may be a factor in contributing to this large variation.
>>> My testing on bare metal only showed a throughput variation within
>>> 10-20% at most.
>> OK. Any guidance on qemu's -numa, or should I just experiment with it?
>> The latter will take me some time, as I must focus on other things
>> this week.
> To really test it, you should have multiple numa nodes, 2 or 4.
>> Alternatively, would it make sense for you to give it a spin in a VM?
>> After all, it is entirely possible that I still have some configuration
>> or another messed up.
> It all depends on what you want to test. If you want just to make sure
> that it won't fail any locking test. Yes, you can certainly do that in
> a VM. If you want to test for performance or fairness of CNA versus
> regular qspinlock, using a VM is not a proper platform. Even with a
> single node, you see 5x difference in locking count. I suspect that
> maybe one or a few vcpus got preempted pretty frequently to perform
> host activities that it screw up the data.
BTW, in a VM, the queued_spin_unlock function pointer will still be
pointing to the paravirt version even if CNA is used for the slowpath.
That shouldn't impact correctness, but is not optimal for performance.


 \ /
  Last update: 2020-01-27 18:18    [W:0.079 / U:0.028 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site