[lkml]   [2016]   [Nov]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [PATCH 0/5] Parallel hugepage migration optimization
On 24 Nov 2016, at 18:59, Balbir Singh wrote:

> On 23/11/16 03:25, Zi Yan wrote:
>> From: Zi Yan <>
>> Hi all,
>> This patchset boosts the hugepage migration throughput and helps THP migration
>> which is added by Naoya's patches:
>> Motivation
>> ===============================
>> In x86, 4KB page migrations are underutilizing the memory bandwidth compared
>> to 2MB THP migrations. I did some page migration benchmarking on a two-socket
>> Intel Xeon E5-2640v3 box, which has 23.4GB/s bandwidth, and discover
>> there are big throughput gap, ~3x, between 4KB and 2MB page migrations.
>> Here are the throughput numbers for different page sizes and page numbers:
>> | 512 4KB pages | 1 2MB THP | 1 4KB page
>> x86_64 | 0.98GB/s | 2.97GB/s | 0.06GB/s
>> As Linux currently use single-threaded page migration, the throughput is still
>> much lower than the hardware bandwidth, 2.97GB/s vs 23.4GB/s. So I parallelize
>> the copy_page() part of THP migration with workqueue and achieve 2.8x throughput.
>> Here are the throughput numbers of 2MB page migration:
>> | single-threaded | 8-thread
>> x86_64 2MB | 2.97GB/s | 8.58GB/s
> Whats the impact on CPU utilization? Is there a huge impact?
> Balbir Singh.

It depends on the throughput we can achieve.

For single-threaded copy, the current routine, it takes one CPU 2MB/(2.97GB/s) = 657.6 us
to copy one 2MB page.

For 8-thread copy, it take 8 CPUs 2MB/(8.58GB/s) = 227.6 us to copy one 2MB page.

If we have 8 idle CPUs, I think it worths using them.

I am going to add code to check idle_cpu() in the system before doing the copy.
If no idle CPUs are present, I can fall back to single-threaded copy.

Best Regards
Yan Zi
[unhandled content-type:application/pgp-signature]
 \ /
  Last update: 2016-11-28 16:38    [W:0.180 / U:0.072 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site