lkml.org 
[lkml]   [2019]   [Jan]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Subject[PATCH 0/5] sched/fair: NOHZ cleanups and misfit improvement
Date
In

commit 5fbdfae5221a ("sched/fair: Kick nohz balance if rq->misfit_task_load")

was added a trigger for nohz kicks, which is required to offload misfit tasks
from LITTLE to big CPUs. However, those kicks could be issued a lot more
frequently than what is strictly needed.

This patch-set brings some unrelated nohz code cleanups, and tunes down
unneeded nohz kicks.

- Patches [1-3] do some cleaning up of the current nohz code
- Patches [4-5] tweak the nohz kick conditions for asymmetric systems

* Testing
** kick_ilb() hits
This causes a large reduction in calls to kick_ilb() (and thus subsequent
rescheduling interrupts & useless nohz balance calls) in most scenarios.

The "best case" one is running NR_BIG_CPUS big tasks, which I tested with
4 50% periodic tasks running for 5 seconds on my HiKey960 (4x4 big.LITTLE):

| CPU | hits (baseline) | hits (patchset) |
|-----+-----------------+-----------------|
| 0 | 31 | 41 |
| 1 | 21 | 3 |
| 2 | 35 | 2 |
| 3 | 9 | 4 |
|-----+-----------------+-----------------|
| 4 | 170 | 4 |
| 5 | 573 | 4 |
| 6 | 544 | 4 |
| 7 | 579 | 4 |


Something a bit less idealistic with NR_CPUS-1 big tasks still shows some
improvements (7 100% tasks running for 5 seconds on my HiKey960):

| CPU | hits (baseline) | hits (patchset) |
|-----+-----------------+-----------------|
| 0 | 14 | 122 |
| 1 | 47 | 162 |
| 2 | 11 | 156 |
| 3 | 9 | 3 |
|-----+-----------------+-----------------|
| 4 | 53 | 6 |
| 5 | 276 | 13 |
| 6 | 312 | 7 |
| 7 | 250 | 11 |

I was surprised to see such an increase in calls to kick_ilb() from LITTLE
CPUs ([0-3]), but after a bit of investigation it turns out that the big
CPUs would always run nohz_balancer_kick() a jiffy before the LITTLEs, so
the LITTLEs would always bail out because nohz.next_balance had just been
updated before they called nohz_balancer_kick(). IOW,

time_before(now, nohz.next_balance)

would always be true on CPUs [0-3] during my workload. Quieting the kicks
issued by the big CPUs allowed the LITTLEs to execute nohz_balancer_kick()
past that condition, explaining the higher number of kicks issued from LITTLE
CPUs.

** misfit behaviour
For good measure I also ran the usual misfit tests [1] which showed no
particular change.

* Notes

Since patch 05 introduces yet another label in kick_nohz_balancer(),
I benchmarked splitting its "slowpath" (all of the rcu derefs) into
its own function and didn't find any real overhead. If folks don't
hate the idea I can add it to a v2 or post it separately.

[1]: https://github.com/ARM-software/lisa/blob/next/lisa/tests/kernel/scheduler/misfit.py

Valentin Schneider (5):
sched/fair: Use for_each_cpu_and for asym-packing nohz kicks
sched/fair: Explain LLC nohz kick condition
sched/fair: Prune nohz_balancer_kick() comment block
sched/fair: Tune down misfit nohz kicks
sched/fair: Skip LLC nohz logic for asymmetric systems

kernel/sched/fair.c | 62 ++++++++++++++++++++++++++++++++-------------
1 file changed, 44 insertions(+), 18 deletions(-)

--
2.20.1

\
 
 \ /
  Last update: 2019-01-17 16:38    [W:0.114 / U:1.100 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site