Messages in this thread |  | | From | Vincent Guittot <> | Date | Tue, 7 Jan 2020 10:43:08 +0100 | Subject | Re: [PATCH] sched, fair: Allow a small load imbalance between low utilisation SD_NUMA domains v3 |
| |
On Tue, 7 Jan 2020 at 10:12, Mel Gorman <mgorman@techsingularity.net> wrote: > > On Tue, Jan 07, 2020 at 09:51:11AM +0800, Hillf Danton wrote: > > > > Hi Folks > > > > On Mon, 06 Jan 2020 11:44:57 -0500 Rik van Riel wrote: > > > On Mon, 2020-01-06 at 16:33 +0000, Mel Gorman wrote: > > > > On Mon, Jan 06, 2020 at 10:47:18AM -0500, Rik van Riel wrote: > > > > > > + imbalance_adj = (100 / (env->sd->imbalance_pct - 100)) - 1; > > > > > > + > > > > > > + /* > > > > > > + * Allow small imbalances when the busiest group has > > > > > > + * low utilisation. > > > > > > + */ > > > > > > + imbalance_max = imbalance_adj << 1; > > > > > > + if (busiest->sum_nr_running < imbalance_max) > > > > > > + env->imbalance -= min(env->imbalance, imbalance_adj); > > > > > > + } > > > > > > + > > > > > > > > > > Wait, so imbalance_max is a function only of > > > > > env->sd->imbalance_pct, and it gets compared > > > > > against busiest->sum_nr_running, which is related > > > > > to the number of CPUs in the node? > > > > > > > > > > > > > It's not directly related to the number of CPUs in the node. Are you > > > > thinking of busiest->group_weight? > > > > > > I am, because as it is right now that if condition > > > looks like it might never be true for imbalance_pct 115. > > > > > > Presumably you put that check there for a reason, and > > > would like it to trigger when the amount by which a node > > > is busy is less than 2 * (imbalance_pct - 100). > > > > > > If three per cent can make any sense in helping determine utilisation > > low then the busy load has to meet > > > > busiest->sum_nr_running < max(3, cpus in the node / 32); > > > > Why 3% and why would the low utilisation cut-off depend on the number of
But in the same way, why only 6 tasks ? which is the value with default imbalance_pct ? I expect a machine with 128 CPUs to have more bandwidth than a machine with only 32 CPUs and as a result to allow more imbalance
Maybe the number of running tasks (or idle cpus) is not the right metrics to choose if we can allow a small degree of imbalance because this doesn't take into account it wether the tasks are long running or short running ones
> CPUs in the node? That simply means that the cut-off scales to machine > size and does not take into account any consideration between local memory > latency and memory bandwidth. > > > And we can't skip pulling tasks from a numa node without comparing it > > to the local load > > > > local->sum_nr_running * env->sd->imbalance_pct < > > busiest->sum_nr_running * 100; > > > > with imbalance_pct taken into account. > > > > Again, why? In this context, an imbalance has already been calculated > and whether based on running tasks or idle CPUs, it's not a negative > number. The imbalance_adj used as already accounted for imbalance_pct > albeit not as a ratio as it's normally used. > > -- > Mel Gorman > SUSE Labs
|  |