Messages in this thread |  | | Date | Tue, 7 Jan 2020 09:12:56 +0000 | From | Mel Gorman <> | Subject | Re: [PATCH] sched, fair: Allow a small load imbalance between low utilisation SD_NUMA domains v3 |
| |
On Tue, Jan 07, 2020 at 09:51:11AM +0800, Hillf Danton wrote: > > Hi Folks > > On Mon, 06 Jan 2020 11:44:57 -0500 Rik van Riel wrote: > > On Mon, 2020-01-06 at 16:33 +0000, Mel Gorman wrote: > > > On Mon, Jan 06, 2020 at 10:47:18AM -0500, Rik van Riel wrote: > > > > > + imbalance_adj = (100 / (env->sd->imbalance_pct - 100)) - 1; > > > > > + > > > > > + /* > > > > > + * Allow small imbalances when the busiest group has > > > > > + * low utilisation. > > > > > + */ > > > > > + imbalance_max = imbalance_adj << 1; > > > > > + if (busiest->sum_nr_running < imbalance_max) > > > > > + env->imbalance -= min(env->imbalance, imbalance_adj); > > > > > + } > > > > > + > > > > > > > > Wait, so imbalance_max is a function only of > > > > env->sd->imbalance_pct, and it gets compared > > > > against busiest->sum_nr_running, which is related > > > > to the number of CPUs in the node? > > > > > > > > > > It's not directly related to the number of CPUs in the node. Are you > > > thinking of busiest->group_weight? > > > > I am, because as it is right now that if condition > > looks like it might never be true for imbalance_pct 115. > > > > Presumably you put that check there for a reason, and > > would like it to trigger when the amount by which a node > > is busy is less than 2 * (imbalance_pct - 100). > > > If three per cent can make any sense in helping determine utilisation > low then the busy load has to meet > > busiest->sum_nr_running < max(3, cpus in the node / 32); >
Why 3% and why would the low utilisation cut-off depend on the number of CPUs in the node? That simply means that the cut-off scales to machine size and does not take into account any consideration between local memory latency and memory bandwidth.
> And we can't skip pulling tasks from a numa node without comparing it > to the local load > > local->sum_nr_running * env->sd->imbalance_pct < > busiest->sum_nr_running * 100; > > with imbalance_pct taken into account. >
Again, why? In this context, an imbalance has already been calculated and whether based on running tasks or idle CPUs, it's not a negative number. The imbalance_adj used as already accounted for imbalance_pct albeit not as a ratio as it's normally used.
-- Mel Gorman SUSE Labs
|  |