Messages in this thread |  | | Subject | Re: [LKP] Re: [sched] bdfcae1140: will-it-scale.per_thread_ops -37.0% regression | From | Xing Zhengjun <> | Date | Tue, 20 Oct 2020 11:24:24 +0800 |
| |
On 10/7/2020 10:50 PM, Mathieu Desnoyers wrote: > ----- On Oct 2, 2020, at 4:33 AM, Rong Chen rong.a.chen@intel.com wrote: > >> Greeting, >> >> FYI, we noticed a -37.0% regression of will-it-scale.per_thread_ops due to >> commit: >> >> >> commit: bdfcae11403e5099769a7c8dc3262e3c4193edef ("[RFC PATCH 2/3] sched: >> membarrier: cover kthread_use_mm (v3)") >> url: >> https://github.com/0day-ci/linux/commits/Mathieu-Desnoyers/Membarrier-updates/20200925-012549 >> base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git >> 848785df48835eefebe0c4eb5da7690690b0a8b7 >> >> in testcase: will-it-scale >> on test machine: 104 threads Skylake with 192G memory >> with following parameters: >> >> nr_task: 50% >> mode: thread >> test: context_switch1 >> cpufreq_governor: performance >> ucode: 0x2006906 >> >> test-description: Will It Scale takes a testcase and runs it from 1 through to n >> parallel copies to see if the testcase will scale. It builds both a process and >> threads based test in order to see any differences between the two. >> test-url: https://github.com/antonblanchard/will-it-scale >> > > Hi, > > I would like to report what I suspect is a random thread placement issue in the > context_switch1 test used by the 0day bot when running on a machine with hyperthread > enabled. > > AFAIU the test code uses hwloc for thread placement which should theoretically ensure > that each thread is placed on same processing unit, core and numa node between runs. > > We can find the test code here: > > https://github.com/antonblanchard/will-it-scale/blob/master/tests/context_switch1.c > > And the main file containing thread setup is here: > > https://github.com/antonblanchard/will-it-scale/blob/master/main.c > > AFAIU, the test is started without the "-m" switch, which therefore affinitizes > tasks on cores rather than on processing units (SMT threads). > > When testcase() creates the child thread with new_task(), it basically issues: > > pthread_create(&threads[nr_threads++], NULL, func, arg); > > passing a NULL pthread_attr_t, and not executing any pre_trampoline on the child. > The pre_trampoline would have issued hwloc_set_thread_cpubind if it were executed on > the child, but it's not. Therefore, we expect the cpu affinity mask of the parent to > be copied on clone and used by the child. > > A quick test on a machine with hyperthreading enabled shows that the cpu affinity mask > for the parent and child has two bits set: > > taskset -p 1868607 > pid 1868607's current affinity mask: 10001 > taskset -p 1868606 > pid 1868606's current affinity mask: 10001 > > So AFAIU the placement of the parent and child will be random on either the same > processing unit, or on separate processing units within the same core. > > I suspect this randomness can significantly affect the performance number between > runs, and trigger unwarranted performance regression warnings. > > Thanks, > > Mathieu > Yes, the randomness may happen in some special cases. But in 0-day, we test multi times (>=3), the report is the average number. For this case, we test 4 times, it is stable, the wave is ± 2%. So I don't think the -37.0% regression is caused by the randomness.
0/stats.json: "will-it-scale.per_thread_ops": 105228, 1/stats.json: "will-it-scale.per_thread_ops": 100443, 2/stats.json: "will-it-scale.per_thread_ops": 98786, 3/stats.json: "will-it-scale.per_thread_ops": 102821,
c2daff748f0ea954 bdfcae11403e5099769a7c8dc32 ---------------- --------------------------- %stddev %change %stddev \ | \ 161714 ± 2% -37.0% 101819 ± 2% will-it-scale.per_thread_ops
-- Zhengjun Xing
|  |