[lkml]   [2020]   [Feb]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: Applying pipe fix this merge window?
On Wed, Feb 12, 2020 at 12:03:26PM -0800, Linus Torvalds wrote:
> And I realized that I find it surprising that it makes your build
> times noticeably better.
> Yes, I have that silly example program to show the issue in the commit
> message, and yes, the exclusive directed write->read wakeups should
> most definitely improve by that commit.
> But the make jobserver code ends up using "poll()/pselect()" and
> non-blocking reads, because of how it handles the child death signals.
> Which means that none of the nice exclusive directed write->read
> wakeups should even trigger in the first place, because the readers
> never block, and he poll/pselect code doesn't use exclusive wakeups
> (because it can't - it doesn't actually consume the data).
> So I was looking at it, and going "it should actually not help GNU
> jobserver at all" in the fixed jobserver case.

I dug into this a little further yesterday and today:

- With hindsight, I realized that the performance improvements I
observed for GNU make didn't measure the pipe fix in isolation; they
measured 5.4 versus ~5.5-rc4 plus the pipe fix, which would include
all the other pipe work in 5.5 and potentially other optimizations.
*That* showed substantial performance improvements in GNU make, on the
order of a couple of seconds in a 30-60 second kernel build. ("5.5-rc4
plus pipe fix" is what I hammered on for a month on various systems.)

- Measuring the pipe fix patch in isolation
(0bf999f9c5e74c7ecf9dafb527146601e5c848b9, with and without the pipe
fix reverted, with nothing else changed), GNU make performance indeed
doesn't show any difference.

- Other things that use the GNU make jobserver (with pipe fds in
blocking mode) benefit much more heavily, in wall-clock time and in
total CPU time. I saw jobs that involved just a minute or two of
wall-clock time, where the total CPU time went down by *minutes*.

Hope that helps,
Josh Triplett

 \ /
  Last update: 2020-02-14 07:08    [W:0.054 / U:1.144 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site