lkml.org 
[lkml]   [2020]   [Sep]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: [PATCH] latency improvement in __smp_call_single_queue
From
Date
On 9/24/2020 3:42 AM, peterz@infradead.org wrote:
> On Wed, Sep 23, 2020 at 10:00:41AM -0500, George Prekas wrote:
>> If an interrupt arrives between llist_add and
>> send_call_function_single_ipi in the following code snippet, then the
>> remote CPU will not receive the IPI in a timely manner and subsequent
>> SMP calls even from other CPUs for other functions will be delayed:
>>
>>     if (llist_add(node, &per_cpu(call_single_queue, cpu)))
>>         send_call_function_single_ipi(cpu);
>>
>> Note: llist_add returns 1 if it was empty before the operation.
>>
>> CPU 0                           | CPU 1 | CPU 2
>> __smp_call_single_q(2,f1)       | __smp_call_single_q(2,f2) |
>>   llist_add returns 1           | |
>>   interrupted                   |   llist_add returns 0 |
>>       ...                       |   branch not taken |
>>       ...                       | |
>>   resumed                       | |
>>   send_call_function_single_ipi | |
>>                                 | | f1
>>                                 | | f2
>>
>> The call from CPU 1 for function f2 will be delayed because CPU 0 was
>> interrupted.
>
> Do you happen to have any actual numbers and a use-case where this was
> relevant?

Hi Peter,

I encountered this problem while developing a device driver that used
smp_call_function_single to communicate with other cores. I observed
latency spikes and after investigation I figured out the problem
described above.

I have written a simple device driver to validate the above fix. It does
smp_call_function_single and measures the latency. I can post it here if
it is appropriate. The latency impact is equal to the duration of the
CPU 0's interruption.

--
George

\
 
 \ /
  Last update: 2020-09-24 17:05    [W:0.051 / U:4.396 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site