lkml.org 
[lkml]   [2018]   [Nov]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
SubjectRe: [RFC] locking/rwsem: Avoid issuing wakeup before setting the reader waiter to nil
On Thu, 29 Nov 2018, Waiman Long wrote:

>That can be costly for x86 which will now have 2 locked instructions.

Yeah, and when used as an actual queue we should really start to notice.
Some users just have a single task in the wake_q because avoiding the cost
of wake_up_process() with locks held is significant.

How about instead of adding the barrier before the cmpxchg, we do it
in the failed branch, right before we return. This is the uncommon
path.

Thanks,
Davidlohr

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 091e089063be..0d844a18a9dc 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -408,8 +408,14 @@ void wake_q_add(struct wake_q_head *head, struct task_struct *task)
* This cmpxchg() executes a full barrier, which pairs with the full
* barrier executed by the wakeup in wake_up_q().
*/
- if (cmpxchg(&node->next, NULL, WAKE_Q_TAIL))
+ if (cmpxchg(&node->next, NULL, WAKE_Q_TAIL)) {
+ /*
+ * Ensure, that when the cmpxchg() fails, the corresponding
+ * wake_up_q() will observe our prior state.
+ */
+ smp_mb__after_atomic();
return;
+ }

get_task_struct(task);

\
 
 \ /
  Last update: 2018-11-29 22:30    [W:0.065 / U:20.772 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site