Messages in this thread |  | | Subject | Re: [GIT PULL] Introduce housekeeping subsystem v4 | From | Chris Metcalf <> | Date | Sat, 21 Oct 2017 12:07:26 -0400 |
| |
On 10/20/2017 10:29 AM, Frederic Weisbecker wrote: > 2017-10-20 10:17 UTC+02:00, Ingo Molnar <mingo@kernel.org>: >> I mean code like: >> >> triton:~/tip> git grep on_each_cpu mm >> mm/page_alloc.c: * cpu to drain that CPU pcps and on_each_cpu_mask >> mm/slab.c: on_each_cpu(do_drain, cachep, 1); >> mm/slub.c: on_each_cpu_cond(has_cpu_slab, flush_cpu_slab, s, 1, >> GFP_ATOMIC); >> mm/vmstat.c: err = schedule_on_each_cpu(refresh_vm_stats); >> >> is something we want to execute on 'housekeeping CPUs' as well, to not >> disturb the >> isolated CPUs, right? > I see, so indeed that's the kind of thing we want to also confine to > housekeeping as > well whenever possible but these cases require special treatment that need to be > handled by the subsystem in charge. For example vmstat has the vmstat_sheperd > thing which allows to drive those timers adaptively on demand to make sure that > userspace isn't interrupted. The others will likely need some similar treatment. > > For now I only see vmstat having such a feature and it acts > transparently. There is > also the LRU flush (IIRC) which needs to be called for example before > returning to > userspace to avoid IPIs. Such things may indeed need special treatment. With the > current patchset it could be a housekeeping flag.
I have been working to update the task isolation support the last few days and though it's not quite ready to post (probably will be Monday or Tuesday), I have sorted out those issues from task isolation's perspective. It turns out that you can both quiesce the vmstat_shepherd, as well as drain the LRU per-cpu pages, while interrupts are disabled on the way back to userspace.
Whether shifting this work to housekeeping cores at all times makes sense seems like a much more open question. The idea of task isolation is to provide a harder guarantee of isolation, and in particular to shift work to the moment that you return to userspace, rather than allowing it to happen later. It does seem likely that there are some things you'd want to do on the core itself most of the time, and just suppress for true task isolation if requested, rather than trying to move them to the housekeeping cores.
But, it's certainly worth looking at both options and seeing how it plays out. The less complicated the task isolation return-to-user path is, the better. (The idea of task isolation seems like a win no matter what, to allow ensuring kernel isolation when you absolutely require it.)
The current task isolation tree is in the "dataplane" branch at git://git.kernel.org/pub/scm/linux/kernel/git/cmetcalf/linux-tile.git
-- Chris Metcalf, Mellanox Technologies http://www.mellanox.com
|  |