[lkml]   [2018]   [Feb]   [5]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified
Hello Pavel,

> Yes, the patch is here:

I tested your patch in my box and it worked well.
Please feel free to add the following.

Tested-by: Masayoshi Mizuma <>

You may repost the patch after adding your reply for
Andrew's comment as [PATCH 0/1]...

- Masayoshi

Wed, 31 Jan 2018 13:24:55 -0500 Pavel Tatashin wrote:
> Hi Koki,
> Yes, the patch is here:
> It has not been reviewed yet.
> Pavel
> On Wed, Jan 31, 2018 at 12:28 PM,
> <> wrote:
>> Pavel,
>> I assume you are working on the fix.
>> Do you have any progress ?
>> Koki
>>>> -----Original Message-----
>>>> From: Mel Gorman []
>>>> Sent: Wednesday, December 06, 2017 5:50 AM
>>>> To: Pavel Tatashin <>
>>>> Cc: Michal Hocko <>; YASUAKI ISHIMATSU
>>>> <>; Andrew Morton <>;
>>>> Linux Memory Management List <>; linux-
>>>>; Sanagi, Koki <>; Steve
>>>> Sistare <>
>>>> Subject: Re: [PATCH] mm, meminit: Serially initialise deferred memory if
>>>> trace_buf_size is specified
>>>> On Wed, Nov 29, 2017 at 10:41:59PM -0500, Pavel Tatashin wrote:
>>>>> Hi Mel,
>>>>> Thank you very much for your feedback, my replies below:
>>>>>> A lack of involvement from admins is indeed desirable. For example,
>>>>>> while I might concede on using a disable-everything-switch, I would
>>>>>> not be happy to introduce a switch that specified how much memory
>>>>>> per node to initialise.
>>>>>> For the forth approach, I really would be only thinking of a blunt
>>>>>> "initialise everything instead of going OOM". I was wary of making
>>>>>> things too complicated and I worried about some side-effects I'll cover later.
>>>>> I see, I misunderstood your suggestion. Switching to serial
>>>>> initialization when OOM works, however, boot time becomes
>>>>> unpredictable, with some configurations boot is fast with others it is
>>>>> slow. All of that depends on whether predictions in
>>>>> reset_deferred_meminit() were good or not which is not easy to debug
>>>>> for users. Also, overtime predictions in reset_deferred_meminit() can
>>>>> become very off, and I do not think that we want to continuously
>>>>> adjust this function.
>>>> You could increase the probabilty of a report by doing a WARN_ON_ONCE if the
>>>> serialised meminit is used.
>>>>>>> With this approach we could always init a very small amount of
>>>>>>> struct pages, and allow the rest to be initialized on demand as
>>>>>>> boot requires until deferred struct pages are initialized. Since,
>>>>>>> having deferred pages feature assumes that the machine is large,
>>>>>>> there is no drawback of having some extra byte of dead code,
>>>>>>> especially that all the checks can be permanently switched of via
>>>>>>> static branches once deferred init is complete.
>>>>>> This is where I fear there may be dragons. If we minimse the number
>>>>>> of struct pages and initialise serially as necessary, there is a
>>>>>> danger that we'll allocate remote memory in cases where local memory
>>>>>> would have done because a remote node had enough memory.
>>>>> True, but is not what we have now has the same issue as well? If one
>>>>> node is gets out of memory we start using memory from another node,
>>>>> before deferred pages are initialized?
>>>> It's possible but I'm not aware of it happening currently.
>>>>> To offset that risk, it would be
>>>>>> necessary at boot-time to force allocations from local node where
>>>>>> possible and initialise more memory as necessary. That starts
>>>>>> getting complicated because we'd need to adjust gfp-flags in the
>>>>>> fast path with init-and-retry logic in the slow path and that could
>>>>>> be a constant penalty. We could offset that in the fast path by
>>>>>> using static branches
>>>>> I will try to implement this, and see how complicated the patch will
>>>>> be, if it gets too complicated for the problem I am trying to solve we
>>>>> can return to one of your suggestions.
>>>>> I was thinking to do something like this:
>>>>> Start with every small amount of initialized pages in every node.
>>>>> If allocation fails, initialize enough struct pages to cover this
>>>>> particular allocation with struct pages rounded up to section size but
>>>>> in every single node.
>>>> Ok, just make sure it's all in the slow paths of the allocator when the alternative
>>>> is to fail the allocation.
>>>>>> but it's getting more and
>>>>>> more complex for what is a minor optimisation -- shorter boot times
>>>>>> on large machines where userspace itself could take a *long* time to
>>>>>> get up and running (think database reading in 1TB of data from disk as it
>>>> warms up).
>>>>> On M6-32 with 32T [1] of memory it saves over 4 minutes of boot time,
>>>>> and this is on SPARC with 8K pages, on x86 it would be around of 8
>>>>> minutes because of twice as many pages. This feature improves
>>>>> availability for larger machines quite a bit. Overtime, systems are
>>>>> growing, so I expect this feature to become a default configuration in
>>>>> the next several years on server configs.
>>>> Ok, when developing the series originally, I had no machine even close to 32T of
>>>> memory.
>>>> --
>>>> Mel Gorman
>>>> SUSE Labs
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to For more info on Linux MM,
>> see: .
>> Don't email: <a hrefmailto:""> </a>

 \ /
  Last update: 2018-02-05 15:15    [W:0.064 / U:5.484 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site