[lkml]   [2019]   [Dec]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [PATCH v4 4/4] powerpc: Book3S 64-bit "heavyweight" KASAN support

Le 19/12/2019 à 10:50, Daniel Axtens a écrit :
> Christophe Leroy <> writes:
>> On 12/19/2019 12:36 AM, Daniel Axtens wrote:
>>> KASAN support on Book3S is a bit tricky to get right:
>>> - It would be good to support inline instrumentation so as to be able to
>>> catch stack issues that cannot be caught with outline mode.
>>> - Inline instrumentation requires a fixed offset.
>>> - Book3S runs code in real mode after booting. Most notably a lot of KVM
>>> runs in real mode, and it would be good to be able to instrument it.
>>> - Because code runs in real mode after boot, the offset has to point to
>>> valid memory both in and out of real mode.
>>> [ppc64 mm note: The kernel installs a linear mapping at effective
>>> address c000... onward. This is a one-to-one mapping with physical
>>> memory from 0000... onward. Because of how memory accesses work on
>>> powerpc 64-bit Book3S, a kernel pointer in the linear map accesses the
>>> same memory both with translations on (accessing as an 'effective
>>> address'), and with translations off (accessing as a 'real
>>> address'). This works in both guests and the hypervisor. For more
>>> details, see s5.7 of Book III of version 3 of the ISA, in particular
>>> the Storage Control Overview, s5.7.3, and s5.7.5 - noting that this
>>> KASAN implementation currently only supports Radix.]
>>> One approach is just to give up on inline instrumentation. This way all
>>> checks can be delayed until after everything set is up correctly, and the
>>> address-to-shadow calculations can be overridden. However, the features and
>>> speed boost provided by inline instrumentation are worth trying to do
>>> better.
>>> If _at compile time_ it is known how much contiguous physical memory a
>>> system has, the top 1/8th of the first block of physical memory can be set
>>> aside for the shadow. This is a big hammer and comes with 3 big
>>> consequences:
>>> - there's no nice way to handle physically discontiguous memory, so only
>>> the first physical memory block can be used.
>>> - kernels will simply fail to boot on machines with less memory than
>>> specified when compiling.
>>> - kernels running on machines with more memory than specified when
>>> compiling will simply ignore the extra memory.
>>> Implement and document KASAN this way. The current implementation is Radix
>>> only.
>>> Despite the limitations, it can still find bugs,
>>> e.g.
>>> At the moment, this physical memory limit must be set _even for outline
>>> mode_. This may be changed in a later series - a different implementation
>>> could be added for outline mode that dynamically allocates shadow at a
>>> fixed offset. For example, see
>>> Suggested-by: Michael Ellerman <>
>>> Cc: Balbir Singh <> # ppc64 out-of-line radix version
>>> Cc: Christophe Leroy <> # ppc32 version
>>> Signed-off-by: Daniel Axtens <>
>>> ---
>>> Changes since v3:
>>> - Address further feedback from Christophe.
>>> - Drop changes to stack walking, it looks like the issue I observed is
>>> related to that particular stack, not stack-walking generally.
>>> Changes since v2:
>>> - Address feedback from Christophe around cleanups and docs.
>>> - Address feedback from Balbir: at this point I don't have a good solution
>>> for the issues you identify around the limitations of the inline implementation
>>> but I think that it's worth trying to get the stack instrumentation support.
>>> I'm happy to have an alternative and more flexible outline mode - I had
>>> envisoned this would be called 'lightweight' mode as it imposes fewer restrictions.
>>> I've linked to your implementation. I think it's best to add it in a follow-up series.
>>> - Made the default PHYS_MEM_SIZE_FOR_KASAN value 1024MB. I think most people have
>>> guests with at least that much memory in the Radix 64s case so it's a much
>>> saner default - it means that if you just turn on KASAN without reading the
>>> docs you're much more likely to have a bootable kernel, which you will never
>>> have if the value is set to zero! I'm happy to bikeshed the value if we want.
>>> Changes since v1:
>>> - Landed kasan vmalloc support upstream
>>> - Lots of feedback from Christophe.
>>> Changes since the rfc:
>>> - Boots real and virtual hardware, kvm works.
>>> - disabled reporting when we're checking the stack for exception
>>> frames. The behaviour isn't wrong, just incompatible with KASAN.
>>> - Documentation!
>>> - Dropped old module stuff in favour of KASAN_VMALLOC.
>>> The bugs with ftrace and kuap were due to kernel bloat pushing
>>> prom_init calls to be done via the plt. Because we did not have
>>> a relocatable kernel, and they are done very early, this caused
>>> everything to explode. Compile with CONFIG_RELOCATABLE!
>>> ---
>>> Documentation/dev-tools/kasan.rst | 8 +-
>>> Documentation/powerpc/kasan.txt | 112 ++++++++++++++++++-
>>> arch/powerpc/Kconfig | 2 +
>>> arch/powerpc/Kconfig.debug | 21 ++++
>>> arch/powerpc/Makefile | 11 ++
>>> arch/powerpc/include/asm/book3s/64/hash.h | 4 +
>>> arch/powerpc/include/asm/book3s/64/pgtable.h | 7 ++
>>> arch/powerpc/include/asm/book3s/64/radix.h | 5 +
>>> arch/powerpc/include/asm/kasan.h | 21 +++-
>>> arch/powerpc/kernel/prom.c | 61 +++++++++-
>>> arch/powerpc/mm/kasan/Makefile | 1 +
>>> arch/powerpc/mm/kasan/init_book3s_64.c | 70 ++++++++++++
>>> arch/powerpc/platforms/Kconfig.cputype | 1 +
>>> 13 files changed, 316 insertions(+), 8 deletions(-)
>>> create mode 100644 arch/powerpc/mm/kasan/init_book3s_64.c
>>> diff --git a/arch/powerpc/include/asm/kasan.h b/arch/powerpc/include/asm/kasan.h
>>> index 296e51c2f066..f18268cbdc33 100644
>>> --- a/arch/powerpc/include/asm/kasan.h
>>> +++ b/arch/powerpc/include/asm/kasan.h
>>> @@ -2,6 +2,9 @@
>>> #ifndef __ASM_KASAN_H
>>> #define __ASM_KASAN_H
>>> +#include <asm/page.h>
>>> +#include <asm/pgtable.h>
>> What do you need asm/pgtable.h for ?
>> Build failure due to circular inclusion of asm/pgtable.h:
> I see there's a lot of ppc32 stuff, I clearly need to bite the bullet
> and get a ppc32 toolchain so I can squash these without chewing up any
> more of your time. I'll sort that out and send a new spin.

I'm using a powerpc64 toolchain to build both ppc32 and ppc64 kernels
(from )

Another thing, did you test PTDUMP stuff with KASAN ? It looks like
KASAN address markers don't depend on PPC32, but are only initialised by
populate_markers() for PPC32.

Regarding kasan.h, I think we should be able to end up with something
where the definition of KASAN_SHADOW_OFFSET should only depend on the
existence of CONFIG_KASAN_SHADOW_OFFSET, and where only
KASAN_SHADOW_SIZE should depend on the target (ie PPC32 or BOOK3S64)
Everything else should be common. KASAN_END should be START+SIZE.

It looks like what you have called KASAN_SHADOW_SIZE is not similar to
what is called KASAN_SHADOW_SIZE for PPC32, as yours only covers the
SHADOW_SIZE for linear mem while PPC32 one covers the full space.


 \ /
  Last update: 2019-12-19 11:05    [W:0.065 / U:0.388 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site