kernel.git/arch/powerpc/mm/slice.c, branch linux-4.17.y

powerpc/8xx: Fix build with hugetlbfs enabled

2018-04-11T02:00:23Z

8xx uses the slice code when hugetlbfs is enabled. We missed a header include on 8xx which resulted in the below build failure: config: mpc885_ads_defconfig + CONFIG_HUGETLBFS arch/powerpc/mm/slice.c: In function 'slice_get_unmapped_area': arch/powerpc/mm/slice.c:655:2: error: implicit declaration of function 'need_extra_context' arch/powerpc/mm/slice.c:656:3: error: implicit declaration of function 'alloc_extended_context' on PPC64 the mmu_context.h was included via linux/pkeys.h Signed-off-by: Aneesh Kumar K.V Signed-off-by: Michael Ellerman

powerpc/mm: Add support for handling > 512TB address in SLB miss

2018-03-30T13:10:38Z

For addresses above 512TB we allocate additional mmu contexts. To make it all easy, addresses above 512TB are handled with IR/DR=1 and with stack frame setup. The mmu_context_t is also updated to track the new extended_ids. To support upto 4PB we need a total 8 contexts. Signed-off-by: Aneesh Kumar K.V [mpe: Minor formatting tweaks and comment wording, switch BUG to WARN in get_ea_context().] Signed-off-by: Michael Ellerman

powerpc/mm/slice: Consolidate return path in slice_get_unmapped_area()

2018-03-30T13:10:37Z

In a following patch, on finding a free area we will need to do allocatinon of extra contexts as needed. Consolidating the return path for slice_get_unmapped_area() will make that easier. Split into a separate patch to make review easy. Signed-off-by: Aneesh Kumar K.V Signed-off-by: Michael Ellerman

powerpc/mm/slice: remove radix calls to the slice code

2018-03-13T12:43:08Z

This is a tidy up which removes radix MMU calls into the slice code. Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman

powerpc/mm/slice: Use const pointers to cached slice masks where possible

2018-03-13T12:43:08Z

The slice_mask cache was a basic conversion which copied the slice mask into caller's structures, because that's how the original code worked. In most cases the pointer can be used directly instead, saving a copy and an on-stack structure. On POWER8, this increases vfork+exec+exit performance by 0.3% and reduces time to mmap+munmap a 64kB page by 2%. Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman

powerpc/mm/slice: remove dead code

2018-03-13T12:43:07Z

This code is never compiled in, and it gets broken by the next patch, so remove it. Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman

powerpc/mm/slice: Switch to 3-operand slice bitops helpers

2018-03-13T12:43:07Z

This converts the slice_mask bit operation helpers to be the usual 3-operand kind, which allows 2 inputs to set a different output without an extra copy, which is used in the next patch. Adds slice_copy_mask, which will be used in the next patch. Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman

powerpc/mm/slice: implement slice_check_range_fits

2018-03-13T12:43:06Z

Rather than build slice masks from a range then use that to check for fit in a candidate mask, implement slice_check_range_fits that checks if a range fits in a mask directly. This allows several structures to be removed from stacks, and also we don't expect a huge range in a lot of these cases, so building and comparing a full mask is going to be more expensive than testing just one or two bits of the range. On POWER8, this increases vfork+exec+exit performance by 0.3% and reduces time to mmap+munmap a 64kB page by 5%. Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman

powerpc/mm/slice: implement a slice mask cache

2018-03-13T12:43:06Z

Calculating the slice mask can become a signifcant overhead for get_unmapped_area. This patch adds a struct slice_mask for each page size in the mm_context, and keeps these in synch with the slices psize arrays and slb_addr_limit. On Book3S/64 this adds 288 bytes to the mm_context_t for the slice mask caches. On POWER8, this increases vfork+exec+exit performance by 9.9% and reduces time to mmap+munmap a 64kB page by 28%. Reduces time to mmap+munmap by about 10% on 8xx. Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman

powerpc/mm/slice: pass pointers to struct slice_mask where possible

2018-03-13T12:43:05Z

Pass around const pointers to struct slice_mask where possible, rather than copies of slice_mask, to reduce stack and call overhead. checkstack.pl gives, before: 0x00000d1c slice_get_unmapped_area [slice.o]: 592 0x00001864 is_hugepage_only_range [slice.o]: 448 0x00000754 slice_find_area_topdown [slice.o]: 400 0x00000484 slice_find_area_bottomup.isra.1 [slice.o]: 272 0x000017b4 slice_set_range_psize [slice.o]: 224 0x00000a4c slice_find_area [slice.o]: 128 0x00000160 slice_check_fit [slice.o]: 112 after: 0x00000ad0 slice_get_unmapped_area [slice.o]: 448 0x00001464 is_hugepage_only_range [slice.o]: 288 0x000006c0 slice_find_area [slice.o]: 144 0x0000016c slice_check_fit [slice.o]: 128 0x00000528 slice_find_area_bottomup.isra.2 [slice.o]: 128 0x000013e4 slice_set_range_psize [slice.o]: 128 This increases vfork+exec+exit performance by 1.5%. Reduces time to mmap+munmap a 64kB page by 17%. Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman