kernel.git/mm/huge_memory.c, branch linux-3.2.y

mm/huge_memory: Fix unused label warning

2017-11-11T13:34:41Z

This label is unused since commit 2ea6895123eb8604c1c0c153e2fcd1305fb96aca "mm/huge_memory.c: fix up "mm/huge_memory.c: respect FOLL_FORCE/FOLL_COW for thp" backport". There's no upstream equivalent of this as the label is still used there. Signed-off-by: Ben Hutchings

mm/huge_memory.c: fix up "mm/huge_memory.c: respect FOLL_FORCE/FOLL_COW for thp" backport

2017-04-04T21:18:32Z

This is a stable follow up fix for an incorrect backport. The issue is not present in the upstream kernel. Miroslav has noticed the following splat when testing my 3.2 forward port of 8310d48b125d ("mm/huge_memory.c: respect FOLL_FORCE/FOLL_COW for thp") to 3.12: BUG: Bad page state in process a.out pfn:26400 page:ffffea000085e000 count:0 mapcount:1 mapping: (null) index:0x7f049d600 page flags: 0x1fffff80108018(uptodate|dirty|head|swapbacked) page dumped because: nonzero mapcount [iii] CPU: 2 PID: 5926 Comm: a.out Tainted: G E 3.12.61-0-default #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.0.0-prebuilt.qemu-project.org 04/01/2014 0000000000000000 ffffffff81515830 ffffea000085e000 ffffffff81800ad7 ffffffff815118a5 ffffea000085e000 0000000000000000 000fffff80000000 ffffffff81140f18 fff000007c000000 ffffea000085e000 0000000000000009 Call Trace: [] dump_trace+0x7d/0x2d0 [] show_stack_log_lvl+0x94/0x170 [] show_stack+0x21/0x50 [] dump_stack+0x5d/0x78 [] bad_page.part.67+0xe8/0x102 [] free_pages_prepare+0x198/0x1b0 [] __free_pages_ok+0x15/0xd0 [] __access_remote_vm+0x7c/0x1e0 [] mem_rw.isra.13+0x14b/0x1a0 [] vfs_write+0xb8/0x1e0 [] SyS_pwrite64+0x6b/0xa0 [] system_call_fastpath+0x16/0x1b [<00007f049da18573>] 0x7f049da18572 The problem is that the original 3.2 backport didn't return NULL page on the FOLL_COW page and so the page got reused. Reported-and-tested-by: Miroslav Beneš Signed-off-by: Michal Hocko Signed-off-by: Ben Hutchings

mm/huge_memory.c: respect FOLL_FORCE/FOLL_COW for thp

2017-03-16T02:18:46Z

commit 8310d48b125d19fcd9521d83b8293e63eb1646aa upstream. In commit 19be0eaffa3a ("mm: remove gup_flags FOLL_WRITE games from __get_user_pages()"), the mm code was changed from unsetting FOLL_WRITE after a COW was resolved to setting the (newly introduced) FOLL_COW instead. Simultaneously, the check in gup.c was updated to still allow writes with FOLL_FORCE set if FOLL_COW had also been set. However, a similar check in huge_memory.c was forgotten. As a result, remote memory writes to ro regions of memory backed by transparent huge pages cause an infinite loop in the kernel (handle_mm_fault sets FOLL_COW and returns 0 causing a retry, but follow_trans_huge_pmd bails out immidiately because `(flags & FOLL_WRITE) && !pmd_write(*pmd)` is true. While in this state the process is stil SIGKILLable, but little else works (e.g. no ptrace attach, no other signals). This is easily reproduced with the following code (assuming thp are set to always): #include #include #include #include #include #include #include #include #include #include #define TEST_SIZE 5 * 1024 * 1024 int main(void) { int status; pid_t child; int fd = open("/proc/self/mem", O_RDWR); void *addr = mmap(NULL, TEST_SIZE, PROT_READ, MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); assert(addr != MAP_FAILED); pid_t parent_pid = getpid(); if ((child = fork()) == 0) { void *addr2 = mmap(NULL, TEST_SIZE, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); assert(addr2 != MAP_FAILED); memset(addr2, 'a', TEST_SIZE); pwrite(fd, addr2, TEST_SIZE, (uintptr_t)addr); return 0; } assert(child == waitpid(child, &status, 0)); assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); return 0; } Fix this by updating follow_trans_huge_pmd in huge_memory.c analogously to the update in gup.c in the original commit. The same pattern exists in follow_devmap_pmd. However, we should not be able to reach that check with FOLL_COW set, so add WARN_ONCE to make sure we notice if we ever do. [akpm@linux-foundation.org: coding-style fixes] Link: http://lkml.kernel.org/r/20170106015025.GA38411@juliacomputing.com Signed-off-by: Keno Fischer Acked-by: Kirill A. Shutemov Cc: Greg Thelen Cc: Nicholas Piggin Cc: Willy Tarreau Cc: Oleg Nesterov Cc: Kees Cook Cc: Andy Lutomirski Cc: Michal Hocko Cc: Hugh Dickins Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds [bwh: Backported to 3.2: - Drop change to follow_devmap_pmd() - pmd_dirty() is not available; check the page flags as in can_follow_write_pte() - Adjust context] Signed-off-by: Ben Hutchings

mm/huge_memory: replace VM_NO_THP VM_BUG_ON with actual VMA check

2016-06-15T20:28:13Z

commit 3486b85a29c1741db99d0c522211c82d2b7a56d0 upstream. Khugepaged detects own VMAs by checking vm_file and vm_ops but this way it cannot distinguish private /dev/zero mappings from other special mappings like /dev/hpet which has no vm_ops and popultes PTEs in mmap. This fixes false-positive VM_BUG_ON and prevents installing THP where they are not expected. Link: http://lkml.kernel.org/r/CACT4Y+ZmuZMV5CjSFOeXviwQdABAgT7T+StKfTqan9YDtgEi5g@mail.gmail.com Fixes: 78f11a255749 ("mm: thp: fix /dev/zero MAP_PRIVATE and vm_flags cleanups") Signed-off-by: Konstantin Khlebnikov Reported-by: Dmitry Vyukov Acked-by: Vlastimil Babka Acked-by: Kirill A. Shutemov Cc: Dmitry Vyukov Cc: Andrea Arcangeli Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds [bwh: Backported to 3.2: - The assertions use VM_BUG_ON() and also check is_linear_pfn_mapping(); keep that check - Adjust context] Signed-off-by: Ben Hutchings

thp: introduce hugepage_vma_check()

2016-06-15T20:28:13Z

commit fa475e517adb422cb3492e636195f9b2c0d009c8 upstream. Multiple places do the same check. Signed-off-by: Bob Liu Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Minchan Kim Cc: Ni zhan Chen Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds [bwh: Backported to 3.2: - Also move the is_linear_pfn_mapping() test - Adjust context] Signed-off-by: Ben Hutchings

mm, thp: fix collapsing of hugepages on madvise

2014-12-14T16:23:52Z

commit 6d50e60cd2edb5a57154db5a6f64eef5aa59b751 upstream. If an anonymous mapping is not allowed to fault thp memory and then madvise(MADV_HUGEPAGE) is used after fault, khugepaged will never collapse this memory into thp memory. This occurs because the madvise(2) handler for thp, hugepage_madvise(), clears VM_NOHUGEPAGE on the stack and it isn't stored in vma->vm_flags until the final action of madvise_behavior(). This causes the khugepaged_enter_vma_merge() to be a no-op in hugepage_madvise() when the vma had previously had VM_NOHUGEPAGE set. Fix this by passing the correct vma flags to the khugepaged mm slot handler. There's no chance khugepaged can run on this vma until after madvise_behavior() returns since we hold mm->mmap_sem. It would be possible to clear VM_NOHUGEPAGE directly from vma->vm_flags in hugepage_advise(), but I didn't want to introduce special case behavior into madvise_behavior(). I think it's best to just let it always set vma->vm_flags itself. Signed-off-by: David Rientjes Reported-by: Suleiman Souhlal Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds [bwh: Backported to 3.2: adjust context, indentation] Signed-off-by: Ben Hutchings

mm/huge_memory.c: fix potential NULL pointer dereference

2013-10-26T20:06:03Z

commit a8f531ebc33052642b4bd7b812eedf397108ce64 upstream. In collapse_huge_page() there is a race window between releasing the mmap_sem read lock and taking the mmap_sem write lock, so find_vma() may return NULL. So check the return value to avoid NULL pointer dereference. collapse_huge_page khugepaged_alloc_page up_read(&mm->mmap_sem) down_write(&mm->mmap_sem) vma = find_vma(mm, address) Signed-off-by: Libin Acked-by: Kirill A. Shutemov Reviewed-by: Wanpeng Li Reviewed-by: Michal Hocko Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Ben Hutchings

mm/THP: use pmd_populate() to update the pmd with pgtable_t pointer

2013-05-30T13:35:06Z

commit 7c3425123ddfdc5f48e7913ff59d908789712b18 upstream. We should not use set_pmd_at to update pmd_t with pgtable_t pointer. set_pmd_at is used to set pmd with huge pte entries and architectures like ppc64, clear few flags from the pte when saving a new entry. Without this change we observe bad pte errors like below on ppc64 with THP enabled. BUG: Bad page map in process ld mm=0xc000001ee39f4780 pte:7fc3f37848000001 pmd:c000001ec0000000 Signed-off-by: Aneesh Kumar K.V Cc: Hugh Dickins Cc: Benjamin Herrenschmidt Reviewed-by: Andrea Arcangeli Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Ben Hutchings

thp, memcg: split hugepage for memcg oom on cow

2013-01-03T03:33:55Z

commit 1f1d06c34f7675026326cd9f39ff91e4555cf355 upstream. On COW, a new hugepage is allocated and charged to the memcg. If the system is oom or the charge to the memcg fails, however, the fault handler will return VM_FAULT_OOM which results in an oom kill. Instead, it's possible to fallback to splitting the hugepage so that the COW results only in an order-0 page being allocated and charged to the memcg which has a higher liklihood to succeed. This is expensive because the hugepage must be split in the page fault handler, but it is much better than unnecessarily oom killing a process. Signed-off-by: David Rientjes Cc: Andrea Arcangeli Cc: Johannes Weiner Acked-by: KAMEZAWA Hiroyuki Cc: Michal Hocko Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Ben Hutchings

mm: thp: fix BUG on mm->nr_ptes

2012-03-12T19:31:26Z

commit 1c641e84719429bbfe62a95ed3545ee7fe24408f upstream. Dave Jones reports a few Fedora users hitting the BUG_ON(mm->nr_ptes...) in exit_mmap() recently. Quoting Hugh's discovery and explanation of the SMP race condition: "mm->nr_ptes had unusual locking: down_read mmap_sem plus page_table_lock when incrementing, down_write mmap_sem (or mm_users 0) when decrementing; whereas THP is careful to increment and decrement it under page_table_lock. Now most of those paths in THP also hold mmap_sem for read or write (with appropriate checks on mm_users), but two do not: when split_huge_page() is called by hwpoison_user_mappings(), and when called by add_to_swap(). It's conceivable that the latter case is responsible for the exit_mmap() BUG_ON mm->nr_ptes that has been reported on Fedora." The simplest way to fix it without having to alter the locking is to make split_huge_page() a noop in nr_ptes terms, so by counting the preallocated pagetables that exists for every mapped hugepage. It was an arbitrary choice not to count them and either way is not wrong or right, because they are not used but they're still allocated. Reported-by: Dave Jones Reported-by: Hugh Dickins Signed-off-by: Andrea Arcangeli Acked-by: Hugh Dickins Cc: David Rientjes Cc: Josh Boyer Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman