kernel.git/mm/hmm.c, branch linux-4.18.y

mm/hmm: fix race between hmm_mirror_unregister() and mmu_notifier callback

2018-11-13T19:12:43Z

commit 86a2d59841ab0b147ffc1b7b3041af87927cf312 upstream. In hmm_mirror_unregister(), mm->hmm is set to NULL and then mmu_notifier_unregister_no_release() is called. That creates a small window where mmu_notifier can call mmu_notifier_ops with mm->hmm equal to NULL. Fix this by first unregistering mmu notifier callbacks and then setting mm->hmm to NULL. Similarly in hmm_register(), set mm->hmm before registering mmu_notifier callbacks so callback functions always see mm->hmm set. Link: http://lkml.kernel.org/r/20181019160442.18723-4-jglisse@redhat.com Signed-off-by: Ralph Campbell Signed-off-by: Jérôme Glisse Reviewed-by: John Hubbard Reviewed-by: Jérôme Glisse Reviewed-by: Balbir Singh Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman

mm, dev_pagemap: Do not clear ->mapping on final put

2018-09-09T08:32:41Z

commit 2fa147bdbf672c53386a8f5f2c7fe358004c3ef8 upstream. MEMORY_DEVICE_FS_DAX relies on typical page semantics whereby ->mapping is only ever cleared by truncation, not final put. Without this fix dax pages may forget their mapping association at the end of every page pin event. Move this atypical behavior that HMM wants into the HMM ->page_free() callback. Cc: Cc: Jan Kara Cc: Jérôme Glisse Cc: Andrew Morton Cc: Ross Zwisler Fixes: d2c997c0f145 ("fs, dax: use page->mapping...") Signed-off-by: Dan Williams Acked-by: Jérôme Glisse Signed-off-by: Dave Jiang Signed-off-by: Greg Kroah-Hartman

mm: introduce MEMORY_DEVICE_FS_DAX and CONFIG_DEV_PAGEMAP_OPS

2018-05-22T13:59:39Z

In preparation for fixing dax-dma-vs-unmap issues, filesystems need to be able to rely on the fact that they will get wakeups on dev_pagemap page-idle events. Introduce MEMORY_DEVICE_FS_DAX and generic_dax_page_free() as common indicator / infrastructure for dax filesytems to require. With this change there are no users of the MEMORY_DEVICE_HOST designation, so remove it. The HMM sub-system extended dev_pagemap to arrange a callback when a dev_pagemap managed page is freed. Since a dev_pagemap page is free / idle when its reference count is 1 it requires an additional branch to check the page-type at put_page() time. Given put_page() is a hot-path we do not want to incur that check if HMM is not in use, so a static branch is used to avoid that overhead when not necessary. Now, the FS_DAX implementation wants to reuse this mechanism for receiving dev_pagemap ->page_free() callbacks. Rework the HMM-specific static-key into a generic mechanism that either HMM or FS_DAX code paths can enable. For ARCH=um builds, and any other arch that lacks ZONE_DEVICE support, care must be taken to compile out the DEV_PAGEMAP_OPS infrastructure. However, we still need to support FS_DAX in the FS_DAX_LIMITED case implemented by the s390/dcssblk driver. Cc: Martin Schwidefsky Cc: Heiko Carstens Cc: Michal Hocko Reported-by: kbuild test robot Reported-by: Thomas Meyer Reported-by: Dave Jiang Cc: "Jérôme Glisse" Reviewed-by: Jan Kara Reviewed-by: Christoph Hellwig Signed-off-by: Dan Williams

mm/hmm.c: remove superfluous RCU protection around radix tree lookup

2018-04-11T17:28:31Z

hmm_devmem_find() requires rcu_read_lock_held() but there's nothing which actually uses the RCU protection. The only caller is hmm_devmem_pages_create() which already grabs the mutex and does superfluous rcu_read_lock/unlock() around the function. This doesn't add anything and just adds to confusion. Remove the RCU protection and open-code the radix tree lookup. If this needs to become more sophisticated in the future, let's add them back when necessary. Link: http://lkml.kernel.org/r/20180314194515.1661824-4-tj@kernel.org Signed-off-by: Tejun Heo Reviewed-by: Jérôme Glisse Cc: Paul E. McKenney Cc: Benjamin LaHaise Cc: Al Viro Cc: Kent Overstreet Cc: Matthew Wilcox Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

mm/hmm: use device driver encoding for HMM pfn

2018-04-11T17:28:31Z

Users of hmm_vma_fault() and hmm_vma_get_pfns() provide a flags array and pfn shift value allowing them to define their own encoding for HMM pfn that are fill inside the pfns array of the hmm_range struct. With this device driver can get pfn that match their own private encoding out of HMM without having to do any conversion. [rcampbell@nvidia.com: don't ignore specific pte fault flag in hmm_vma_fault()] Link: http://lkml.kernel.org/r/20180326213009.2460-2-jglisse@redhat.com [rcampbell@nvidia.com: clarify fault logic for device private memory] Link: http://lkml.kernel.org/r/20180326213009.2460-3-jglisse@redhat.com Link: http://lkml.kernel.org/r/20180323005527.758-16-jglisse@redhat.com Signed-off-by: Jérôme Glisse Signed-off-by: Ralph Campbell Cc: Evgeny Baskakov Cc: Ralph Campbell Cc: Mark Hairgrove Cc: John Hubbard Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

mm/hmm: change hmm_vma_fault() to allow write fault on page basis

2018-04-11T17:28:31Z

This changes hmm_vma_fault() to not take a global write fault flag for a range but instead rely on caller to populate HMM pfns array with proper fault flag ie HMM_PFN_VALID if driver want read fault for that address or HMM_PFN_VALID and HMM_PFN_WRITE for write. Moreover by setting HMM_PFN_DEVICE_PRIVATE the device driver can ask for device private memory to be migrated back to system memory through page fault. This is more flexible API and it better reflects how device handles and reports fault. Link: http://lkml.kernel.org/r/20180323005527.758-15-jglisse@redhat.com Signed-off-by: Jérôme Glisse Cc: Evgeny Baskakov Cc: Ralph Campbell Cc: Mark Hairgrove Cc: John Hubbard Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

mm/hmm: factor out pte and pmd handling to simplify hmm_vma_walk_pmd()

2018-04-11T17:28:31Z

No functional change, just create one function to handle pmd and one to handle pte (hmm_vma_handle_pmd() and hmm_vma_handle_pte()). Link: http://lkml.kernel.org/r/20180323005527.758-14-jglisse@redhat.com Signed-off-by: Jérôme Glisse Reviewed-by: John Hubbard Cc: Evgeny Baskakov Cc: Ralph Campbell Cc: Mark Hairgrove Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

mm/hmm: move hmm_pfns_clear() closer to where it is used

2018-04-11T17:28:30Z

Move hmm_pfns_clear() closer to where it is used to make it clear it is not use by page table walkers. Link: http://lkml.kernel.org/r/20180323005527.758-13-jglisse@redhat.com Signed-off-by: Jérôme Glisse Reviewed-by: John Hubbard Cc: Evgeny Baskakov Cc: Ralph Campbell Cc: Mark Hairgrove Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

mm/hmm: rename HMM_PFN_DEVICE_UNADDRESSABLE to HMM_PFN_DEVICE_PRIVATE

2018-04-11T17:28:30Z

Make naming consistent across code, DEVICE_PRIVATE is the name use outside HMM code so use that one. Link: http://lkml.kernel.org/r/20180323005527.758-12-jglisse@redhat.com Signed-off-by: Jérôme Glisse Reviewed-by: John Hubbard Cc: Evgeny Baskakov Cc: Ralph Campbell Cc: Mark Hairgrove Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

mm/hmm: do not differentiate between empty entry or missing directory

2018-04-11T17:28:30Z

There is no point in differentiating between a range for which there is not even a directory (and thus entries) and empty entry (pte_none() or pmd_none() returns true). Simply drop the distinction ie remove HMM_PFN_EMPTY flag and merge now duplicate hmm_vma_walk_hole() and hmm_vma_walk_clear() functions. Link: http://lkml.kernel.org/r/20180323005527.758-11-jglisse@redhat.com Signed-off-by: Jérôme Glisse Reviewed-by: John Hubbard Cc: Evgeny Baskakov Cc: Ralph Campbell Cc: Mark Hairgrove Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds