kernel.git/arch/mips/mm/cache.c, branch linux-4.19.y

MIPS: simplify CONFIG_DMA_NONCOHERENT ifdefs

2018-06-24T16:26:02Z

CONFIG_DMA_MAYBE_COHERENT already selects CONFIG_DMA_NONCOHERENT, so we can remove the extra conditions. Signed-off-by: Christoph Hellwig Reviewed-by: Paul Burton Patchwork: https://patchwork.linux-mips.org/patch/19529/ Signed-off-by: Paul Burton Cc: Florian Fainelli Cc: David Daney Cc: Kevin Cernekee Cc: Jiaxun Yang Cc: Tom Bogendoerfer Cc: Huacai Chen Cc: iommu@lists.linux-foundation.org Cc: linux-mips@linux-mips.org

mm: fix races between swapoff and flush dcache

2018-04-06T04:36:26Z

Thanks to commit 4b3ef9daa4fc ("mm/swap: split swap cache into 64MB trunks"), after swapoff the address_space associated with the swap device will be freed. So page_mapping() users which may touch the address_space need some kind of mechanism to prevent the address_space from being freed during accessing. The dcache flushing functions (flush_dcache_page(), etc) in architecture specific code may access the address_space of swap device for anonymous pages in swap cache via page_mapping() function. But in some cases there are no mechanisms to prevent the swap device from being swapoff, for example, CPU1 CPU2 __get_user_pages() swapoff() flush_dcache_page() mapping = page_mapping() ... exit_swap_address_space() ... kvfree(spaces) mapping_mapped(mapping) The address space may be accessed after being freed. But from cachetlb.txt and Russell King, flush_dcache_page() only care about file cache pages, for anonymous pages, flush_anon_page() should be used. The implementation of flush_dcache_page() in all architectures follows this too. They will check whether page_mapping() is NULL and whether mapping_mapped() is true to determine whether to flush the dcache immediately. And they will use interval tree (mapping->i_mmap) to find all user space mappings. While mapping_mapped() and mapping->i_mmap isn't used by anonymous pages in swap cache at all. So, to fix the race between swapoff and flush dcache, __page_mapping() is add to return the address_space for file cache pages and NULL otherwise. All page_mapping() invoking in flush dcache functions are replaced with page_mapping_file(). [akpm@linux-foundation.org: simplify page_mapping_file(), per Mike] Link: http://lkml.kernel.org/r/20180305083634.15174-1-ying.huang@intel.com Signed-off-by: "Huang, Ying" Reviewed-by: Andrew Morton Cc: Minchan Kim Cc: Michal Hocko Cc: Johannes Weiner Cc: Mel Gorman Cc: Dave Hansen Cc: Chen Liqin Cc: Russell King Cc: Yoshinori Sato Cc: "James E.J. Bottomley" Cc: Guan Xuetao Cc: "David S. Miller" Cc: Chris Zankel Cc: Vineet Gupta Cc: Ley Foon Tan Cc: Ralf Baechle Cc: Andi Kleen Cc: Mike Rapoport Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

MIPS: Remove __invalidate_kernel_vmap_range

2017-08-29T13:21:55Z

The __invalidate_kernel_vmap_range function pointer global variable isn't used anywhere. Remove it. Signed-off-by: Paul Burton Cc: linux-mips@linux-mips.org Cc: trivial@kernel.org Patchwork: https://patchwork.linux-mips.org/patch/17174/ Signed-off-by: Ralf Baechle

MIPS: Include asm/setup.h for cpu_cache_init()

2017-08-29T13:21:54Z

arch/mips/mm/cache.c provides our implementation of the cpu_cache_init() function, but doesn't include the asm/setup.h header which declares it. This leads to a warning from sparse: arch/mips/mm/cache.c:274:6: warning: symbol 'cpu_cache_init' was not declared. Should it be static? Fix this by including asm/setup.h to get the declaration of cpu_cache_init(). Signed-off-by: Paul Burton Cc: linux-mips@linux-mips.org Cc: trivial@kernel.org Patchwork: https://patchwork.linux-mips.org/patch/17168/ Signed-off-by: Ralf Baechle

KVM: MIPS/Emulate: Adapt T&E CACHE emulation for Octeon

2017-03-28T14:36:17Z

Cache management is implemented separately for Cavium Octeon CPUs, so r4k_blast_[id]cache aren't available. Instead for Octeon perform a local icache flush using local_flush_icache_range(), and for other platforms which don't use c-r4k.c use __flush_cache_all() / flush_icache_all(). Signed-off-by: James Hogan Cc: Paolo Bonzini Cc: "Radim Krčmář" Cc: Ralf Baechle Cc: David Daney Cc: Andreas Herrmann Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org

MIPS: mm: Audit and remove any unnecessary uses of module.h

2016-10-04T23:31:20Z

Historically a lot of these existed because we did not have a distinction between what was modular code and what was providing support to modules via EXPORT_SYMBOL and friends. That changed when we forked out support for the latter into the export.h file. This means we should be able to reduce the usage of module.h in code that is obj-y Makefile or bool Kconfig. The advantage in doing so is that module.h itself sources about 15 other headers; adding significantly to what we feed cpp, and it can obscure what headers we are effectively using. Since module.h was the source for init.h (for __init) and for export.h (for EXPORT_SYMBOL) we consider each obj-y/bool instance for the presence of either and replace as needed. Signed-off-by: Paul Gortmaker Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/14033/ Signed-off-by: Ralf Baechle

MIPS: cacheflush: Use __flush_icache_user_range()

2016-10-04T14:13:57Z

The cacheflush(2) system call uses flush_icache_range() to flush a range of usermode addresses from the icache, so change it to utilise the new __flush_icache_user_range() API to allow the more generic flush_icache_range() to be changed to work on kernel addresses only. Signed-off-by: James Hogan Cc: Leonid Yegoshin Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/14153/ Signed-off-by: Ralf Baechle

MIPS: c-r4k: Split user/kernel flush_icache_range()

2016-10-04T14:13:57Z

flush_icache_range() is used for both user addresses (i.e. cacheflush(2)), and kernel addresses (as the API documentation describes). This isn't really suitable however for Enhanced Virtual Addressing (EVA) where cache operations on usermode addresses must use a different instruction, and the protected cache ops assume user addresses, making flush_icache_range() ineffective on kernel addresses. Split out a new __flush_icache_user_range() and __local_flush_icache_user_range() for users which actually want to flush usermode addresses (note that flush_icache_user_range() already exists on various architectures but with different arguments). The implementation of flush_icache_range() will be changed in an upcoming commit to use unprotected normal cache ops so as to always work on the kernel mode address space. Signed-off-by: James Hogan Cc: Leonid Yegoshin Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/14152/ Signed-off-by: Ralf Baechle

MIPS: Sync icache & dcache in set_pte_at

2016-05-13T12:01:58Z

It's possible for pages to become visible prior to update_mmu_cache running if a thread within the same address space preempts the current thread or runs simultaneously on another CPU. That is, the following scenario is possible: CPU0 CPU1 write to page flush_dcache_page flush_icache_page set_pte_at map page update_mmu_cache If CPU1 maps the page in between CPU0's set_pte_at, which marks it valid & visible, and update_mmu_cache where the dcache flush occurs then CPU1s icache will fill from stale data (unless it fills from the dcache, in which case all is good, but most MIPS CPUs don't have this property). Commit 4d46a67a3eb8 ("MIPS: Fix race condition in lazy cache flushing.") attempted to fix that by performing the dcache flush in flush_icache_page such that it occurs before the set_pte_at call makes the page visible. However it has the problem that not all code that writes to pages exposed to userland call flush_icache_page. There are many callers of set_pte_at under mm/ and only 2 of them do call flush_icache_page. Thus the race window between a page becoming visible & being coherent between the icache & dcache remains open in some cases. To illustrate some of the cases, a WARN was added to __update_cache with this patch applied that triggered in cases where a page about to be flushed from the dcache was not the last page provided to flush_icache_page. That is, backtraces were obtained for cases in which the race window is left open without this patch. The 2 standout examples follow. When forking a process: [ 15.271842] [<80417630>] __update_cache+0xcc/0x188 [ 15.277274] [<80530394>] copy_page_range+0x56c/0x6ac [ 15.282861] [<8042936c>] copy_process.part.54+0xd40/0x17ac [ 15.289028] [<80429f80>] do_fork+0xe4/0x420 [ 15.293747] [<80413808>] handle_sys+0x128/0x14c When exec'ing an ELF binary: [ 14.445964] [<80417630>] __update_cache+0xcc/0x188 [ 14.451369] [<80538d88>] move_page_tables+0x414/0x498 [ 14.457075] [<8055d848>] setup_arg_pages+0x220/0x318 [ 14.462685] [<805b0f38>] load_elf_binary+0x530/0x12a0 [ 14.468374] [<8055ec3c>] search_binary_handler+0xbc/0x214 [ 14.474444] [<8055f6c0>] do_execveat_common+0x43c/0x67c [ 14.480324] [<8055f938>] do_execve+0x38/0x44 [ 14.485137] [<80413808>] handle_sys+0x128/0x14c These code paths write into a page, call flush_dcache_page then call set_pte_at without flush_icache_page inbetween. The end result is that the icache can become corrupted & userland processes may execute unexpected or invalid code, typically resulting in a reserved instruction exception, a trap or a segfault. Fix this race condition fully by performing any cache maintenance required to keep the icache & dcache in sync in set_pte_at, before the page is made valid. This has the added bonus of ensuring the cache maintenance always happens in one location, rather than being duplicated in flush_icache_page & update_mmu_cache. It also matches the way other architectures solve the same problem (see arm, ia64 & powerpc). Signed-off-by: Paul Burton Reported-by: Ionela Voinescu Cc: Lars Persson Fixes: 4d46a67a3eb8 ("MIPS: Fix race condition in lazy cache flushing.") Cc: Steven J. Hill Cc: David Daney Cc: Huacai Chen Cc: Aneesh Kumar K.V Cc: Andrew Morton Cc: Jerome Marchand Cc: Kirill A. Shutemov Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Cc: stable # v4.1+ Patchwork: https://patchwork.linux-mips.org/patch/12722/ Signed-off-by: Ralf Baechle

MIPS: Handle highmem pages in __update_cache

2016-05-13T12:01:58Z

The following patch will expose __update_cache to highmem pages. Handle them by mapping them in for the duration of the cache maintenance, just like in __flush_dcache_page. The code for that isn't shared because we need the page address in __update_cache so sharing became messy. Given that the entirity is an extra 5 lines, just duplicate it. Signed-off-by: Paul Burton Cc: Lars Persson Cc: Andrew Morton Cc: Jerome Marchand Cc: Kirill A. Shutemov Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Cc: stable # v4.1+ Patchwork: https://patchwork.linux-mips.org/patch/12721/ Signed-off-by: Ralf Baechle