kernel.git/arch/parisc/kernel/pacache.S, branch linux-4.16.y

parisc: Fix ordering of cache and TLB flushes

2018-03-02T09:03:28Z

The change to flush_kernel_vmap_range() wasn't sufficient to avoid the SMP stalls. The problem is some drivers call these routines with interrupts disabled. Interrupts need to be enabled for flush_tlb_all() and flush_cache_all() to work. This version adds checks to ensure interrupts are not disabled before calling routines that need IPI interrupts. When interrupts are disabled, we now drop into slower code. The attached change fixes the ordering of cache and TLB flushes in several cases. When we flush the cache using the existing PTE/TLB entries, we need to flush the TLB after doing the cache flush. We don't need to do this when we flush the entire instruction and data caches as these flushes don't use the existing TLB entries. The same is true for tmpalias region flushes. The flush_kernel_vmap_range() and invalidate_kernel_vmap_range() routines have been updated. Secondly, we added a new purge_kernel_dcache_range_asm() routine to pacache.S and use it in invalidate_kernel_vmap_range(). Nominally, purges are faster than flushes as the cache lines don't have to be written back to memory. Hopefully, this is sufficient to resolve the remaining problems due to cache speculation. So far, testing indicates that this is the case. I did work up a patch using tmpalias flushes, but there is a performance hit because we need the physical address for each page, and we also need to sequence access to the tmpalias flush code. This increases the probability of stalls. Signed-off-by: John David Anglin Cc: stable@vger.kernel.org # 4.9+ Signed-off-by: Helge Deller

parisc: Fix alignment of pa_tlb_lock in assembly on 32-bit SMP kernel

2018-01-02T21:21:54Z

Qemu for PARISC reported on a 32bit SMP parisc kernel strange failures about "Not-handled unaligned insn 0x0e8011d6 and 0x0c2011c9." Those opcodes evaluate to the ldcw() assembly instruction which requires (on 32bit) an alignment of 16 bytes to ensure atomicity. As it turns out, qemu is correct and in our assembly code in entry.S and pacache.S we don't pay attention to the required alignment. This patch fixes the problem by aligning the lock offset in assembly code in the same manner as we do in our C-code. Signed-off-by: Helge Deller Cc: # v4.0+

parisc: Remove unnecessary TLB purges from flush_dcache_page_asm and flush_icache_page_asm

2016-12-07T08:01:21Z

We have four routines in pacache.S that use temporary alias pages: copy_user_page_asm(), clear_user_page_asm(), flush_dcache_page_asm() and flush_icache_page_asm(). copy_user_page_asm() and clear_user_page_asm() don't purge the TLB entry used for the operation. flush_dcache_page_asm() and flush_icache_page_asm do purge the entry. Presumably, this was thought to optimize TLB use. However, the operation is quite heavy weight on PA 1.X processors as we need to take the TLB lock and a TLB broadcast is sent to all processors. This patch removes the purges from flush_dcache_page_asm() and flush_icache_page_asm. Signed-off-by: John David Anglin Cc: # v3.16+ Signed-off-by: Helge Deller

parisc: Also flush data TLB in flush_icache_page_asm

2016-11-25T11:32:01Z

This is the second issue I noticed in reviewing the parisc TLB code. The fic instruction may use either the instruction or data TLB in flushing the instruction cache. Thus, on machines with a split TLB, we should also flush the data TLB after setting up the temporary alias registers. Although this has no functional impact, I changed the pdtlb and pitlb instructions to consistently use the index register %r0. These instructions do not support integer displacements. Tested on rp3440 and c8000. Signed-off-by: John David Anglin Cc: # v3.16+ Signed-off-by: Helge Deller

parisc: Add cfi_startproc and cfi_endproc to assembly code

2016-10-05T20:54:40Z

Add ENTRY_CFI() and ENDPROC_CFI() macros for dwarf debug info and convert assembly users to new macros. Signed-off-by: Helge Deller

parisc: Update comment regarding implementation of copy_user_page_asm

2016-09-20T18:00:12Z

The attached patch describes the current implementation of copy_user_page_asm(). It is possible to implement this routine using either the kernel page mappings or equivalent aliases. I tested both and decided the former was more efficient. Signed-off-by: John David Anglin Signed-off-by: Helge Deller

parisc: Use unshadowed index register for flush instructions in flush_dcache_page_asm and flush_icache_page_asm

2013-06-18T18:29:10Z

The comment at the start of pacache.S states that the base and index registers used for fdc,fic, and pdc instructions should not use shadowed registers. Although this is probably unnecessary for tmpalias flushes, there is also no reason not to comply. Signed-off-by: John David Anglin Signed-off-by: Helge Deller

parisc: use PAGE_SHIFT instead of hardcoded value 12 in pacache.S

2013-05-24T20:30:28Z

additionally clean up some whitespaces & tabs. Signed-off-by: Helge Deller

parisc: fix partly 16/64k PAGE_SIZE boot

2013-05-06T21:08:32Z

This patch fixes partly PAGE_SIZEs of 16K or 64K by adjusting the assembler PTE lookup code and the assembler TEMPALIAS code. Furthermore some data alignments for PAGE_SIZE have been limited to 4K (or less) to not waste too much memory with greater page sizes. As a side note, the palo loader can (currently) only handle up to 10 ELF segments which is fixed with tighter aligning as well. My testings indicated that the ldci command in the sba iommu coding needed adjustment by the PAGE_SHIFT value and that the I/O PDIR Page size was only set to 4K for my machine (C3000). All this fixes partly the boot, but there are still quite some caching problems left. Examples are e.g. the symbios logic driver which is failing: sym0: <896> rev 0x7 at pci 0000:00:0f.0 irq 69 sym0: PA-RISC Firmware, ID 7, Fast-40, SE, parity checking CACHE TEST FAILED: DMA error (dstat=0x81).sym0: CACHE INCORRECTLY CONFIGURED. and the tulip network driver which doesn't seem to work correctly either: Sending BOOTP requests .net eth0: Setting full-duplex based on MII#1 link partner capability of 05e1 ..... timed out! Beside those kernel fixes glibc will need fixes too to be able to handle >4K page sizes. Signed-off-by: Helge Deller

parisc: fixes and cleanups in page cache flushing (2/4)

2013-02-20T21:49:29Z

Implement clear_page_asm and copy_page_asm. These are optimized routines to clear and copy a page. I tested prefetch optimizations in clear_page_asm and copy_page_asm but didn't see any significant performance improvement on rp3440. I'm not sure if these are routines are significantly faster than memset and/or memcpy, but they are there for further performance evaluation. TLB purge operations on PA 1.X SMP machines are now serialized with the help of the new tlb_lock() and tlb_unlock() macros, since on some PA-RISC machines, TLB purges need to be serialized in software. Obviously, lock isn't needed in UP kernels. On PA 2.0 machines, there is a local TLB instruction which is much less disruptive to the memory subsystem. No lock is needed for local purge. Loops are also unrolled in flush_instruction_cache_local and flush_data_cache_local. The implementation of what used to be copy_user_page (now copy_user_page_asm) is now fixed. Additionally 64-bit support is now added. Read the preceding comment which I didn't change. I left the comment but it is now inaccurate. Signed-off-by: John David Anglin Signed-off-by: Helge Deller