kernel.git/arch/arm/mm/context.c, branch linux-3.17.y

ARM: 7926/1: mm: flesh out and fix the comments in the ASID allocator

2013-12-29T12:46:49Z

The ASID allocator has to deal with some pretty horrible behaviours by the CPU, so expand on some of the comments in there so I remember why we can never allocate ASID zero to a userspace task. Reviewed-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Russell King

ARM: 7925/1: mm: keep track of last ASID allocation to improve bitmap searching

2013-12-29T12:46:48Z

Since we only clear entries in the ASID bitmap on a rollover event, the bitmap tends to consist of a block of consecutive set bits followed by a block of consecutive clear bits. The exception to this rule is for ASIDs which have been carried over from a previous generation, but these are bound by the number of CPUs. This patch optimises our bitmap searching strategy, so that we search from the last successful allocation, rather than search from index 1 each time we allocate a new ASID. Reviewed-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Russell King

ARM: 7924/1: mm: don't bother with reserved ttbr0 when running with LPAE

2013-12-29T12:46:47Z

With the new ASID allocation algorithm, active ASIDs at the time of a rollover event will be marked as reserved, so active mm_structs can continue to operate with the same ASID as before. This in turn means that we don't need to worry about allocating a new ASID to an mm that is currently active (installed in TTBR0). Since updating the pgd and ASID is atomic on LPAE systems (by virtue of the two being fields in the same hardware register), we can dispose of the reserved TTBR0 and rely on whatever tables we currently have live. Reviewed-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Russell King

ARM: tlb: don't perform inner-shareable invalidation for local TLB ops

2013-08-12T11:25:44Z

Inner-shareable TLB invalidation is typically more expensive than local (non-shareable) invalidation, so performing the broadcasting for local_flush_tlb_* operations is a waste of cycles and needlessly clobbers entries in the TLBs of other CPUs. This patch introduces __flush_tlb_* versions for many of the TLB invalidation functions, which only respect inner-shareable variants of the invalidation instructions when presented with the TLB_V7_UIS_FULL flag. The local version is also inlined to prevent SMP_ON_UP kernels from missing flushes, where the __flush variant would be called with the UP flags. This gains us around 0.5% in hackbench scores for a dual-core A15, but I would expect this to improve as more cores (and clusters) are added to the equation. Reviewed-by: Catalin Marinas Reported-by: Albin Tonnerre Signed-off-by: Will Deacon

ARM: 7789/1: Do not run dummy_flush_tlb_a15_erratum() on non-Cortex-A15

2013-07-26T11:02:09Z

Commit 93dc688 (ARM: 7684/1: errata: Workaround for Cortex-A15 erratum 798181 (TLBI/DSB operations)) causes the following undefined instruction error on a mx53 (Cortex-A8): Internal error: Oops - undefined instruction: 0 [#1] SMP ARM CPU: 0 PID: 275 Comm: modprobe Not tainted 3.11.0-rc2-next-20130722-00009-g9b0f371 #881 task: df46cc00 ti: df48e000 task.ti: df48e000 PC is at check_and_switch_context+0x17c/0x4d0 LR is at check_and_switch_context+0xdc/0x4d0 This problem happens because check_and_switch_context() calls dummy_flush_tlb_a15_erratum() without checking if we are really running on a Cortex-A15 or not. To avoid this issue, only call dummy_flush_tlb_a15_erratum() inside check_and_switch_context() if erratum_a15_798181() returns true, which means that we are really running on a Cortex-A15. Signed-off-by: Fabio Estevam Acked-by: Catalin Marinas Reviewed-by: Roger Quadros Signed-off-by: Russell King

Merge branch 'devel-stable' into for-next

2013-06-29T10:44:43Z

Conflicts: arch/arm/Makefile arch/arm/include/asm/glue-proc.h

ARM: 7769/1: Cortex-A15: fix erratum 798181 implementation

2013-06-24T14:27:35Z

Looking into the active_asids array is not enough, as we also need to look into the reserved_asids array (they both represent processes that are currently running). Also, not holding the ASID allocator lock is racy, as another CPU could schedule that process and trigger a rollover, making the erratum workaround miss an IPI. Exposing this outside of context.c is a little ugly on the side, so let's define a new entry point that the erratum workaround can call to obtain the cpumask. Cc: # 3.9 Acked-by: Will Deacon Acked-by: Catalin Marinas Signed-off-by: Marc Zyngier Signed-off-by: Russell King

ARM: 7768/1: prevent risks of out-of-bound access in ASID allocator

2013-06-24T14:26:41Z

On a CPU that never ran anything, both the active and reserved ASID fields are set to zero. In this case the ASID_TO_IDX() macro will return -1, which is not a very useful value to index a bitmap. Instead of trying to offset the ASID so that ASID #1 is actually bit 0 in the asid_map bitmap, just always ignore bit 0 and start the search from bit 1. This makes the code a bit more readable, and without risk of OoB access. Cc: # 3.9 Acked-by: Will Deacon Acked-by: Catalin Marinas Reported-by: Catalin Marinas Signed-off-by: Marc Zyngier Signed-off-by: Russell King

ARM: 7767/1: let the ASID allocator handle suspended animation

2013-06-24T14:24:54Z

When a CPU is running a process, the ASID for that process is held in a per-CPU variable (the "active ASIDs" array). When the ASID allocator handles a rollover, it copies the active ASIDs into a "reserved ASIDs" array to ensure that a process currently running on another CPU will continue to run unaffected. The active array is zero-ed to indicate that a rollover occurred. Because of this mechanism, a reserved ASID is only remembered for a single rollover. A subsequent rollover will completely refill the reserved ASIDs array. In a severely oversubscribed environment where a CPU can be prevented from running for extended periods of time (think virtual machines), the above has a horrible side effect: [P{a} denotes process P running with ASID a] CPU-0 CPU-1 A{x} [active = ] [suspended] runs B{y} [active = ] [rollover: active = <0 0> reserved = ] runs B{y} [active = <0 y> reserved = ] [rollover: active = <0 0> reserved = <0 y>] runs C{x} [active = <0 x>] [resumes] runs A{x} At that stage, both A and C have the same ASID, with deadly consequences. The fix is to preserve reserved ASIDs across rollovers if the CPU doesn't have an active ASID when the rollover occurs. Cc: # 3.9 Acked-by: Will Deacon Acked-by: Catalin Carinas Signed-off-by: Marc Zyngier Signed-off-by: Russell King

ARM: LPAE: use 64-bit accessors for TTBR registers

2013-05-30T15:02:07Z

This patch adds TTBR accessor macros, and modifies cpu_get_pgd() and the LPAE version of cpu_set_reserved_ttbr0() to use these instead. In the process, we also fix these functions to correctly handle cases where the physical address lies beyond the 4G limit of 32-bit addressing. Signed-off-by: Cyril Chemparathy Signed-off-by: Vitaly Andrianov Acked-by: Nicolas Pitre Reviewed-by: Catalin Marinas Tested-by: Santosh Shilimkar Tested-by: Subash Patel Signed-off-by: Will Deacon