kernel.git/arch/arm64/kernel/irq.c, branch linux-6.3.y

Merge branch 'for-next/stacks' into for-next/core

2022-12-06T11:26:40Z

* for-next/stacks: arm64: move on_thread_stack() to arm64: remove current_top_of_stack()

arm64: move on_thread_stack() to

2022-11-18T14:36:47Z

Currently on_thread_stack() is defined in , depending upon definitiong from despite this header not being included. This ends up being fragile, and any user of on_thread_stack() must include both and . We organised things this way due to header dependencies back in commit: 0b3e336601b82c6a ("arm64: Add support for STACKLEAK gcc plugin") ... but now that we no longer use current_top_of_stack(), and given that stackleak includes via , we no longer need the definition to live in . Move on_thread_stack() to , where all its dependencies are guaranteed to be defined. This requires having arm64's irq.c explicitly include , and I've taken the opportunity to sort the includes, which were slightly out of order. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Kees Cook Cc: Will Deacon Reviewed-by: Kees Cook Reviewed-by: Mark Brown Link: https://lore.kernel.org/r/20221117120902.3974163-3-mark.rutland@arm.com Signed-off-by: Will Deacon

arm64: implement dynamic shadow call stack for Clang

2022-11-09T18:06:35Z

Implement dynamic shadow call stack support on Clang, by parsing the unwind tables at init time to locate all occurrences of PACIASP/AUTIASP instructions, and replacing them with the shadow call stack push and pop instructions, respectively. This is useful because the overhead of the shadow call stack is difficult to justify on hardware that implements pointer authentication (PAC), and given that the PAC instructions are executed as NOPs on hardware that doesn't, we can just replace them without breaking anything. As PACIASP/AUTIASP are guaranteed to be paired with respect to manipulations of the return address, replacing them 1:1 with shadow call stack pushes and pops is guaranteed to result in the desired behavior. Signed-off-by: Ard Biesheuvel Reviewed-by: Sami Tolvanen Tested-by: Sami Tolvanen Link: https://lore.kernel.org/r/20221027155908.1940624-4-ardb@kernel.org Signed-off-by: Will Deacon

arm64: run softirqs on the per-CPU IRQ stack

2022-09-09T18:01:38Z

Currently arm64 supports per-CPU IRQ stack, but softirqs are still handled in the task context. Since any call to local_bh_enable() at any level in the task's call stack may trigger a softirq processing run, which could potentially cause a task stack overflow if the combined stack footprints exceed the stack's size, let's run these softirqs on the IRQ stack as well. Signed-off-by: Qi Zheng Reviewed-by: Arnd Bergmann Acked-by: Will Deacon Link: https://lore.kernel.org/r/20220815124739.15948-1-zhengqi.arch@bytedance.com Signed-off-by: Catalin Marinas

arm64: irq: allow FIQs to be handled

2021-03-24T20:19:30Z

On contemporary platforms we don't use FIQ, and treat any stray FIQ as a fatal event. However, some platforms have an interrupt controller wired to FIQ, and need to handle FIQ as part of regular operation. So that we can support both cases dynamically, this patch updates the FIQ exception handling code to operate the same way as the IRQ handling code, with its own handle_arch_fiq handler. Where a root FIQ handler is not registered, an unexpected FIQ exception will trigger the default FIQ handler, which will panic() as today. Where a root FIQ handler is registered, handling of the FIQ is deferred to that handler. As el0_fiq_invalid_compat is supplanted by el0_fiq, the former is removed. For !CONFIG_COMPAT builds we never expect to take an exception from AArch32 EL0, so we keep the common el0_fiq_invalid handler. Signed-off-by: Mark Rutland Tested-by: Hector Martin Cc: James Morse Cc: Marc Zyngier Cc: Thomas Gleixner Cc: Will Deacon Acked-by: Will Deacon Link: https://lore.kernel.org/r/20210315115629.57191-7-mark.rutland@arm.com Signed-off-by: Catalin Marinas

arm64: irq: rework root IRQ handler registration

2021-03-24T20:19:30Z

If we accidentally unmask IRQs before we've registered a root IRQ handler, handle_arch_irq will be NULL, and the IRQ exception handler will branch to a bogus address. To make this easier to debug, this patch initialises handle_arch_irq to a default handler which will panic(), making such problems easier to debug. When we add support for FIQ handlers, we can follow the same approach. When we add support for a root FIQ handler, it's possible to have root IRQ handler without an root FIQ handler, and in theory the inverse is also possible. To permit this, and to keep the IRQ/FIQ registration logic similar, this patch removes the panic in the absence of a root IRQ controller. Instead, set_handle_irq() logs when a handler is registered, which is sufficient for debug purposes. Signed-off-by: Mark Rutland Tested-by: Hector Martin Cc: James Morse Cc: Marc Zyngier Cc: Thomas Gleixner Cc: Will Deacon Acked-by: Will Deacon Link: https://lore.kernel.org/r/20210315115629.57191-4-mark.rutland@arm.com Signed-off-by: Catalin Marinas

arm64: don't use GENERIC_IRQ_MULTI_HANDLER

2021-03-24T20:19:30Z

In subsequent patches we want to allow irqchip drivers to register as FIQ handlers, with a set_handle_fiq() function. To keep the IRQ/FIQ paths similar, we want arm64 to provide both set_handle_irq() and set_handle_fiq(), rather than using GENERIC_IRQ_MULTI_HANDLER for the former. This patch adds an arm64-specific implementation of set_handle_irq(). There should be no functional change as a result of this patch. Signed-off-by: Marc Zyngier [Mark: use a single handler pointer] Signed-off-by: Mark Rutland Tested-by: Hector Martin Cc: James Morse Cc: Thomas Gleixner Cc: Will Deacon Link: https://lore.kernel.org/r/20210315115629.57191-3-mark.rutland@arm.com Signed-off-by: Catalin Marinas

Merge remote-tracking branch 'arm64/for-next/fixes' into for-next/core

2020-12-09T18:04:55Z

* arm64/for-next/fixes: (26 commits) arm64: mte: fix prctl(PR_GET_TAGGED_ADDR_CTRL) if TCF0=NONE arm64: mte: Fix typo in macro definition arm64: entry: fix EL1 debug transitions arm64: entry: fix NMI {user, kernel}->kernel transitions arm64: entry: fix non-NMI kernel<->kernel transitions arm64: ptrace: prepare for EL1 irq/rcu tracking arm64: entry: fix non-NMI user<->kernel transitions arm64: entry: move el1 irq/nmi logic to C arm64: entry: prepare ret_to_user for function call arm64: entry: move enter_from_user_mode to entry-common.c arm64: entry: mark entry code as noinstr arm64: mark idle code as noinstr arm64: syscall: exit userspace before unmasking exceptions arm64: pgtable: Ensure dirty bit is preserved across pte_wrprotect() arm64: pgtable: Fix pte_accessible() ACPI/IORT: Fix doc warnings in iort.c arm64/fpsimd: add to to fix fpsimd build arm64: cpu_errata: Apply Erratum 845719 to KRYO2XX Silver arm64: proton-pack: Add KRYO2XX silver CPUs to spectre-v2 safe-list arm64: kpti: Add KRYO2XX gold/silver CPU cores to kpti safelist ... # Conflicts: # arch/arm64/include/asm/exception.h # arch/arm64/kernel/sdei.c

arm64: scs: use vmapped IRQ and SDEI shadow stacks

2020-12-01T11:23:34Z

Use scs_alloc() to allocate also IRQ and SDEI shadow stacks instead of using statically allocated stacks. Signed-off-by: Sami Tolvanen Acked-by: Will Deacon Link: https://lore.kernel.org/r/20201130233442.2562064-3-samitolvanen@google.com [will: Move CONFIG_SHADOW_CALL_STACK check into init_irq_scs()] Signed-off-by: Will Deacon

arm64: entry: move el1 irq/nmi logic to C

2020-11-30T12:11:38Z

In preparation for reworking the EL1 irq/nmi entry code, move the existing logic to C. We no longer need the asm_nmi_enter() and asm_nmi_exit() wrappers, so these are removed. The new C functions are marked noinstr, which prevents compiler instrumentation and runtime probing. In subsequent patches we'll want the new C helpers to be called in all cases, so we don't bother wrapping the calls with ifdeferry. Even when the new C functions are stubs the trivial calls are unlikely to have a measurable impact on the IRQ or NMI paths anyway. Prototypes are added to as otherwise (in some configurations) GCC will complain about the lack of a forward declaration. We already do this for existing function, e.g. enter_from_user_mode(). The new helpers are marked as noinstr (which prevents all instrumentation, tracing, and kprobes). Otherwise, there should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: James Morse Cc: Will Deacon Link: https://lore.kernel.org/r/20201130115950.22492-7-mark.rutland@arm.com Signed-off-by: Will Deacon