kernel.git/arch/arm64/kvm/hyp/exception.c, branch master

KVM: arm64: Simplify sysreg access on exception delivery

2025-08-28T18:39:48Z

Distinguishing between NV and VHE is slightly pointless, and only serves as an extra complication, or a way to introduce bugs, such as the way SPSR_EL1 gets written without checking for the state being resident. Get rid if this silly distinction, and fix the bug in one go. Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/20250817121926.217900-3-maz@kernel.org Signed-off-by: Oliver Upton

KVM: arm64: Check for SYSREGS_ON_CPU before accessing the 32bit state

2025-08-28T18:39:48Z

Just like c6e35dff58d3 ("KVM: arm64: Check for SYSREGS_ON_CPU before accessing the CPU state") fixed the 64bit state access, add a check for the 32bit state actually being on the CPU before writing it. Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/20250817121926.217900-2-maz@kernel.org Signed-off-by: Oliver Upton

KVM: arm64: Check for SYSREGS_ON_CPU before accessing the CPU state

2025-07-21T16:34:57Z

Mark Brown reports that since we commit to making exceptions visible without the vcpu being loaded, the external abort selftest fails. Upon investigation, it turns out that the code that makes registers affected by an exception visible to the guest is completely broken on VHE, as we don't check whether the system registers are loaded on the CPU at this point. We managed to get away with this so far, but that's obviously as bad as it gets, Add the required checksm and document the absolute need to check for the SYSREGS_ON_CPU flag before calling into any of the __vcpu_write_sys_reg_to_cpu()__vcpu_read_sys_reg_from_cpu() helpers. Reported-by: Mark Brown Signed-off-by: Marc Zyngier Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/18535df8-e647-4643-af9a-bb780af03a70@sirena.org.uk Link: https://lore.kernel.org/r/20250720102229.179114-1-maz@kernel.org Signed-off-by: Oliver Upton

KVM: arm64: Route SEAs to the SError vector when EASE is set

2025-07-08T18:36:35Z

One of the finest additions of FEAT_DoubleFault2 is the ability for software to request *synchronous* external aborts be taken to the SError vector, which of coure are *asynchronous* in nature. Opinions be damned, implement the architecture and send SEAs to the SError vector if EASE is set for the target context. Reviewed-by: Marc Zyngier Link: https://lore.kernel.org/r/20250708172532.1699409-18-oliver.upton@linux.dev Signed-off-by: Oliver Upton

KVM: arm64: nv: Honor SError exception routing / masking

2025-07-08T18:36:31Z

To date KVM has used HCR_EL2.VSE to track the state of a pending SError for the guest. With this bit set, hardware respects the EL1 exception routing / masking rules and injects the vSError when appropriate. This isn't correct for NV guests as hardware is oblivious to vEL2's intentions for SErrors. Better yet, with FEAT_NV2 the guest can change the routing behind our back as HCR_EL2 is redirected to memory. Cope with this mess by: - Using a flag (instead of HCR_EL2.VSE) to track the pending SError state when SErrors are unconditionally masked for the current context - Resampling the routing / masking of a pending SError on every guest entry/exit - Emulating exception entry when SError routing implies a translation regime change Reviewed-by: Marc Zyngier Link: https://lore.kernel.org/r/20250708172532.1699409-7-oliver.upton@linux.dev Signed-off-by: Oliver Upton

KVM: arm64: Add assignment-specific sysreg accessor

2025-06-05T13:17:32Z

Assigning a value to a system register doesn't do what it is supposed to be doing if that register is one that has RESx bits. The main problem is that we use __vcpu_sys_reg(), which can be used both as a lvalue and rvalue. When used as a lvalue, the bit masking occurs *before* the new value is assigned, meaning that we (1) do pointless work on the old cvalue, and (2) potentially assign an invalid value as we fail to apply the masks to it. Fix this by providing a new __vcpu_assign_sys_reg() that does what it says on the tin, and sanitises the *new* value instead of the old one. This comes with a significant amount of churn. Reviewed-by: Miguel Luis Reviewed-by: Oliver Upton Link: https://lore.kernel.org/r/20250603070824.1192795-2-maz@kernel.org Signed-off-by: Marc Zyngier

KVM: arm64: nv: Support virtual EL2 exceptions

2023-02-11T09:16:11Z

Support injecting exceptions and performing exception returns to and from virtual EL2. This must be done entirely in software except when taking an exception from vEL0 to vEL2 when the virtual HCR_EL2.{E2H,TGE} == {1,1} (a VHE guest hypervisor). [maz: switch to common exception injection framework, illegal exeption return handling] Reviewed-by: Ganapatrao Kulkarni Signed-off-by: Jintack Lim Signed-off-by: Christoffer Dall Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/20230209175820.1939006-10-maz@kernel.org Signed-off-by: Oliver Upton

KVM: arm64: Fix bad dereference on MTE-enabled systems

2022-10-27T18:49:40Z

enter_exception64() performs an MTE check, which involves dereferencing vcpu->kvm. While vcpu has already been fixed up to be a HYP VA pointer, kvm is still a pointer in the kernel VA space. This only affects nVHE configurations with MTE enabled, as in other cases, the pointer is either valid (VHE) or not dereferenced (!MTE). Fix this by first converting kvm to a HYP VA pointer. Fixes: ea7fc1bb1cd1 ("KVM: arm64: Introduce MTE VM feature") Signed-off-by: Ryan Roberts Reviewed-by: Steven Price [maz: commit message tidy-up] Signed-off-by: Marc Zyngier Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20221027120945.29679-1-ryan.roberts@arm.com

KVM: arm64: Move vcpu PC/Exception flags to the input flag set

2022-06-10T08:54:34Z

The PC update flags (which also deal with exception injection) is one of the most complicated use of the flag we have. Make it more fool prof by: - moving it over to the new accessors and assign it to the input flag set - turn the combination of generic ELx flags with another flag indicating the target EL itself into an explicit set of flags for each EL and vector combination - add a new accessor to pend the exception This is otherwise a pretty straightformward conversion. Reviewed-by: Fuad Tabba Reviewed-by: Reiji Watanabe Signed-off-by: Marc Zyngier

KVM: arm64: Use shadow SPSR_EL1 when injecting exceptions on !VHE

2022-01-24T09:39:03Z

Injecting an exception into a guest with non-VHE is risky business. Instead of writing in the shadow register for the switch code to restore it, we override the CPU register instead. Which gets overriden a few instructions later by said restore code. The result is that although the guest correctly gets the exception, it will return to the original context in some random state, depending on what was there the first place... Boo. Fix the issue by writing to the shadow register. The original code is absolutely fine on VHE, as the state is already loaded, and writing to the shadow register in that case would actually be a bug. Fixes: bb666c472ca2 ("KVM: arm64: Inject AArch64 exceptions from HYP") Cc: stable@vger.kernel.org Signed-off-by: Marc Zyngier Reviewed-by: Fuad Tabba Link: https://lore.kernel.org/r/20220121184207.423426-1-maz@kernel.org