kernel.git/arch/x86/virt, branch linux-6.15.y

x86/virt/tdx: Avoid indirect calls to TDX assembly functions

2025-06-27T10:13:34Z

commit 0b3bc018e86afdc0cbfef61328c63d5c08f8b370 upstream. Two 'static inline' TDX helper functions (sc_retry() and sc_retry_prerr()) take function pointer arguments which refer to assembly functions. Normally, the compiler inlines the TDX helper, realizes that the function pointer targets are completely static -- thus can be resolved at compile time -- and generates direct call instructions. But, other times (like when CONFIG_CC_OPTIMIZE_FOR_SIZE=y), the compiler declines to inline the helpers and will instead generate indirect call instructions. Indirect calls to assembly functions require special annotation (for various Control Flow Integrity mechanisms). But TDX assembly functions lack the special annotations and can only be called directly. Annotate both the helpers as '__always_inline' to prod the compiler into maintaining the direct calls. There is no guarantee here, but Peter has volunteered to report the compiler bug if this assumption ever breaks[1]. Fixes: 1e66a7e27539 ("x86/virt/tdx: Handle SEAMCALL no entropy error in common code") Fixes: df01f5ae07dd ("x86/virt/tdx: Add SEAMCALL error printing for module initialization") Signed-off-by: Kai Huang Signed-off-by: Dave Hansen Cc: stable@vger.kernel.org Link: https://lore.kernel.org/lkml/20250605145914.GW39944@noisy.programming.kicks-ass.net/ [1] Link: https://lore.kernel.org/all/20250606130737.30713-1-kai.huang%40intel.com Signed-off-by: Greg Kroah-Hartman

x86/kexec: Export e820_table_kexec[] to sysfs

2025-02-22T11:44:45Z

Previously the e820_table_kexec[] was exported to sysfs since kexec-tools uses the memmap entries to prepare the e820 table for the new kernel. The following commit, ~8 years ago, introduced e820_table_firmware[] and changed the behavior to export the firmware table instead: 12df216c61c8 ("x86/boot/e820: Introduce the bootloader provided e820_table_firmware[] table") Originally the kexec_file_load and kexec_load syscalls both used e820_table_kexec[]. Since the sysfs exported entries are from e820_table_firmware[] people now need to tune both tables for kexec. Restore the old behavior so the kexec_load and kexec_file_load syscalls work with only one table update. The e820_table_firmware[] is used by hibernation kernel code and it works without the sysfs exporting. Also remove the SEV e820_table_firmware[] updating code. Also update the code comments here and drop the comments about setup_data reservation since it is not needed any more after this change was made a year ago: fc7f27cda843 ("x86/kexec: Do not update E820 kexec table for setup_data") [ mingo: Tidy up the changelog and comments. ] Signed-off-by: Dave Young Signed-off-by: Ingo Molnar Cc: Ashish Kalra Cc: Sean Christopherson Cc: Joerg Roedel Cc: Baoquan He Cc: Vivek Goyal Cc: Eric Biederman Link: https://lore.kernel.org/r/Z5jcb1GKhLvH8kDc@darkstar.users.ipa.redhat.com

x86/sev: Fix broken SNP support with KVM module built-in

2025-02-14T23:39:19Z

Fix issues with enabling SNP host support and effectively SNP support which is broken with respect to the KVM module being built-in. SNP host support is enabled in snp_rmptable_init() which is invoked as device_initcall(). SNP check on IOMMU is done during IOMMU PCI init (IOMMU_PCI_INIT stage). And for that reason snp_rmptable_init() is currently invoked via device_initcall() and cannot be invoked via subsys_initcall() as core IOMMU subsystem gets initialized via subsys_initcall(). Now, if kvm_amd module is built-in, it gets initialized before SNP host support is enabled in snp_rmptable_init() : [ 10.131811] kvm_amd: TSC scaling supported [ 10.136384] kvm_amd: Nested Virtualization enabled [ 10.141734] kvm_amd: Nested Paging enabled [ 10.146304] kvm_amd: LBR virtualization supported [ 10.151557] kvm_amd: SEV enabled (ASIDs 100 - 509) [ 10.156905] kvm_amd: SEV-ES enabled (ASIDs 1 - 99) [ 10.162256] kvm_amd: SEV-SNP enabled (ASIDs 1 - 99) [ 10.171508] kvm_amd: Virtual VMLOAD VMSAVE supported [ 10.177052] kvm_amd: Virtual GIF supported ... ... [ 10.201648] kvm_amd: in svm_enable_virtualization_cpu And then svm_x86_ops->enable_virtualization_cpu() (svm_enable_virtualization_cpu) programs MSR_VM_HSAVE_PA as following: wrmsrl(MSR_VM_HSAVE_PA, sd->save_area_pa); So VM_HSAVE_PA is non-zero before SNP support is enabled on all CPUs. snp_rmptable_init() gets invoked after svm_enable_virtualization_cpu() as following : ... [ 11.256138] kvm_amd: in svm_enable_virtualization_cpu ... [ 11.264918] SEV-SNP: in snp_rmptable_init This triggers a #GP exception in snp_rmptable_init() when snp_enable() is invoked to set SNP_EN in SYSCFG MSR: [ 11.294289] unchecked MSR access error: WRMSR to 0xc0010010 (tried to write 0x0000000003fc0000) at rIP: 0xffffffffaf5d5c28 (native_write_msr+0x8/0x30) ... [ 11.294404] Call Trace: [ 11.294482] [ 11.294513] ? show_stack_regs+0x26/0x30 [ 11.294522] ? ex_handler_msr+0x10f/0x180 [ 11.294529] ? search_extable+0x2b/0x40 [ 11.294538] ? fixup_exception+0x2dd/0x340 [ 11.294542] ? exc_general_protection+0x14f/0x440 [ 11.294550] ? asm_exc_general_protection+0x2b/0x30 [ 11.294557] ? __pfx_snp_enable+0x10/0x10 [ 11.294567] ? native_write_msr+0x8/0x30 [ 11.294570] ? __snp_enable+0x5d/0x70 [ 11.294575] snp_enable+0x19/0x20 [ 11.294578] __flush_smp_call_function_queue+0x9c/0x3a0 [ 11.294586] generic_smp_call_function_single_interrupt+0x17/0x20 [ 11.294589] __sysvec_call_function+0x20/0x90 [ 11.294596] sysvec_call_function+0x80/0xb0 [ 11.294601] [ 11.294603] [ 11.294605] asm_sysvec_call_function+0x1f/0x30 ... [ 11.294631] arch_cpu_idle+0xd/0x20 [ 11.294633] default_idle_call+0x34/0xd0 [ 11.294636] do_idle+0x1f1/0x230 [ 11.294643] ? complete+0x71/0x80 [ 11.294649] cpu_startup_entry+0x30/0x40 [ 11.294652] start_secondary+0x12d/0x160 [ 11.294655] common_startup_64+0x13e/0x141 [ 11.294662] This #GP exception is getting triggered due to the following errata for AMD family 19h Models 10h-1Fh Processors: Processor may generate spurious #GP(0) Exception on WRMSR instruction: Description: The Processor will generate a spurious #GP(0) Exception on a WRMSR instruction if the following conditions are all met: - the target of the WRMSR is a SYSCFG register. - the write changes the value of SYSCFG.SNPEn from 0 to 1. - One of the threads that share the physical core has a non-zero value in the VM_HSAVE_PA MSR. The document being referred to above: https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/revision-guides/57095-PUB_1_01.pdf To summarize, with kvm_amd module being built-in, KVM/SVM initialization happens before host SNP is enabled and this SVM initialization sets VM_HSAVE_PA to non-zero, which then triggers a #GP when SYSCFG.SNPEn is being set and this will subsequently cause SNP_INIT(_EX) to fail with INVALID_CONFIG error as SYSCFG[SnpEn] is not set on all CPUs. Essentially SNP host enabling code should be invoked before KVM initialization, which is currently not the case when KVM is built-in. Add fix to call snp_rmptable_init() early from iommu_snp_enable() directly and not invoked via device_initcall() which enables SNP host support before KVM initialization with kvm_amd module built-in. Add additional handling for `iommu=off` or `amd_iommu=off` options. Note that IOMMUs need to be enabled for SNP initialization, therefore, if host SNP support is enabled but late IOMMU initialization fails then that will cause PSP driver's SNP_INIT to fail as IOMMU SNP sanity checks in SNP firmware will fail with invalid configuration error as below: [ 9.723114] ccp 0000:23:00.1: sev enabled [ 9.727602] ccp 0000:23:00.1: psp enabled [ 9.732527] ccp 0000:a2:00.1: enabling device (0000 -> 0002) [ 9.739098] ccp 0000:a2:00.1: no command queues available [ 9.745167] ccp 0000:a2:00.1: psp enabled [ 9.805337] ccp 0000:23:00.1: SEV-SNP: failed to INIT rc -5, error 0x3 [ 9.866426] ccp 0000:23:00.1: SEV API:1.53 build:5 Fixes: c3b86e61b756 ("x86/cpufeatures: Enable/unmask SEV-SNP CPU feature") Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Co-developed-by: Vasant Hegde Signed-off-by: Vasant Hegde Cc: Signed-off-by: Ashish Kalra Acked-by: Joerg Roedel Message-ID: <138b520fb83964782303b43ade4369cd181fdd9c.1739226950.git.ashish.kalra@amd.com> Signed-off-by: Paolo Bonzini

Merge tag 'x86_tdx_for_6.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2025-01-24T13:58:31Z

Pull x86 TDX updates from Dave Hansen: "Intel Trust Domain updates. The existing TDX code needs a _bit_ of metadata from the TDX module. But KVM is going to need a bunch more very shortly. Rework the interface with the TDX module to be more consistent and handle the new higher volume. The TDX module has added a few new features. The first is a promise not to clobber RBP under any circumstances. Basically the kernel now will refuse to use any modules that don't have this promise. Second, enable the new "REDUCE_VE" feature. This ensures that the TDX module will not send some silly virtualization exceptions that the guest had no good way to handle anyway. - Centralize global metadata infrastructure - Use new TDX module features for exception suppression and RBP clobbering" * tag 'x86_tdx_for_6.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: x86/virt/tdx: Require the module to assert it has the NO_RBP_MOD mitigation x86/virt/tdx: Switch to use auto-generated global metadata reading code x86/virt/tdx: Use dedicated struct members for PAMT entry sizes x86/virt/tdx: Use auto-generated code to read global metadata x86/virt/tdx: Start to track all global metadata in one structure x86/virt/tdx: Rename 'struct tdx_tdmr_sysinfo' to reflect the spec better x86/tdx: Dump attributes and TD_CTLS on boot x86/tdx: Disable unnecessary virtualization exceptions

x86/virt/tdx: Require the module to assert it has the NO_RBP_MOD mitigation

2024-12-18T22:36:02Z

Old TDX modules can clobber RBP in the TDH.VP.ENTER SEAMCALL. However RBP is used as frame pointer in the x86_64 calling convention, and clobbering RBP could result in bad things like being unable to unwind the stack if any non-maskable exceptions (NMI, #MC etc) happens in that gap. A new "NO_RBP_MOD" feature was introduced to more recent TDX modules to not clobber RBP. KVM will need to use the TDH.VP.ENTER SEAMCALL to run TDX guests. It won't be safe to run TDX guests w/o this feature. To prevent it, just don't initialize the TDX module if this feature is not supported [1]. Note the bit definitions of TDX_FEATURES0 are not auto-generated in tdx_global_metadata.h. Manually define a macro for it in "tdx.h". Signed-off-by: Kai Huang Signed-off-by: Dave Hansen Reviewed-by: Nikolay Borisov Reviewed-by: Adrian Hunter Reviewed-by: Dan Williams Link: https://lore.kernel.org/fc0e8ab7-86d4-4428-be31-82e1ece6dd21@intel.com/ [1] Link: https://lore.kernel.org/all/76ae5025502c84d799e3a56a6fc4f69a82da8f93.1734188033.git.kai.huang%40intel.com

x86/virt/tdx: Switch to use auto-generated global metadata reading code

2024-12-18T22:36:02Z

Continue the process to have a centralized solution for TDX global metadata reading. Now that the new autogenerated solution is ready for use, switch to it and remove the old one. Signed-off-by: Kai Huang Signed-off-by: Dave Hansen Reviewed-by: Nikolay Borisov Reviewed-by: Dan Williams Link: https://lore.kernel.org/all/fc025d1e13b92900323f47cfe9aac3157bf08ee7.1734188033.git.kai.huang%40intel.com

x86/virt/tdx: Use dedicated struct members for PAMT entry sizes

2024-12-18T22:36:02Z

Currently, the 'struct tdmr_sys_info_tdmr' which includes TDMR related fields defines the PAMT entry sizes for TDX supported page sizes (4KB, 2MB and 1GB) as an array: struct tdx_sys_info_tdmr { ... u16 pamt_entry_sizes[TDX_PS_NR]; }; PAMT entry sizes are needed when allocating PAMTs for each TDMR. Using the array to contain PAMT entry sizes reduces the number of arguments that need to be passed when calling tdmr_set_up_pamt(). It also makes the code pattern like below clearer: for (pgsz = TDX_PS_4K; pgsz < TDX_PS_NR; pgsz++) { pamt_size[pgsz] = tdmr_get_pamt_sz(tdmr, pgsz, pamt_entry_size[pgsz]); tdmr_pamt_size += pamt_size[pgsz]; } However, the auto-generated metadata reading code generates a structure member for each field. The 'global_metadata.json' has a dedicated field for each PAMT entry size, and the new 'struct tdx_sys_info_tdmr' looks like: struct tdx_sys_info_tdmr { ... u16 pamt_4k_entry_size; u16 pamt_2m_entry_size; u16 pamt_1g_entry_size; }; Prepare to use the autogenerated code by making the existing 'struct tdx_sys_info_tdmr' look like the generated one. When passing to tdmrs_set_up_pamt_all(), build a local array of PAMT entry sizes from the structure so the code to allocate PAMTs can stay the same. Signed-off-by: Kai Huang Signed-off-by: Dave Hansen Reviewed-by: Nikolay Borisov Reviewed-by: Dan Williams Link: https://lore.kernel.org/all/ccf46f3dacb01be1fb8309592616d443ac17caba.1734188033.git.kai.huang%40intel.com

x86/virt/tdx: Use auto-generated code to read global metadata

2024-12-18T22:36:01Z

The TDX module provides a set of "Global Metadata Fields". They report things like TDX module version, supported features, and fields related to create/run TDX guests and so on. Currently the kernel only reads "TD Memory Region" (TDMR) related fields for module initialization. There are needs to read more global metadata fields for future use: - Supported features ("TDX_FEATURES0") to fail module initialization when the module doesn't support "not clobbering host RBP when exiting from TDX guest" feature [1]. - KVM TDX baseline support and other features like TDX Connect will need to read more. The current global metadata reading code has limitations (e.g., it only has a primitive helper to read metadata field with 16-bit element size, while TDX supports 8/16/32/64 bits metadata element sizes). It needs tweaks in order to read more metadata fields. But even with the tweaks, when new code is added to read a new field, the reviewers will still need to review against the spec to make sure the new code doesn't screw up things like using the wrong metadata field ID (each metadata field is associated with a unique field ID, which is a TDX-defined u64 constant) etc. TDX documents all global metadata fields in a 'global_metadata.json' file as part of TDX spec [2]. JSON format is machine readable. Instead of tweaking the metadata reading code, use a script to generate the code so that: 1) Using the generated C is simple. 2) Adding a field is simple, e.g., the script just pulls the field ID out of the JSON for a given field thus no manual review is needed. Specifically, to match the layout of the 'struct tdx_sys_info' and its sub-structures, the script uses a table with each entry containing the the name of the sub-structures (which reflects the "Class") and the "Field Name" of all its fields, and auto-generate: 1) The 'struct tdx_sys_info' and all 'struct tdx_sys_info_xx' sub-structures in 'tdx_global_metadata.h'. 2) The main function 'get_tdx_sys_info()' which reads all metadata to 'struct tdx_sys_info' and the 'get_tdx_sys_info_xx()' functions which read 'struct tdx_sys_info_xx()' in 'tdx_global_metadata.c'. Using the generated C is simple: 1) include "tdx_global_metadata.h" to the local "tdx.h"; 2) explicitly include "tdx_global_metadata.c" to the local "tdx.c" after the read_sys_metadata_field() primitive (which is a wrapper of TDH.SYS.RD SEAMCALL to read global metadata). Adding a field is also simple: 1) just add the new field to an existing structure, or add it with a new structure; 2) re-run the script to generate the new code; 3) update the existing tdx_global_metadata.{hc} with the new ones. For now, use the auto-generated code to read the TDMR related fields and the aforesaid metadata field "TDX_FEATURES0". The tdx_global_metadata.{hc} can be generated by running below: #python tdx_global_metadata.py global_metadata.json \ tdx_global_metadata.h tdx_global_metadata.c .. where the 'global_metadata.json' can be fetched from [2] and the 'tdx_global_metadata.py' can be found from [3]. Co-developed-by: Kai Huang Signed-off-by: Paolo Bonzini Signed-off-by: Kai Huang Signed-off-by: Dave Hansen Reviewed-by: Dan Williams Link: https://lore.kernel.org/fc0e8ab7-86d4-4428-be31-82e1ece6dd21@intel.com/ [1] Link: https://cdrdv2.intel.com/v1/dl/getContent/795381 [2] Link: https://lore.kernel.org/762a50133300710771337398284567b299a86f67.camel@intel.com/ [3] Link: https://lore.kernel.org/all/cbe3f12b1e5479399b53f4873f2ff783d9fc669b.1734188033.git.kai.huang%40intel.com

x86/virt/tdx: Start to track all global metadata in one structure

2024-12-18T22:36:01Z

The TDX module provides a set of "Global Metadata Fields". They report things like TDX module version, supported features, and fields related to create/run TDX guests and so on. Today the kernel only reads "TD Memory Region" (TDMR) related fields for module initialization. KVM will need to read additional metadata fields to run TDX guests. Move towards having the TDX host core-kernel provide a centralized, canonical, and immutable structure for the global metadata that comes out from the TDX module for all kernel components to use. As the first step, introduce a new 'struct tdx_sys_info' to track all global metadata fields. TDX categorizes global metadata fields into different "Classes". E.g., the TDMR related fields are under class "TDMR Info". Instead of making 'struct tdx_sys_info' a plain structure to contain all metadata fields, organize them in smaller structures based on the "Class". This allows those metadata fields to be used in finer granularity thus makes the code clearer. E.g., construct_tdmrs() can just take the structure which contains "TDMR Info" metadata fields. Add get_tdx_sys_info() as the placeholder to read all metadata fields. Have it only call get_tdx_sys_info_tdmr() to read TDMR related fields for now. Place get_tdx_sys_info() as the first step of init_tdx_module() to enable early prerequisite checks on the metadata to support early module initialization abort. This results in moving get_tdx_sys_info_tdmr() to be before build_tdx_memlist(), but this is fine because there are no dependencies between these two functions. Signed-off-by: Kai Huang Signed-off-by: Dave Hansen Reviewed-by: Adrian Hunter Reviewed-by: Dan Williams Link: https://lore.kernel.org/all/bfacb4e90527cf79d4be0d1753e6f318eea21118.1734188033.git.kai.huang%40intel.com

x86/virt/tdx: Rename 'struct tdx_tdmr_sysinfo' to reflect the spec better

2024-12-18T22:36:01Z

The TDX module provides a set of "Global Metadata Fields". They report things like TDX module version, supported features, and fields related to create/run TDX guests and so on. TDX organizes those metadata fields by "Classes" based on the meaning of those fields. E.g., for now the kernel only reads "TD Memory Region" (TDMR) related fields for module initialization. Those fields are defined under class "TDMR Info". Today the kernel reads some of the global metadata to initialize the TDX module. KVM will need to read additional metadata fields to run TDX guests. Move towards having the TDX host core-kernel provide a centralized, canonical, and immutable structure for the global metadata that comes out from the TDX module for all kernel components to use. More specifically, prepare the code to end up with an organization like: struct tdx_sys_info { struct tdx_sys_info_classA a; struct tdx_sys_info_classB b; ... }; Currently the kernel organizes all fields under "TDMR Info" class in 'struct tdx_tdmr_sysinfo'. Prepare for the above by renaming the structure to 'struct tdx_sys_info_tdmr' to follow the class name better. No functional change intended. Signed-off-by: Kai Huang Signed-off-by: Dave Hansen Reviewed-by: Adrian Hunter Reviewed-by: Dan Williams Link: https://lore.kernel.org/all/de165d09e0b571cfeb119a368f4be6e2888ebb93.1734188033.git.kai.huang%40intel.com