kernel.git/arch/x86/Makefile, branch linux-4.14.y

x86/build: Propagate $(CLANG_FLAGS) to $(REALMODE_FLAGS)

2021-05-22T08:57:16Z

[ Upstream commit 8abe7fc26ad8f28bfdf78adbed56acd1fa93f82d ] When cross-compiling with Clang, the `$(CLANG_FLAGS)' variable contains additional flags needed to build C and assembly sources for the target platform. Normally this variable is automatically included in `$(KBUILD_CFLAGS)' via the top-level Makefile. The x86 real-mode makefile builds `$(REALMODE_CFLAGS)' from a plain assignment and therefore drops the Clang flags. This causes Clang to not recognize x86-specific assembler directives: arch/x86/realmode/rm/header.S:36:1: error: unknown directive .type real_mode_header STT_OBJECT ; .size real_mode_header, .-real_mode_header ^ Explicit propagation of `$(CLANG_FLAGS)' to `$(REALMODE_CFLAGS)', which is inherited by real-mode make rules, fixes cross-compilation with Clang for x86 targets. Relevant flags: * `--target' sets the target architecture when cross-compiling. This flag must be set for both compilation and assembly (`KBUILD_AFLAGS') to support architecture-specific assembler directives. * `-no-integrated-as' tells clang to assemble with GNU Assembler instead of its built-in LLVM assembler. This flag is set by default unless `LLVM_IAS=1' is set, because the LLVM assembler can't yet parse certain GNU extensions. Signed-off-by: John Millikin Signed-off-by: Nathan Chancellor Signed-off-by: Borislav Petkov Acked-by: Ard Biesheuvel Tested-by: Sedat Dilek Link: https://lkml.kernel.org/r/20210326000435.4785-2-nathan@kernel.org Signed-off-by: Sasha Levin

x86/build: Turn off -fcf-protection for realmode targets

2021-04-10T11:20:11Z

[ Upstream commit 9fcb51c14da2953de585c5c6e50697b8a6e91a7b ] The new Ubuntu GCC packages turn on -fcf-protection globally, which causes a build failure in the x86 realmode code: cc1: error: ‘-fcf-protection’ is not compatible with this target Turn it off explicitly on compilers that understand this option. Signed-off-by: Arnd Bergmann Signed-off-by: Ingo Molnar Link: https://lore.kernel.org/r/20210323124846.1584944-1-arnd@kernel.org Signed-off-by: Sasha Levin

x86/build: Disable CET instrumentation in the kernel for 32-bit too

2021-02-23T13:00:33Z

commit 256b92af784d5043eeb7d559b6d5963dcc2ecb10 upstream. Commit 20bf2b378729 ("x86/build: Disable CET instrumentation in the kernel") disabled CET instrumentation which gets added by default by the Ubuntu gcc9 and 10 by default, but did that only for 64-bit builds. It would still fail when building a 32-bit target. So disable CET for all x86 builds. Fixes: 20bf2b378729 ("x86/build: Disable CET instrumentation in the kernel") Reported-by: AC Signed-off-by: Borislav Petkov Acked-by: Josh Poimboeuf Tested-by: AC Link: https://lkml.kernel.org/r/YCCIgMHkzh/xT4ex@arch-chirva.localdomain Signed-off-by: Greg Kroah-Hartman

x86/build: Disable CET instrumentation in the kernel

2021-02-10T08:12:10Z

commit 20bf2b378729c4a0366a53e2018a0b70ace94bcd upstream. With retpolines disabled, some configurations of GCC, and specifically the GCC versions 9 and 10 in Ubuntu will add Intel CET instrumentation to the kernel by default. That breaks certain tracing scenarios by adding a superfluous ENDBR64 instruction before the fentry call, for functions which can be called indirectly. CET instrumentation isn't currently necessary in the kernel, as CET is only supported in user space. Disable it unconditionally and move it into the x86's Makefile as CET/CFI... enablement should be a per-arch decision anyway. [ bp: Massage and extend commit message. ] Fixes: 29be86d7f9cb ("kbuild: add -fcf-protection=none when using retpoline flags") Reported-by: Nikolay Borisov Signed-off-by: Josh Poimboeuf Signed-off-by: Borislav Petkov Reviewed-by: Nikolay Borisov Tested-by: Nikolay Borisov Cc: Cc: Seth Forshee Cc: Masahiro Yamada Link: https://lkml.kernel.org/r/20210128215219.6kct3h2eiustncws@treble Signed-off-by: Greg Kroah-Hartman

x86/retpolines: Fix up backport of a9d57ef15cbe

2019-10-05T10:48:05Z

Commit a9d57ef15cbe ("x86/retpolines: Disable switch jump tables when retpolines are enabled") added -fno-jump-tables to workaround a GCC issue while deliberately avoiding adding this flag when CONFIG_CC_IS_CLANG is set, which is defined by the kconfig system when CC=clang is provided. However, this symbol was added in 4.18 in commit 469cb7376c06 ("kconfig: add CC_IS_CLANG and CLANG_VERSION") so it is always undefined in 4.14, meaning -fno-jump-tables gets added when using Clang. Fix this up by using the equivalent $(cc-name) comparison, which matches what upstream did until commit 076f421da5d4 ("kbuild: replace cc-name test with CONFIG_CC_IS_CLANG"). Fixes: e28951100515 ("x86/retpolines: Disable switch jump tables when retpolines are enabled") Signed-off-by: Nathan Chancellor Signed-off-by: Greg Kroah-Hartman

x86/build: Add -Wnoaddress-of-packed-member to REALMODE_CFLAGS, to silence GCC9 build warning

2019-09-19T07:08:09Z

commit 42e0e95474fc6076b5cd68cab8fa0340a1797a72 upstream. One of the very few warnings I have in the current build comes from arch/x86/boot/edd.c, where I get the following with a gcc9 build: arch/x86/boot/edd.c: In function ‘query_edd’: arch/x86/boot/edd.c:148:11: warning: taking address of packed member of ‘struct boot_params’ may result in an unaligned pointer value [-Waddress-of-packed-member] 148 | mbrptr = boot_params.edd_mbr_sig_buffer; | ^~~~~~~~~~~ This warning triggers because we throw away all the CFLAGS and then make a new set for REALMODE_CFLAGS, so the -Wno-address-of-packed-member we added in the following commit is not present: 6f303d60534c ("gcc-9: silence 'address-of-packed-member' warning") The simplest solution for now is to adjust the warning for this version of CFLAGS as well, but it would definitely make sense to examine whether REALMODE_CFLAGS could be derived from CFLAGS, so that it picks up changes in the compiler flags environment automatically. Signed-off-by: Linus Torvalds Acked-by: Borislav Petkov Cc: Peter Zijlstra Cc: Thomas Gleixner Signed-off-by: Ingo Molnar Signed-off-by: Greg Kroah-Hartman

x86/build: Keep local relocations with ld.lld

2019-05-31T13:47:29Z

[ Upstream commit 7c21383f3429dd70da39c0c7f1efa12377a47ab6 ] The LLVM linker (ld.lld) defaults to removing local relocations, which causes KASLR boot failures. ld.bfd and ld.gold already handle this correctly. This adds the explicit instruction "--discard-none" during the link phase. There is no change in output for ld.bfd and ld.gold, but ld.lld now produces an image with all the needed relocations. Signed-off-by: Kees Cook Signed-off-by: Borislav Petkov Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: Nick Desaulniers Cc: Thomas Gleixner Cc: clang-built-linux@googlegroups.com Cc: x86-ml Link: https://lkml.kernel.org/r/20190404214027.GA7324@beast Link: https://github.com/ClangBuiltLinux/linux/issues/404 Signed-off-by: Sasha Levin

x86/retpolines: Disable switch jump tables when retpolines are enabled

2019-05-02T07:40:33Z

commit a9d57ef15cbe327fe54416dd194ee0ea66ae53a4 upstream. Commit ce02ef06fcf7 ("x86, retpolines: Raise limit for generating indirect calls from switch-case") raised the limit under retpolines to 20 switch cases where gcc would only then start to emit jump tables, and therefore effectively disabling the emission of slow indirect calls in this area. After this has been brought to attention to gcc folks [0], Martin Liska has then fixed gcc to align with clang by avoiding to generate switch jump tables entirely under retpolines. This is taking effect in gcc starting from stable version 8.4.0. Given kernel supports compilation with older versions of gcc where the fix is not being available or backported anymore, we need to keep the extra KBUILD_CFLAGS around for some time and generally set the -fno-jump-tables to align with what more recent gcc is doing automatically today. More than 20 switch cases are not expected to be fast-path critical, but it would still be good to align with gcc behavior for versions < 8.4.0 in order to have consistency across supported gcc versions. vmlinux size is slightly growing by 0.27% for older gcc. This flag is only set to work around affected gcc, no change for clang. [0] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86952 Suggested-by: Martin Liska Signed-off-by: Daniel Borkmann Signed-off-by: Thomas Gleixner Cc: David Woodhouse Cc: Linus Torvalds Cc: Jesper Dangaard Brouer Cc: Björn Töpel Cc: Magnus Karlsson Cc: Alexei Starovoitov Cc: H.J. Lu Cc: Alexei Starovoitov Cc: David S. Miller Link: https://lkml.kernel.org/r/20190325135620.14882-1-daniel@iogearbox.net Signed-off-by: Greg Kroah-Hartman

x86, retpolines: Raise limit for generating indirect calls from switch-case

2019-05-02T07:40:32Z

commit ce02ef06fcf7a399a6276adb83f37373d10cbbe1 upstream. From networking side, there are numerous attempts to get rid of indirect calls in fast-path wherever feasible in order to avoid the cost of retpolines, for example, just to name a few: * 283c16a2dfd3 ("indirect call wrappers: helpers to speed-up indirect calls of builtin") * aaa5d90b395a ("net: use indirect call wrappers at GRO network layer") * 028e0a476684 ("net: use indirect call wrappers at GRO transport layer") * 356da6d0cde3 ("dma-mapping: bypass indirect calls for dma-direct") * 09772d92cd5a ("bpf: avoid retpoline for lookup/update/delete calls on maps") * 10870dd89e95 ("netfilter: nf_tables: add direct calls for all builtin expressions") [...] Recent work on XDP from Björn and Magnus additionally found that manually transforming the XDP return code switch statement with more than 5 cases into if-else combination would result in a considerable speedup in XDP layer due to avoidance of indirect calls in CONFIG_RETPOLINE enabled builds. On i40e driver with XDP prog attached, a 20-26% speedup has been observed [0]. Aside from XDP, there are many other places later in the networking stack's critical path with similar switch-case processing. Rather than fixing every XDP-enabled driver and locations in stack by hand, it would be good to instead raise the limit where gcc would emit expensive indirect calls from the switch under retpolines and stick with the default as-is in case of !retpoline configured kernels. This would also have the advantage that for archs where this is not necessary, we let compiler select the underlying target optimization for these constructs and avoid potential slow-downs by if-else hand-rewrite. In case of gcc, this setting is controlled by case-values-threshold which has an architecture global default that selects 4 or 5 (latter if target does not have a case insn that compares the bounds) where some arch back ends like arm64 or s390 override it with their own target hooks, for example, in gcc commit db7a90aa0de5 ("S/390: Disable prediction of indirect branches") the threshold pretty much disables jump tables by limit of 20 under retpoline builds. Comparing gcc's and clang's default code generation on x86-64 under O2 level with retpoline build results in the following outcome for 5 switch cases: * gcc with -mindirect-branch=thunk-inline -mindirect-branch-register: # gdb -batch -ex 'disassemble dispatch' ./c-switch Dump of assembler code for function dispatch: 0x0000000000400be0 <+0>: cmp $0x4,%edi 0x0000000000400be3 <+3>: ja 0x400c35 0x0000000000400be5 <+5>: lea 0x915f8(%rip),%rdx # 0x4921e4 0x0000000000400bec <+12>: mov %edi,%edi 0x0000000000400bee <+14>: movslq (%rdx,%rdi,4),%rax 0x0000000000400bf2 <+18>: add %rdx,%rax 0x0000000000400bf5 <+21>: callq 0x400c01 0x0000000000400bfa <+26>: pause 0x0000000000400bfc <+28>: lfence 0x0000000000400bff <+31>: jmp 0x400bfa 0x0000000000400c01 <+33>: mov %rax,(%rsp) 0x0000000000400c05 <+37>: retq 0x0000000000400c06 <+38>: nopw %cs:0x0(%rax,%rax,1) 0x0000000000400c10 <+48>: jmpq 0x400c90 0x0000000000400c15 <+53>: nopl (%rax) 0x0000000000400c18 <+56>: jmpq 0x400c70 0x0000000000400c1d <+61>: nopl (%rax) 0x0000000000400c20 <+64>: jmpq 0x400c50 0x0000000000400c25 <+69>: nopl (%rax) 0x0000000000400c28 <+72>: jmpq 0x400c40 0x0000000000400c2d <+77>: nopl (%rax) 0x0000000000400c30 <+80>: jmpq 0x400cb0 0x0000000000400c35 <+85>: push %rax 0x0000000000400c36 <+86>: callq 0x40dd80 End of assembler dump. * clang with -mretpoline emitting search tree: # gdb -batch -ex 'disassemble dispatch' ./c-switch Dump of assembler code for function dispatch: 0x0000000000400b30 <+0>: cmp $0x1,%edi 0x0000000000400b33 <+3>: jle 0x400b44 0x0000000000400b35 <+5>: cmp $0x2,%edi 0x0000000000400b38 <+8>: je 0x400b4d 0x0000000000400b3a <+10>: cmp $0x3,%edi 0x0000000000400b3d <+13>: jne 0x400b52 0x0000000000400b3f <+15>: jmpq 0x400c50 0x0000000000400b44 <+20>: test %edi,%edi 0x0000000000400b46 <+22>: jne 0x400b5c 0x0000000000400b48 <+24>: jmpq 0x400c20 0x0000000000400b4d <+29>: jmpq 0x400c40 0x0000000000400b52 <+34>: cmp $0x4,%edi 0x0000000000400b55 <+37>: jne 0x400b66 0x0000000000400b57 <+39>: jmpq 0x400c60 0x0000000000400b5c <+44>: cmp $0x1,%edi 0x0000000000400b5f <+47>: jne 0x400b66 0x0000000000400b61 <+49>: jmpq 0x400c30 0x0000000000400b66 <+54>: push %rax 0x0000000000400b67 <+55>: callq 0x40dd20 End of assembler dump. For sake of comparison, clang without -mretpoline: # gdb -batch -ex 'disassemble dispatch' ./c-switch Dump of assembler code for function dispatch: 0x0000000000400b30 <+0>: cmp $0x4,%edi 0x0000000000400b33 <+3>: ja 0x400b57 0x0000000000400b35 <+5>: mov %edi,%eax 0x0000000000400b37 <+7>: jmpq *0x492148(,%rax,8) 0x0000000000400b3e <+14>: jmpq 0x400bf0 0x0000000000400b43 <+19>: jmpq 0x400c30 0x0000000000400b48 <+24>: jmpq 0x400c10 0x0000000000400b4d <+29>: jmpq 0x400c20 0x0000000000400b52 <+34>: jmpq 0x400c00 0x0000000000400b57 <+39>: push %rax 0x0000000000400b58 <+40>: callq 0x40dcf0 End of assembler dump. Raising the cases to a high number (e.g. 100) will still result in similar code generation pattern with clang and gcc as above, in other words clang generally turns off jump table emission by having an extra expansion pass under retpoline build to turn indirectbr instructions from their IR into switch instructions as a built-in -mno-jump-table lowering of a switch (in this case, even if IR input already contained an indirect branch). For gcc, adding --param=case-values-threshold=20 as in similar fashion as s390 in order to raise the limit for x86 retpoline enabled builds results in a small vmlinux size increase of only 0.13% (before=18,027,528 after=18,051,192). For clang this option is ignored due to i) not being needed as mentioned and ii) not having above cmdline parameter. Non-retpoline-enabled builds with gcc continue to use the default case-values-threshold setting, so nothing changes here. [0] https://lore.kernel.org/netdev/20190129095754.9390-1-bjorn.topel@gmail.com/ and "The Path to DPDK Speeds for AF_XDP", LPC 2018, networking track: - http://vger.kernel.org/lpc_net2018_talks/lpc18_pres_af_xdp_perf-v3.pdf - http://vger.kernel.org/lpc_net2018_talks/lpc18_paper_af_xdp_perf-v2.pdf Signed-off-by: Daniel Borkmann Signed-off-by: Thomas Gleixner Acked-by: Jesper Dangaard Brouer Acked-by: Björn Töpel Acked-by: Linus Torvalds Cc: netdev@vger.kernel.org Cc: David S. Miller Cc: Magnus Karlsson Cc: Alexei Starovoitov Cc: Peter Zijlstra Cc: David Woodhouse Cc: Andy Lutomirski Cc: Borislav Petkov Link: https://lkml.kernel.org/r/20190221221941.29358-1-daniel@iogearbox.net Signed-off-by: Greg Kroah-Hartman

x86/build: Fix compiler support check for CONFIG_RETPOLINE

2018-12-21T13:13:07Z

commit 25896d073d8a0403b07e6dec56f58e6c33678207 upstream. It is troublesome to add a diagnostic like this to the Makefile parse stage because the top-level Makefile could be parsed with a stale include/config/auto.conf. Once you are hit by the error about non-retpoline compiler, the compilation still breaks even after disabling CONFIG_RETPOLINE. The easiest fix is to move this check to the "archprepare" like this commit did: 829fe4aa9ac1 ("x86: Allow generating user-space headers without a compiler") Reported-by: Meelis Roos Tested-by: Meelis Roos Signed-off-by: Masahiro Yamada Acked-by: Zhenzhong Duan Cc: Borislav Petkov Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Zhenzhong Duan Fixes: 4cd24de3a098 ("x86/retpoline: Make CONFIG_RETPOLINE depend on compiler support") Link: http://lkml.kernel.org/r/1543991239-18476-1-git-send-email-yamada.masahiro@socionext.com Link: https://lkml.org/lkml/2018/12/4/206 Signed-off-by: Ingo Molnar Signed-off-by: Sasha Levin Cc: Gi-Oh Kim Signed-off-by: Greg Kroah-Hartman