kernel.git/arch/s390/net, branch linux-5.8.y

s390/bpf: Tolerate not converging code shrinking

2020-08-19T06:26:42Z

[ Upstream commit 1491b73311a15bb5beeab5d30e03bff761ef6c18 ] "BPF_MAXINSNS: Maximum possible literals" unnecessarily falls back to the interpreter because of failing sanity check in bpf_set_addr. The problem is that there are a lot of branches that can be shrunk, and doing so opens up the possibility to shrink even more. This process does not converge after 3 passes, causing code offsets to change during the codegen pass, which must never happen. Fix by inserting nops during codegen pass in order to preserve code offets. Fixes: 4e9b4a6883dd ("s390/bpf: Use relative long branches") Signed-off-by: Ilya Leoshkevich Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20200717165326.6786-5-iii@linux.ibm.com Signed-off-by: Sasha Levin

s390/bpf: Use brcl for jumping to exit_ip if necessary

2020-08-19T06:26:42Z

[ Upstream commit 5fa6974471c5518a50bdd814067508dbcb477251 ] "BPF_MAXINSNS: Maximum possible literals" test causes panic with bpf_jit_harden = 2. The reason is that BPF_JMP | BPF_EXIT is always emitted as brc, however, after removal of JITed image size limitations, brcl might be required. Fix by using brcl when necessary. Fixes: 4e9b4a6883dd ("s390/bpf: Use relative long branches") Signed-off-by: Ilya Leoshkevich Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20200717165326.6786-4-iii@linux.ibm.com Signed-off-by: Sasha Levin

s390/bpf: Fix sign extension in branch_ku

2020-08-19T06:26:41Z

[ Upstream commit 7477d43be5b1448bc0d4c85cb185a0144cc080e1 ] Both signed and unsigned variants of BPF_JMP | BPF_K require sign-extending the immediate. JIT emits cgfi for the signed case, which is correct, and clgfi for the unsigned case, which is not correct: clgfi zero-extends the immediate. s390 does not provide an instruction that does sign-extension and unsigned comparison at the same time. Therefore, fix by first loading the sign-extended immediate into work register REG_1 and proceeding as if it's BPF_X. Fixes: 4e9b4a6883dd ("s390/bpf: Use relative long branches") Reported-by: Seth Forshee Signed-off-by: Ilya Leoshkevich Signed-off-by: Alexei Starovoitov Tested-by: Seth Forshee Link: https://lore.kernel.org/bpf/20200717165326.6786-3-iii@linux.ibm.com Signed-off-by: Sasha Levin

s390/bpf: Use bcr 0,%0 as tail call nop filler

2020-06-02T19:04:04Z

Currently used 0x0000 filler confuses bfd disassembler, making bpftool prog dump xlated output nearly useless. Fix by using a real instruction. Signed-off-by: Ilya Leoshkevich Signed-off-by: Daniel Borkmann Link: https://lore.kernel.org/bpf/20200602174555.2501389-1-iii@linux.ibm.com

s390/bpf: Maintain 8-byte stack alignment

2020-06-02T19:00:56Z

Certain kernel functions (e.g. get_vtimer/set_vtimer) cause kernel panic when the stack is not 8-byte aligned. Currently JITed BPF programs may trigger this by allocating stack frames with non-rounded sizes and then being interrupted. Fix by using rounded fp->aux->stack_depth. Signed-off-by: Ilya Leoshkevich Signed-off-by: Daniel Borkmann Link: https://lore.kernel.org/bpf/20200602174339.2501066-1-iii@linux.ibm.com

s390/bpf: Remove JITed image size limitations

2019-11-19T03:51:16Z

Now that jump and long displacement ranges are no longer a problem, remove the limit on JITed image size. In practice it's still limited by 2G, but with verifier allowing "only" 1M instructions, it's not an issue. Signed-off-by: Ilya Leoshkevich Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20191118180340.68373-7-iii@linux.ibm.com

s390/bpf: Use lg(f)rl when long displacement cannot be used

2019-11-19T03:51:16Z

If literal pool grows past 524287 mark, it's no longer possible to use long displacement to reference literal pool entries. In JIT setting maintaining multiple literal pool registers is next to impossible, since we operate on one instruction at a time. Therefore, fall back to loading literal pool entry using PC-relative addressing, and then using a register-register form of the following machine instruction. Signed-off-by: Ilya Leoshkevich Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20191118180340.68373-6-iii@linux.ibm.com

s390/bpf: Use lgrl instead of lg where possible

2019-11-19T03:51:16Z

lg and lgrl have the same performance characteristics, but the former requires a base register and is subject to long displacement range limits, while the latter does not. Therefore, lgrl is totally superior to lg and should be used instead whenever possible. Signed-off-by: Ilya Leoshkevich Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20191118180340.68373-5-iii@linux.ibm.com

s390/bpf: Load literal pool register using larl

2019-11-19T03:51:16Z

Currently literal pool register is loaded using basr, which makes it point not to the beginning of the literal pool, but rather to the next instruction. In case JITed code is larger than 512k, this renders literal pool register absolutely useless due to long displacement range restrictions. The solution is to use larl to make literal pool register point to the very beginning of the literal pool. This makes it always possible to address 512k worth of literal pool entries using long displacement. However, for short programs, in which the entire literal pool is covered by basr-generated base, it is still beneficial to use basr, since it is 4 bytes shorter than larl. Detect situations when basr-generated base does not cover the entire literal pool, and in such cases use larl instead. Signed-off-by: Ilya Leoshkevich Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20191118180340.68373-4-iii@linux.ibm.com

s390/bpf: Align literal pool entries

2019-11-19T03:51:16Z

When literal pool size exceeds 512k, it's no longer possible to reference all the entries in it using a single base register and long displacement. Therefore, PC-relative lgfrl and lgrl instructions need to be used. Unfortunately, they require their arguments to be aligned to 4- and 8-byte boundaries respectively. This generates certain overhead due to necessary padding bytes. Grouping 4- and 8-byte entries together reduces the maximum overhead to 6 bytes (2 for aligning 4-byte entries and 4 for aligning 8-byte entries). While in theory it is possible to detect whether or not alignment is needed by comparing the literal pool size with 512k, in practice this leads to having two ways of emitting constants, making the code more complicated. Prefer code simplicity over trivial size saving, and always group and align literal pool entries. Signed-off-by: Ilya Leoshkevich Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20191118180340.68373-3-iii@linux.ibm.com