kernel.git/arch/arm64/crypto, branch linux-5.4.y

crypto: arm64/gcm - Select AEAD for GHASH_ARM64_CE

2022-08-25T09:17:41Z

[ Upstream commit fac76f2260893dde5aa05bb693b4c13e8ed0454b ] Otherwise, we could fail to compile. ld: arch/arm64/crypto/ghash-ce-glue.o: in function 'ghash_ce_mod_exit': ghash-ce-glue.c:(.exit.text+0x24): undefined reference to 'crypto_unregister_aead' ld: arch/arm64/crypto/ghash-ce-glue.o: in function 'ghash_ce_mod_init': ghash-ce-glue.c:(.init.text+0x34): undefined reference to 'crypto_register_aead' Fixes: 537c1445ab0b ("crypto: arm64/gcm - implement native driver using v8 Crypto Extensions") Signed-off-by: Qian Cai Signed-off-by: Herbert Xu Signed-off-by: Sasha Levin

crypto: arm64/sha - add missing module aliases

2021-03-04T09:26:44Z

commit 0df07d8117c3576f1603b05b84089742a118d10a upstream. The accelerated, instruction based implementations of SHA1, SHA2 and SHA3 are autoloaded based on CPU capabilities, given that the code is modest in size, and widely used, which means that resolving the algo name, loading all compatible modules and picking the one with the highest priority is taken to be suboptimal. However, if these algorithms are requested before this CPU feature based matching and autoloading occurs, these modules are not even considered, and we end up with suboptimal performance. So add the missing module aliases for the various SHA implementations. Cc: Signed-off-by: Ard Biesheuvel Signed-off-by: Herbert Xu Signed-off-by: Greg Kroah-Hartman

crypto: arm64/aes-ce - really hide slower algos when faster ones are enabled

2021-03-04T09:26:19Z

[ Upstream commit 15deb4333cd6d4e1e3216582e4c531ec40a6b060 ] Commit 69b6f2e817e5b ("crypto: arm64/aes-neon - limit exposed routines if faster driver is enabled") intended to hide modes from the plain NEON driver that are also implemented by the faster bit sliced NEON one if both are enabled. However, the defined() CPP function does not detect if the bit sliced NEON driver is enabled as a module. So instead, let's use IS_ENABLED() here. Fixes: 69b6f2e817e5b ("crypto: arm64/aes-neon - limit exposed routines if ...") Signed-off-by: Ard Biesheuvel Signed-off-by: Herbert Xu Signed-off-by: Sasha Levin

crypto: arch/nhpoly1305 - process in explicit 4k chunks

2020-05-14T05:58:25Z

commit a9a8ba90fa5857c2c8a0e32eef2159cec717da11 upstream. Rather than chunking via PAGE_SIZE, this commit changes the arch implementations to chunk in explicit 4k parts, so that calculations on maximum acceptable latency don't suddenly become invalid on platforms where PAGE_SIZE isn't 4k, such as arm64. Fixes: 0f961f9f670e ("crypto: x86/nhpoly1305 - add AVX2 accelerated NHPoly1305") Fixes: 012c82388c03 ("crypto: x86/nhpoly1305 - add SSE2 accelerated NHPoly1305") Fixes: a00fa0c88774 ("crypto: arm64/nhpoly1305 - add NEON-accelerated NHPoly1305") Fixes: 16aae3595a9d ("crypto: arm/nhpoly1305 - add NEON-accelerated NHPoly1305") Cc: stable@vger.kernel.org Signed-off-by: Jason A. Donenfeld Reviewed-by: Eric Biggers Signed-off-by: Herbert Xu Signed-off-by: Greg Kroah-Hartman

crypto: arm64/ghash-neon - bump priority to 150

2020-02-11T12:35:31Z

commit 5441c6507bc84166e9227e9370a56c57ba13794a upstream. The SIMD based GHASH implementation for arm64 is typically much faster than the generic one, and doesn't use any lookup tables, so it is clearly preferred when available. So bump the priority to reflect that. Fixes: 5a22b198cd527447 ("crypto: arm64/ghash - register PMULL variants ...") Signed-off-by: Ard Biesheuvel Signed-off-by: Herbert Xu Signed-off-by: Greg Kroah-Hartman

crypto: arm64/aes-neonbs - add return value of skcipher_walk_done() in __xts_crypt()

2020-01-17T18:48:46Z

commit 9b537997b669c42cec67893538037e8d1c83c91c upstream. A warning is found by the static code analysis tool: "Identical condition 'err', second condition is always false" Fix this by adding return value of skcipher_walk_done(). Fixes: 67cfa5d3b721 ("crypto: arm64/aes-neonbs - implement ciphertext stealing for XTS") Signed-off-by: Yunfeng Ye Acked-by: Ard Biesheuvel Signed-off-by: Herbert Xu Signed-off-by: Greg Kroah-Hartman

crypto: arm64/aes-neonbs - implement ciphertext stealing for XTS

2019-09-09T07:35:39Z

Update the AES-XTS implementation based on NEON instructions so that it can deal with inputs whose size is not a multiple of the cipher block size. This is part of the original XTS specification, but was never implemented before in the Linux kernel. Since the bit slicing driver is only faster if it can operate on at least 7 blocks of input at the same time, let's reuse the alternate path we are adding for CTS to process any data tail whose size is not a multiple of 128 bytes. Signed-off-by: Ard Biesheuvel Signed-off-by: Herbert Xu

crypto: arm64/aes - implement support for XTS ciphertext stealing

2019-09-09T07:35:39Z

Add the missing support for ciphertext stealing in the implementation of AES-XTS, which is part of the XTS specification but was omitted up until now due to lack of a need for it. The asm helpers are updated so they can deal with any input size, as long as the last full block and the final partial block are presented at the same time. The glue code is updated so that the common case of operating on a sector or page is mostly as before. When CTS is needed, the walk is split up into two pieces, unless the entire input is covered by a single step. Signed-off-by: Ard Biesheuvel Signed-off-by: Herbert Xu

crypto: arm64/aes-cts-cbc - move request context data to the stack

2019-09-09T07:35:39Z

Since the CTS-CBC code completes synchronously, there is no point in keeping part of the scratch data it uses in the request context, so move it to the stack instead. Signed-off-by: Ard Biesheuvel Signed-off-by: Herbert Xu

crypto: arm64/aes-cts-cbc-ce - performance tweak

2019-09-09T07:35:38Z

Optimize away one of the tbl instructions in the decryption path, which turns out to be unnecessary. Signed-off-by: Ard Biesheuvel Signed-off-by: Herbert Xu