kernel.git/fs/verity/verify.c, branch linux-6.15.y

fsverity: remove hash page spin lock

2024-02-01T23:19:23Z

The spin lock is not necessary here as it can be replaced with memory barrier which should be better performance-wise. When Merkle tree block size differs from page size, in is_hash_block_verified() two things are modified during check - a bitmap and PG_checked flag of the page. Each bit in the bitmap represent verification status of the Merkle tree blocks. PG_checked flag tells if page was just re-instantiated or was in pagecache. Both of this states are shared between verification threads. Page which was re-instantiated can not have already verified blocks (bit set in bitmap). The spin lock was used to allow only one thread to modify both of these states and keep order of operations. The only requirement here is that PG_Checked is set strictly after bitmap is updated. This way other threads which see that PG_Checked=1 (page cached) knows that bitmap is up-to-date. Otherwise, if PG_Checked is set before bitmap is cleared, other threads can see bit=1 and therefore will not perform verification of that Merkle tree block. However, there's still the case when one thread is setting a bit in verify_data_block() and other thread is clearing it in is_hash_block_verified(). This can happen if two threads get to !PageChecked branch and one of the threads is rescheduled before resetting the bitmap. This is fine as at worst blocks are re-verified in each thread. Signed-off-by: Andrey Albershteyn [ebiggers: improved the comment and removed the 'verified' variable] Link: https://lore.kernel.org/r/20240201052813.68380-1-ebiggers@kernel.org Signed-off-by: Eric Biggers

fsverity: simplify handling of errors during initcall

2023-07-12T05:49:18Z

Since CONFIG_FS_VERITY is a bool, not a tristate, fs/verity/ can only be builtin or absent entirely; it can't be a loadable module. Therefore, the error code that gets returned from the fsverity_init() initcall is never used. If any part of the initcall does fail, which should never happen, the kernel will be left in a bad state. Following the usual convention for builtin code, just panic the kernel if any of part of the initcall fails. Link: https://lore.kernel.org/r/20230705212743.42180-2-ebiggers@kernel.org Signed-off-by: Eric Biggers

fsverity: simplify error handling in verify_data_block()

2023-06-04T12:56:11Z

Clean up the error handling in verify_data_block() to (a) eliminate the 'err' variable which has caused some confusion because the function actually returns a bool, (b) reduce the compiled code size slightly, and (c) execute one fewer branch in the success case. Link: https://lore.kernel.org/r/20230604022312.48532-1-ebiggers@kernel.org Signed-off-by: Eric Biggers

fsverity: don't use bio_first_page_all() in fsverity_verify_bio()

2023-06-04T12:56:11Z

bio_first_page_all(bio)->mapping->host is not compatible with large folios, since the first page of the bio is not necessarily the head page of the folio, and therefore it might not have the mapping pointer set. Therefore, move the dereference of ->mapping->host into verify_data_blocks(), which works with a folio. (Like the commit that this Fixes, this hasn't actually been tested with large folios yet, since the filesystems that use fs/verity/ don't support that yet. But based on code review, I think this is needed.) Fixes: 5d0f0e57ed90 ("fsverity: support verifying data from large folios") Link: https://lore.kernel.org/r/20230604022101.48342-1-ebiggers@kernel.org Reviewed-by: Matthew Wilcox (Oracle) Signed-off-by: Eric Biggers

fsverity: use shash API instead of ahash API

2023-06-04T12:54:28Z

The "ahash" API, like the other scatterlist-based crypto APIs such as "skcipher", comes with some well-known limitations. First, it can't easily be used with vmalloc addresses. Second, the request struct can't be allocated on the stack. This adds complexity and a possible failure point that needs to be worked around, e.g. using a mempool. The only benefit of ahash over "shash" is that ahash is needed to access traditional memory-to-memory crypto accelerators, i.e. drivers/crypto/. However, this style of crypto acceleration has largely fallen out of favor and been superseded by CPU-based acceleration or inline crypto engines. Also, ahash needs to be used asynchronously to take full advantage of such hardware, but fs/verity/ has never done this. On all systems that aren't actually using one of these ahash-only crypto accelerators, ahash just adds unnecessary overhead as it sits between the user and the underlying shash algorithms. Also, XFS is planned to cache fsverity Merkle tree blocks in the existing XFS buffer cache. As a result, it will be possible for a single Merkle tree block to be split across discontiguous pages (https://lore.kernel.org/r/20230405233753.GU3223426@dread.disaster.area). This data will need to be hashed. It is easiest to work with a vmapped address in this case. However, ahash is incompatible with this. Therefore, let's convert fs/verity/ from ahash to shash. This simplifies the code, and it should also slightly improve performance for everyone who wasn't actually using one of these ahash-only crypto accelerators, i.e. almost everyone (or maybe even everyone)! Link: https://lore.kernel.org/r/20230516052306.99600-1-ebiggers@kernel.org Reviewed-by: Christoph Hellwig Acked-by: Ard Biesheuvel Signed-off-by: Eric Biggers

fsverity: Remove WQ_UNBOUND from fsverity read workqueue

2023-03-14T23:16:36Z

WQ_UNBOUND causes significant scheduler latency on ARM64/Android. This is problematic for latency sensitive workloads, like I/O post-processing. Removing WQ_UNBOUND gives a 96% reduction in fsverity workqueue related scheduler latency and improves app cold startup times by ~30ms. WQ_UNBOUND was also removed from the dm-verity workqueue for the same reason [1]. This code was tested by running Android app startup benchmarks and measuring how long the fsverity workqueue spent in the runnable state. Before Total workqueue scheduler latency: 553800us After Total workqueue scheduler latency: 18962us [1]: https://lore.kernel.org/all/20230202012348.885402-1-nhuck@google.com/ Signed-off-by: Nathan Huckleberry Fixes: 8a1d0f9cacc9 ("fs-verity: add data verification hooks for ->readpages()") Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20230310193325.620493-1-nhuck@google.com Signed-off-by: Eric Biggers

fsverity: support verifying data from large folios

2023-01-27T22:46:31Z

Try to make fs/verity/verify.c aware of large folios. This includes making fsverity_verify_bio() support the case where the bio contains large folios, and adding a function fsverity_verify_folio() which is the equivalent of fsverity_verify_page(). There's no way to actually test this with large folios yet, but I've tested that this doesn't cause any regressions. Signed-off-by: Eric Biggers Link: https://lore.kernel.org/r/20230127221529.299560-1-ebiggers@kernel.org

fsverity: support verification with tree block size < PAGE_SIZE

2023-01-10T03:06:03Z

Add support for verifying data from verity files whose Merkle tree block size is less than the page size. The main use case for this is to allow a single Merkle tree block size to be used across all systems, so that only one set of fsverity file digests and signatures is needed. To do this, eliminate various assumptions that the Merkle tree block size and the page size are the same: - Make fsverity_verify_page() a wrapper around a new function fsverity_verify_blocks() which verifies one or more blocks in a page. - When a Merkle tree block is needed, get the corresponding page and only verify and use the needed portion. (The Merkle tree continues to be read and cached in page-sized chunks; that doesn't need to change.) - When the Merkle tree block size and page size differ, use a bitmap fsverity_info::hash_block_verified to keep track of which Merkle tree blocks have been verified, as PageChecked cannot be used directly. Signed-off-by: Eric Biggers Reviewed-by: Andrey Albershteyn Tested-by: Ojaswin Mujoo Link: https://lore.kernel.org/r/20221223203638.41293-7-ebiggers@kernel.org

fsverity: replace fsverity_hash_page() with fsverity_hash_block()

2023-01-10T03:06:01Z

In preparation for allowing the Merkle tree block size to differ from PAGE_SIZE, replace fsverity_hash_page() with fsverity_hash_block(). The new function is similar to the old one, but it operates on the block at the given offset in the page instead of on the full page. (For now, all callers still pass a full page.) Signed-off-by: Eric Biggers Reviewed-by: Andrey Albershteyn Tested-by: Ojaswin Mujoo Link: https://lore.kernel.org/r/20221223203638.41293-6-ebiggers@kernel.org

fsverity: store log2(digest_size) precomputed

2023-01-10T03:05:56Z

Add log_digestsize to struct merkle_tree_params so that it can be used in verify.c. Also save memory by using u8 for all the log_* fields. Signed-off-by: Eric Biggers Reviewed-by: Andrey Albershteyn Tested-by: Ojaswin Mujoo Link: https://lore.kernel.org/r/20221223203638.41293-4-ebiggers@kernel.org