kernel.git/drivers/md/dm-crypt.c, branch linux-6.16.y

dm: dm-crypt: Do not partially accept write BIOs with zoned targets

2025-08-28T14:34:12Z

commit e549663849e5bb3b985dc2d293069f0d9747ae72 upstream. Read and write operations issued to a dm-crypt target may be split according to the dm-crypt internal limits defined by the max_read_size and max_write_size module parameters (default is 128 KB). The intent is to improve processing time of large BIOs by splitting them into smaller operations that can be parallelized on different CPUs. For zoned dm-crypt targets, this BIO splitting is still done but without the parallel execution to ensure that the issuing order of write operations to the underlying devices remains sequential. However, the splitting itself causes other problems: 1) Since dm-crypt relies on the block layer zone write plugging to handle zone append emulation using regular write operations, the reminder of a split write BIO will always be plugged into the target zone write plugged. Once the on-going write BIO finishes, this reminder BIO is unplugged and issued from the zone write plug work. If this reminder BIO itself needs to be split, the reminder will be re-issued and plugged again, but that causes a call to a blk_queue_enter(), which may block if a queue freeze operation was initiated. This results in a deadlock as DM submission still holds BIOs that the queue freeze side is waiting for. 2) dm-crypt relies on the emulation done by the block layer using regular write operations for processing zone append operations. This still requires to properly return the written sector as the BIO sector of the original BIO. However, this can be done correctly only and only if there is a single clone BIO used for processing the original zone append operation issued by the user. If the size of a zone append operation is larger than dm-crypt max_write_size, then the orginal BIO will be split and processed as a chain of regular write operations. Such chaining result in an incorrect written sector being returned to the zone append issuer using the original BIO sector. This in turn results in file system data corruptions using xfs or btrfs. Fix this by modifying get_max_request_size() to always return the size of the BIO to avoid it being split with dm_accpet_partial_bio() in crypt_map(). get_max_request_size() is renamed to get_max_request_sectors() to clarify the unit of the value returned and its interface is changed to take a struct dm_target pointer and a pointer to the struct bio being processed. In addition to this change, to ensure that crypt_alloc_buffer() works correctly, set the dm-crypt device max_hw_sectors limit to be at most BIO_MAX_VECS << PAGE_SECTORS_SHIFT (1 MB with a 4KB page architecture). This forces DM core to split write BIOs before passing them to crypt_map(), and thus guaranteeing that dm-crypt can always accept an entire write BIO without needing to split it. This change does not have any effect on the read path of dm-crypt. Read operations can still be split and the BIO fragments processed in parallel. There is also no impact on the performance of the write path given that all zone write BIOs were already processed inline instead of in parallel. This change also does not affect in any way regular dm-crypt block devices. Fixes: f211268ed1f9 ("dm: Use the block layer zone append emulation") Cc: stable@vger.kernel.org Signed-off-by: Damien Le Moal Reviewed-by: Mikulas Patocka Link: https://lore.kernel.org/r/20250625093327.548866-5-dlemoal@kernel.org Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman

dm-crypt: Extend state buffer size in crypt_iv_lmk_one

2025-06-23T11:50:02Z

Add a macro CRYPTO_MD5_STATESIZE for the Crypto API export state size of md5 and use that in dm-crypt instead of relying on the size of struct md5_state (the latter is currently undergoing a transition and may shrink). This commit fixes a crash on 32-bit machines: Oops: Oops: 0000 [#1] SMP CPU: 1 UID: 0 PID: 12 Comm: kworker/u16:0 Not tainted 6.16.0-rc2+ #993 PREEMPT(full) Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020 Workqueue: kcryptd-254:0-1 kcryptd_crypt [dm_crypt] EIP: __crypto_shash_export+0xf/0x90 Code: 4a c1 c7 40 20 a0 b4 4a c1 81 cf 0e 00 04 08 89 78 50 e9 2b ff ff ff 8d 74 26 00 55 89 e5 57 56 53 89 c3 89 d6 8b 00 8b 40 14 <8b> 50 fc f6 40 13 01 74 04 4a 2b 50 14 85 c9 74 10 89 f2 89 d8 ff EAX: 303a3435 EBX: c3007c90 ECX: 00000000 EDX: c3007c38 ESI: c3007c38 EDI: c3007c90 EBP: c3007bfc ESP: c3007bf0 DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00010216 CR0: 80050033 CR2: 303a3431 CR3: 04fbe000 CR4: 00350e90 Call Trace: crypto_shash_export+0x65/0xc0 crypt_iv_lmk_one+0x106/0x1a0 [dm_crypt] Fixes: efd62c85525e ("crypto: md5-generic - Use API partial block handling") Reported-by: Milan Broz Signed-off-by: Herbert Xu Tested-by: Milan Broz Closes: https://lore.kernel.org/linux-crypto/f1625ddc-e82e-4b77-80c2-dc8e45b54848@gmail.com/T/ Signed-off-by: Mikulas Patocka

dm-crypt: switch to using the crc32 library

2025-02-03T13:10:10Z

Now that the crc32() library function takes advantage of architecture-specific optimizations, it is unnecessary to go through the crypto API. Just use crc32(). This is much simpler, and it improves performance due to eliminating the crypto API overhead. (However, this only affects the TCW IV mode of dm-crypt, which is a compatibility mode that is rarely used compared to other dm-crypt modes.) Signed-off-by: Eric Biggers Signed-off-by: Mikulas Patocka

dm-crypt: track tag_offset in convert_context

2025-01-21T21:02:12Z

dm-crypt uses tag_offset to index the integrity metadata for each crypt sector. When the initial crypt_convert() returns BLK_STS_DEV_RESOURCE, dm-crypt will try to continue the crypt/decrypt procedure in a kworker. However, it resets tag_offset as zero instead of using the tag_offset related with current sector. It may return unexpected data when using random IV or return unexpected integrity related error. Fix the problem by tracking tag_offset in per-IO convert_context. Therefore, when the crypt/decrypt procedure continues in a kworker, it could use the next tag_offset saved in convert_context. Fixes: 8abec36d1274 ("dm crypt: do not wait for backlogged crypto request completion in softirq") Cc: stable@vger.kernel.org Signed-off-by: Hou Tao Signed-off-by: Mikulas Patocka

dm-crypt: don't initialize cc_sector again

2025-01-21T21:02:12Z

For aead_recheck case, cc_sector has already been initialized in crypt_convert_init() when trying to re-read the read. Therefore, remove the duplicated initialization. Signed-off-by: Hou Tao Signed-off-by: Mikulas Patocka

dm-crypt: don't update io->sector after kcryptd_crypt_write_io_submit()

2025-01-21T21:02:12Z

The updates of io->sector are the leftovers when dm-crypt allocated pages for partial write request. However, since commit cf2f1abfbd0db ("dm crypt: don't allocate pages for a partial request"), there is no partial request anymore. After the introduction of write request rb-tree, the updates of io->sectors may interfere the insertion procedure, because ->sectors of these write requests which have already been added in the rb-tree may be changed during the insertion of new write request. Fix it by removing these buggy updates of io->sectors. Considering these updates only effect the write request rb-tree, the commit which introduces the write request rb-tree is used as the fix tag. Fixes: b3c5fd305249 ("dm crypt: sort writes") Cc: stable@vger.kernel.org Signed-off-by: Hou Tao Signed-off-by: Mikulas Patocka

dm-crypt: use bi_sector in bio when initialize integrity seed

2025-01-21T21:00:57Z

bio->bi_iter.bi_sector has already been initialized when initialize the integrity seed in dm_crypt_integrity_io_alloc(). There is no need to calculate it again. Therefore, use the helper bip_set_seed() to initialize the seed and pass bi_iter.bi_sector to it instead. Mikulas: We can't use bip_set_seed because it doesn't compile without CONFIG_BLK_DEV_INTEGRITY. Signed-off-by: Hou Tao Signed-off-by: Mikulas Patocka

dm-crypt: fully initialize clone->bi_iter in crypt_alloc_buffer()

2025-01-21T12:25:09Z

Both kcryptd_io_read() and kcryptd_crypt_write_convert() will invoke crypt_alloc_buffer() to allocate a new bio. Both of these two callers initialize bi_iter.bi_sector for the new bio separatedly after crypt_alloc_buffer() returns. However, kcryptd_crypt_write_convert() will copy the bi_iter of the new bio into ctx.iter_out or ctx.iter_in. Although it doesn't incur any harm now, it is better to fully initialize bi_iter before it is used. Therefore, initialize bi_iter.bi_sector in crypt_alloc_buffer() instead. Signed-off-by: Hou Tao Signed-off-by: Mikulas Patocka

dm-crypt: set atomic as false when calling crypt_convert() in kworker

2025-01-21T12:24:35Z

Both kcryptd_crypt_write_continue() and kcryptd_crypt_read_continue() are running in the kworker context, it is OK to call cond_resched(), Therefore, set atomic as false when invoking crypt_convert() under kworker context. Signed-off-by: Hou Tao Reviewed-by: Ignat Korchagin Signed-off-by: Mikulas Patocka

move asm/unaligned.h to linux/unaligned.h

2024-10-02T21:23:23Z

asm/unaligned.h is always an include of asm-generic/unaligned.h; might as well move that thing to linux/unaligned.h and include that - there's nothing arch-specific in that header. auto-generated by the following: for i in `git grep -l -w asm/unaligned.h`; do sed -i -e "s/asm\/unaligned.h/linux\/unaligned.h/" $i done for i in `git grep -l -w asm-generic/unaligned.h`; do sed -i -e "s/asm-generic\/unaligned.h/linux\/unaligned.h/" $i done git mv include/asm-generic/unaligned.h include/linux/unaligned.h git mv tools/include/asm-generic/unaligned.h tools/include/linux/unaligned.h sed -i -e "/unaligned.h/d" include/asm-generic/Kbuild sed -i -e "s/__ASM_GENERIC/__LINUX/" include/linux/unaligned.h tools/include/linux/unaligned.h