diff options
| author | Hao Li | 2025-12-10 08:26:31 +0800 |
|---|---|---|
| committer | Vlastimil Babka | 2026-01-27 18:03:19 +0100 |
| commit | 4b038a9670154e8bb4832d80f0f2b68b1b812171 (patch) | |
| tree | 45fc25d8e8b5249de9323d0005d39a54dd971b82 /mm/slub.c | |
| parent | d907bf434fcd64c9609aa2983574e7c1f28e5493 (diff) | |
slub: keep empty main sheaf as spare in __pcs_replace_empty_main()
When __pcs_replace_empty_main() fails to obtain a full sheaf directly
from the barn, it may either:
- Refill an empty sheaf obtained via barn_get_empty_sheaf(), or
- Allocate a brand new full sheaf via alloc_full_sheaf().
After reacquiring the per-CPU lock, if pcs->main is still empty and
pcs->spare is NULL, the current code donates the empty main sheaf to
the barn via barn_put_empty_sheaf() and installs the full sheaf as
pcs->main, leaving pcs->spare unpopulated.
Instead, keep the existing empty main sheaf locally as the spare:
pcs->spare = pcs->main;
pcs->main = full;
This populates pcs->spare earlier, which can reduce future barn traffic.
Suggested-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Hao Li <haolee.swjtu@gmail.com>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Tested-by: Zhao Liu <zhao1.liu@intel.com>
Diffstat (limited to 'mm/slub.c')
| -rw-r--r-- | mm/slub.c | 5 |
1 files changed, 4 insertions, 1 deletions
diff --git a/mm/slub.c b/mm/slub.c index afc3e511ff39..e90f3e558ae7 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -5049,7 +5049,10 @@ __pcs_replace_empty_main(struct kmem_cache *s, struct slub_percpu_sheaves *pcs, */ if (pcs->main->size == 0) { - barn_put_empty_sheaf(barn, pcs->main); + if (!pcs->spare) + pcs->spare = pcs->main; + else + barn_put_empty_sheaf(barn, pcs->main); pcs->main = full; return pcs; } |