kernel.git/fs/overlayfs/dir.c, branch linux-4.16.y

ovl: whiteout index when union nlink drops to zero

2018-01-24T10:25:56Z

With NFS export feature enabled, when overlay inode nlink drops to zero, instead of removing the index entry, replace it with a whiteout index entry. This is needed for NFS export in order to prevent future open by handle from opening the lower file directly. Signed-off-by: Amir Goldstein Signed-off-by: Miklos Szeredi

ovl: take lower dir inode mutex outside upper sb_writers lock

2018-01-19T16:43:23Z

The functions ovl_lower_positive() and ovl_check_empty_dir() both take inode mutex on the real lower dir under ovl_want_write() which takes the upper_mnt sb_writers lock. While this is not a clear locking order or layering violation, it creates an undesired lock dependency between two unrelated layers for no good reason. This lock dependency materializes to a false(?) positive lockdep warning when calling rmdir() on a nested overlayfs, where both nested and underlying overlayfs both use the same fs type as upper layer. rmdir() on the nested overlayfs creates the lock chain: sb_writers of upper_mnt (e.g. tmpfs) in ovl_do_remove() ovl_i_mutex_dir_key[] of lower overlay dir in ovl_lower_positive() rmdir() on the underlying overlayfs creates the lock chain in reverse order: ovl_i_mutex_dir_key[] of lower overlay dir in vfs_rmdir() sb_writers of nested upper_mnt (e.g. tmpfs) in ovl_do_remove() To rid of the unneeded locking dependency, move both ovl_lower_positive() and ovl_check_empty_dir() to before ovl_want_write() in rmdir() and rename() implementation. This change spreads the pieces of ovl_check_empty_and_clear() directly inside the rmdir()/rename() implementations so the helper is no longer needed and removed. Signed-off-by: Amir Goldstein Signed-off-by: Miklos Szeredi

ovl: fix overlay: warning prefix

2017-12-14T10:14:52Z

Conform two stray warning messages to the standard overlayfs: prefix. Signed-off-by: Amir Goldstein Signed-off-by: Miklos Szeredi

ovl: update cache version of impure parent on rename

2017-11-09T09:23:27Z

ovl_rename() updates dir cache version for impure old parent if an entry with copy up origin is moved into old parent, but it did not update cache version if the entry moved out of old parent has a copy up origin. [SzM] Same for new dir: we updated the version if an entry with origin was moved in, but not if an entry with origin was moved out. Signed-off-by: Amir Goldstein Signed-off-by: Miklos Szeredi

ovl: fix rmdir problem on non-merge dir with origin xattr

2017-11-09T09:23:27Z

An "origin && non-merge" upper dir may have leftover whiteouts that were created in past mount. overlayfs does no clear this dir when we delete it, which may lead to rmdir fail or temp file left in workdir. Simple reproducer: mkdir lower upper work merge mkdir -p lower/dir touch lower/dir/a mount -t overlay overlay -olowerdir=lower,upperdir=upper,\ workdir=work merge rm merge/dir/a umount merge rm -rf lower/* touch lower/dir (*) mount -t overlay overlay -olowerdir=lower,upperdir=upper,\ workdir=work merge rm -rf merge/dir Syslog dump: overlayfs: cleanup of 'work/#7' failed (-39) (*): if we do not create the regular file, the result is different: rm: cannot remove "dir/": Directory not empty This patch adds a check for the case of non-merge dir that may contain whiteouts, and calls ovl_check_empty_dir() to check and clear whiteouts from upper dir when an empty dir is being deleted. [amir: split patch from ovl_check_empty_dir() cleanup rename ovl_is_origin() to ovl_may_have_whiteouts() check OVL_WHITEOUTS flag instead of checking origin xattr] Signed-off-by: zhangyi (F) Signed-off-by: Amir Goldstein Signed-off-by: Miklos Szeredi

ovl: simplify ovl_check_empty_and_clear()

2017-11-09T09:23:27Z

Filter out non-whiteout non-upper entries from list of merge dir entries while checking if merge dir is empty in ovl_check_empty_dir(). The remaining work for ovl_clear_empty() is to clear all entries on the list. [amir: split patch from rmdir bug fix] Signed-off-by: zhangyi (F) Signed-off-by: Amir Goldstein Signed-off-by: Miklos Szeredi

ovl: fix missing unlock_rename() in ovl_do_copy_up()

2017-10-05T13:53:18Z

Use the ovl_lock_rename_workdir() helper which requires unlock_rename() only on lock success. Fixes: ("fd210b7d67ee ovl: move copy up lock out") Cc: # v4.13 Signed-off-by: Amir Goldstein Signed-off-by: Miklos Szeredi

mm: treewide: remove GFP_TEMPORARY allocation flag

2017-09-14T01:53:16Z

GFP_TEMPORARY was introduced by commit e12ba74d8ff3 ("Group short-lived and reclaimable kernel allocations") along with __GFP_RECLAIMABLE. It's primary motivation was to allow users to tell that an allocation is short lived and so the allocator can try to place such allocations close together and prevent long term fragmentation. As much as this sounds like a reasonable semantic it becomes much less clear when to use the highlevel GFP_TEMPORARY allocation flag. How long is temporary? Can the context holding that memory sleep? Can it take locks? It seems there is no good answer for those questions. The current implementation of GFP_TEMPORARY is basically GFP_KERNEL | __GFP_RECLAIMABLE which in itself is tricky because basically none of the existing caller provide a way to reclaim the allocated memory. So this is rather misleading and hard to evaluate for any benefits. I have checked some random users and none of them has added the flag with a specific justification. I suspect most of them just copied from other existing users and others just thought it might be a good idea to use without any measuring. This suggests that GFP_TEMPORARY just motivates for cargo cult usage without any reasoning. I believe that our gfp flags are quite complex already and especially those with highlevel semantic should be clearly defined to prevent from confusion and abuse. Therefore I propose dropping GFP_TEMPORARY and replace all existing users to simply use GFP_KERNEL. Please note that SLAB users with shrinkers will still get __GFP_RECLAIMABLE heuristic and so they will be placed properly for memory fragmentation prevention. I can see reasons we might want some gfp flag to reflect shorterm allocations but I propose starting from a clear semantic definition and only then add users with proper justification. This was been brought up before LSF this year by Matthew [1] and it turned out that GFP_TEMPORARY really doesn't have a clear semantic. It seems to be a heuristic without any measured advantage for most (if not all) its current users. The follow up discussion has revealed that opinions on what might be temporary allocation differ a lot between developers. So rather than trying to tweak existing users into a semantic which they haven't expected I propose to simply remove the flag and start from scratch if we really need a semantic for short term allocations. [1] http://lkml.kernel.org/r/20170118054945.GD18349@bombadil.infradead.org [akpm@linux-foundation.org: fix typo] [akpm@linux-foundation.org: coding-style fixes] [sfr@canb.auug.org.au: drm/i915: fix up] Link: http://lkml.kernel.org/r/20170816144703.378d4f4d@canb.auug.org.au Link: http://lkml.kernel.org/r/20170728091904.14627-1-mhocko@kernel.org Signed-off-by: Michal Hocko Signed-off-by: Stephen Rothwell Acked-by: Mel Gorman Acked-by: Vlastimil Babka Cc: Matthew Wilcox Cc: Neil Brown Cc: "Theodore Ts'o" Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

ovl: constant d_ino for non-merge dirs

2017-07-27T19:54:06Z

Impure directories are ones which contain objects with origins (i.e. those that have been copied up). These are relevant to readdir operation only because of the d_ino field, no other transformation is necessary. Also a directory can become impure between two getdents(2) calls. This patch creates a cache for impure directories. Unlike the cache for merged directories, this one only contains entries with origin and is not refcounted but has a its lifetime tied to that of the dentry. Similarly to the merged cache, the impure cache is invalidated based on a version number. This version number is incremented when an entry with origin is added or removed from the directory. If the cache is empty, then the impure xattr is removed from the directory. This patch also fixes up handling of d_ino for the ".." entry if the parent directory is merged. Signed-off-by: Miklos Szeredi

ovl: mark parent impure on ovl_link()

2017-07-13T20:06:45Z

When linking a file with copy up origin into a new parent, mark the new parent dir "impure". Fixes: ee1d6d37b6b8 ("ovl: mark upper dir with type origin entries "impure"") Cc: # v4.12 Signed-off-by: Amir Goldstein Signed-off-by: Miklos Szeredi