Fixing livelocks in shrink_dcache_tree()
If shrink_dcache_tree() finds a dentry in the middle of being killed
by another thread, it has to wait until the victim finishes dying,
gets detached from the tree and ceases to pin its parent.
The way we used to deal with that amounted to busy-wait; unfortunately,
it's not just inefficient but can lead to reliably reproducible hard
livelocks.
Solved by having shrink_dentry_tree() attach a completion to such dentry,
with dentry_unlist() calling complete() on all objects attached to it.
With a bit of care it can be done without growing struct dentry or adding
overhead in normal case.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
get rid of busy-waiting in shrink_dcache_tree()
If shrink_dcache_tree() runs into a potential victim that is already
dying, it must wait for that dentry to go away. To avoid busy-waiting
we need some object to wait on and a way for dentry_unlist() to see that
we need to be notified.
The obvious place for the object to wait on would be on our stack frame.
We will store a pointer to that object (struct completion_list) in victim
dentry; if there's more than one thread wanting to wait for the same
dentry to finish dying, we'll have their instances linked into a list,
with reference in dentry pointing to the head of that list.
* new object - struct completion_list. A pair of struct completion and
pointer to the next instance. That's what shrink_dcache_tree() will wait
on if needed.
* add a new member (->waiters, opaque pointer to struct completion_list)
to struct dentry. It is defined for negative live dentries that are
not in-lookup ones and it will remain NULL for almost all of them.
It does not conflict with ->d_rcu (defined for killed dentries), ->d_alias
(defined for positive dentries, all live) or ->d_in_lookup_hash (defined
for in-lookup dentries, all live negative). That allows to colocate
all four members.
* make sure that all places where dentry enters the state where ->waiters
is defined (live, negative, not-in-lookup) initialize ->waiters to NULL.
* if select_collect2() runs into a dentry that is already dying, have
its caller insert a local instance of struct completion_list into the
head of the list hanging off dentry->waiters and wait for completion.
* if dentry_unlist() sees non-NULL ->waiters, have it carefully walk
through the completion_list instances in that list, calling complete()
for each.
For now struct completion_list is local to fs/dcache.c; it's obviously
dentry-agnostic, and it can be trivially lifted into linux/completion.h
if somebody finds a reason to do so...
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2 files changed