| Definitions |
| ~~~~~~~~~~~ |
| |
| Userspace filesystem: |
| |
| A filesystem in which data and metadata are provided by an ordinary |
| userspace process. The filesystem can be accessed normally through |
| the kernel interface. |
| |
| Filesystem daemon: |
| |
| The process(es) providing the data and metadata of the filesystem. |
| |
| Non-privileged mount (or user mount): |
| |
| A userspace filesystem mounted by a non-privileged (non-root) user. |
| The filesystem daemon is running with the privileges of the mounting |
| user. NOTE: this is not the same as mounts allowed with the "user" |
| option in /etc/fstab, which is not discussed here. |
| |
| Mount owner: |
| |
| The user who does the mounting. |
| |
| User: |
| |
| The user who is performing filesystem operations. |
| |
| What is FUSE? |
| ~~~~~~~~~~~~~ |
| |
| FUSE is a userspace filesystem framework. It consists of a kernel |
| module (fuse.ko), a userspace library (libfuse.*) and a mount utility |
| (fusermount). |
| |
| One of the most important features of FUSE is allowing secure, |
| non-privileged mounts. This opens up new possibilities for the use of |
| filesystems. A good example is sshfs: a secure network filesystem |
| using the sftp protocol. |
| |
| The userspace library and utilities are available from the FUSE |
| homepage: |
| |
| http://fuse.sourceforge.net/ |
| |
| Mount options |
| ~~~~~~~~~~~~~ |
| |
| 'fd=N' |
| |
| The file descriptor to use for communication between the userspace |
| filesystem and the kernel. The file descriptor must have been |
| obtained by opening the FUSE device ('/dev/fuse'). |
| |
| 'rootmode=M' |
| |
| The file mode of the filesystem's root in octal representation. |
| |
| 'user_id=N' |
| |
| The numeric user id of the mount owner. |
| |
| 'group_id=N' |
| |
| The numeric group id of the mount owner. |
| |
| 'default_permissions' |
| |
| By default FUSE doesn't check file access permissions, the |
| filesystem is free to implement it's access policy or leave it to |
| the underlying file access mechanism (e.g. in case of network |
| filesystems). This option enables permission checking, restricting |
| access based on file mode. This is option is usually useful |
| together with the 'allow_other' mount option. |
| |
| 'allow_other' |
| |
| This option overrides the security measure restricting file access |
| to the user mounting the filesystem. This option is by default only |
| allowed to root, but this restriction can be removed with a |
| (userspace) configuration option. |
| |
| 'max_read=N' |
| |
| With this option the maximum size of read operations can be set. |
| The default is infinite. Note that the size of read requests is |
| limited anyway to 32 pages (which is 128kbyte on i386). |
| |
| Sysfs |
| ~~~~~ |
| |
| FUSE sets up the following hierarchy in sysfs: |
| |
| /sys/fs/fuse/connections/N/ |
| |
| where N is an increasing number allocated to each new connection. |
| |
| For each connection the following attributes are defined: |
| |
| 'waiting' |
| |
| The number of requests which are waiting to be transfered to |
| userspace or being processed by the filesystem daemon. If there is |
| no filesystem activity and 'waiting' is non-zero, then the |
| filesystem is hung or deadlocked. |
| |
| 'abort' |
| |
| Writing anything into this file will abort the filesystem |
| connection. This means that all waiting requests will be aborted an |
| error returned for all aborted and new requests. |
| |
| Only a privileged user may read or write these attributes. |
| |
| Aborting a filesystem connection |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| It is possible to get into certain situations where the filesystem is |
| not responding. Reasons for this may be: |
| |
| a) Broken userspace filesystem implementation |
| |
| b) Network connection down |
| |
| c) Accidental deadlock |
| |
| d) Malicious deadlock |
| |
| (For more on c) and d) see later sections) |
| |
| In either of these cases it may be useful to abort the connection to |
| the filesystem. There are several ways to do this: |
| |
| - Kill the filesystem daemon. Works in case of a) and b) |
| |
| - Kill the filesystem daemon and all users of the filesystem. Works |
| in all cases except some malicious deadlocks |
| |
| - Use forced umount (umount -f). Works in all cases but only if |
| filesystem is still attached (it hasn't been lazy unmounted) |
| |
| - Abort filesystem through the sysfs interface. Most powerful |
| method, always works. |
| |
| How do non-privileged mounts work? |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| Since the mount() system call is a privileged operation, a helper |
| program (fusermount) is needed, which is installed setuid root. |
| |
| The implication of providing non-privileged mounts is that the mount |
| owner must not be able to use this capability to compromise the |
| system. Obvious requirements arising from this are: |
| |
| A) mount owner should not be able to get elevated privileges with the |
| help of the mounted filesystem |
| |
| B) mount owner should not get illegitimate access to information from |
| other users' and the super user's processes |
| |
| C) mount owner should not be able to induce undesired behavior in |
| other users' or the super user's processes |
| |
| How are requirements fulfilled? |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| A) The mount owner could gain elevated privileges by either: |
| |
| 1) creating a filesystem containing a device file, then opening |
| this device |
| |
| 2) creating a filesystem containing a suid or sgid application, |
| then executing this application |
| |
| The solution is not to allow opening device files and ignore |
| setuid and setgid bits when executing programs. To ensure this |
| fusermount always adds "nosuid" and "nodev" to the mount options |
| for non-privileged mounts. |
| |
| B) If another user is accessing files or directories in the |
| filesystem, the filesystem daemon serving requests can record the |
| exact sequence and timing of operations performed. This |
| information is otherwise inaccessible to the mount owner, so this |
| counts as an information leak. |
| |
| The solution to this problem will be presented in point 2) of C). |
| |
| C) There are several ways in which the mount owner can induce |
| undesired behavior in other users' processes, such as: |
| |
| 1) mounting a filesystem over a file or directory which the mount |
| owner could otherwise not be able to modify (or could only |
| make limited modifications). |
| |
| This is solved in fusermount, by checking the access |
| permissions on the mountpoint and only allowing the mount if |
| the mount owner can do unlimited modification (has write |
| access to the mountpoint, and mountpoint is not a "sticky" |
| directory) |
| |
| 2) Even if 1) is solved the mount owner can change the behavior |
| of other users' processes. |
| |
| i) It can slow down or indefinitely delay the execution of a |
| filesystem operation creating a DoS against the user or the |
| whole system. For example a suid application locking a |
| system file, and then accessing a file on the mount owner's |
| filesystem could be stopped, and thus causing the system |
| file to be locked forever. |
| |
| ii) It can present files or directories of unlimited length, or |
| directory structures of unlimited depth, possibly causing a |
| system process to eat up diskspace, memory or other |
| resources, again causing DoS. |
| |
| The solution to this as well as B) is not to allow processes |
| to access the filesystem, which could otherwise not be |
| monitored or manipulated by the mount owner. Since if the |
| mount owner can ptrace a process, it can do all of the above |
| without using a FUSE mount, the same criteria as used in |
| ptrace can be used to check if a process is allowed to access |
| the filesystem or not. |
| |
| Note that the ptrace check is not strictly necessary to |
| prevent B/2/i, it is enough to check if mount owner has enough |
| privilege to send signal to the process accessing the |
| filesystem, since SIGSTOP can be used to get a similar effect. |
| |
| I think these limitations are unacceptable? |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| If a sysadmin trusts the users enough, or can ensure through other |
| measures, that system processes will never enter non-privileged |
| mounts, it can relax the last limitation with a "user_allow_other" |
| config option. If this config option is set, the mounting user can |
| add the "allow_other" mount option which disables the check for other |
| users' processes. |
| |
| Kernel - userspace interface |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The following diagram shows how a filesystem operation (in this |
| example unlink) is performed in FUSE. |
| |
| NOTE: everything in this description is greatly simplified |
| |
| | "rm /mnt/fuse/file" | FUSE filesystem daemon |
| | | |
| | | >sys_read() |
| | | >fuse_dev_read() |
| | | >request_wait() |
| | | [sleep on fc->waitq] |
| | | |
| | >sys_unlink() | |
| | >fuse_unlink() | |
| | [get request from | |
| | fc->unused_list] | |
| | >request_send() | |
| | [queue req on fc->pending] | |
| | [wake up fc->waitq] | [woken up] |
| | >request_wait_answer() | |
| | [sleep on req->waitq] | |
| | | <request_wait() |
| | | [remove req from fc->pending] |
| | | [copy req to read buffer] |
| | | [add req to fc->processing] |
| | | <fuse_dev_read() |
| | | <sys_read() |
| | | |
| | | [perform unlink] |
| | | |
| | | >sys_write() |
| | | >fuse_dev_write() |
| | | [look up req in fc->processing] |
| | | [remove from fc->processing] |
| | | [copy write buffer to req] |
| | [woken up] | [wake up req->waitq] |
| | | <fuse_dev_write() |
| | | <sys_write() |
| | <request_wait_answer() | |
| | <request_send() | |
| | [add request to | |
| | fc->unused_list] | |
| | <fuse_unlink() | |
| | <sys_unlink() | |
| |
| There are a couple of ways in which to deadlock a FUSE filesystem. |
| Since we are talking about unprivileged userspace programs, |
| something must be done about these. |
| |
| Scenario 1 - Simple deadlock |
| ----------------------------- |
| |
| | "rm /mnt/fuse/file" | FUSE filesystem daemon |
| | | |
| | >sys_unlink("/mnt/fuse/file") | |
| | [acquire inode semaphore | |
| | for "file"] | |
| | >fuse_unlink() | |
| | [sleep on req->waitq] | |
| | | <sys_read() |
| | | >sys_unlink("/mnt/fuse/file") |
| | | [acquire inode semaphore |
| | | for "file"] |
| | | *DEADLOCK* |
| |
| The solution for this is to allow requests to be interrupted while |
| they are in userspace: |
| |
| | [interrupted by signal] | |
| | <fuse_unlink() | |
| | [release semaphore] | [semaphore acquired] |
| | <sys_unlink() | |
| | | >fuse_unlink() |
| | | [queue req on fc->pending] |
| | | [wake up fc->waitq] |
| | | [sleep on req->waitq] |
| |
| If the filesystem daemon was single threaded, this will stop here, |
| since there's no other thread to dequeue and execute the request. |
| In this case the solution is to kill the FUSE daemon as well. If |
| there are multiple serving threads, you just have to kill them as |
| long as any remain. |
| |
| Moral: a filesystem which deadlocks, can soon find itself dead. |
| |
| Scenario 2 - Tricky deadlock |
| ---------------------------- |
| |
| This one needs a carefully crafted filesystem. It's a variation on |
| the above, only the call back to the filesystem is not explicit, |
| but is caused by a pagefault. |
| |
| | Kamikaze filesystem thread 1 | Kamikaze filesystem thread 2 |
| | | |
| | [fd = open("/mnt/fuse/file")] | [request served normally] |
| | [mmap fd to 'addr'] | |
| | [close fd] | [FLUSH triggers 'magic' flag] |
| | [read a byte from addr] | |
| | >do_page_fault() | |
| | [find or create page] | |
| | [lock page] | |
| | >fuse_readpage() | |
| | [queue READ request] | |
| | [sleep on req->waitq] | |
| | | [read request to buffer] |
| | | [create reply header before addr] |
| | | >sys_write(addr - headerlength) |
| | | >fuse_dev_write() |
| | | [look up req in fc->processing] |
| | | [remove from fc->processing] |
| | | [copy write buffer to req] |
| | | >do_page_fault() |
| | | [find or create page] |
| | | [lock page] |
| | | * DEADLOCK * |
| |
| Solution is again to let the the request be interrupted (not |
| elaborated further). |
| |
| An additional problem is that while the write buffer is being |
| copied to the request, the request must not be interrupted. This |
| is because the destination address of the copy may not be valid |
| after the request is interrupted. |
| |
| This is solved with doing the copy atomically, and allowing |
| interruption while the page(s) belonging to the write buffer are |
| faulted with get_user_pages(). The 'req->locked' flag indicates |
| when the copy is taking place, and interruption is delayed until |
| this flag is unset. |
| |
| Scenario 3 - Tricky deadlock with asynchronous read |
| --------------------------------------------------- |
| |
| The same situation as above, except thread-1 will wait on page lock |
| and hence it will be uninterruptible as well. The solution is to |
| abort the connection with forced umount (if mount is attached) or |
| through the abort attribute in sysfs. |
