Annotation of wikisrc/puffs.mdwn, revision 1.3

1.1       jdf         1: **Contents**
                      2: 
                      3: [[!toc levels=2]]
                      4: 
                      5: # Filesystems in userspace: puffs, refuse, FUSE, and more
                      6: 
                      7: ## About
                      8: 
                      9: NetBSD now offers full support for running file systems in userspace.  Major
                     10: components are "puffs", which is the kernel subsystem that realizes the
                     11: pass-to-userspace framework file system, as well as the userland libraries that
                     12: support constructing file system implementations. These are libpuffs and the
                     13: FUSE compatibility library, librefuse.
                     14: 
                     15: This page describes the situation in NetBSD 5.0 and later.
                     16: 
                     17: ## Components
                     18: 
                     19: There are a number of components interacting to provide routines for userland
                     20: file systems: puffs as the kernel part, and libpuffs and librefuse to provide
                     21: functions for userland file systems to call. The following image gives an
                     22: overview of the components and their connections:
                     23: 
1.3     ! jdf        24: ![Overview about puffs](/puffs/puffs.png)  
1.1       jdf        25: **Overview about puffs**
                     26: 
                     27: The components themselves are:
                     28: 
                     29:  * **puffs** is the core component inside the kernel. It exposes a file system
                     30:    interface towards the userland programs (accessible via `/dev/puffs` and
                     31:    libpuffs), and communicates with the kernel's own idea of files, which are
                     32:    represented as vnodes. The puffs kernel component and how to interface it is
                     33:    described further in the MAN.PUFFS.4 manpage.
                     34: 
                     35:  * The **libpuffs** library is the interface between userland file systems
                     36:    and the kernel component. It provides a number of convenience routines that
                     37:    userland file systems can use, and is described in the MAN.PUFFS.3 manpage.
                     38: 
                     39:  * To facilitate running the huge amount of file systems already available for
                     40:    the FUSE interface, but not dictate the capabilities of puffs by it, it was
                     41:    decided that FUSE support should be provided as a compatibility layer on top
                     42:    of the native puffs interface, which is offered by **librefuse**. The
                     43:    re-implementation of the FUSE functionality is designed to be source code
                     44:    compatible with FUSE, and it is further described in the
                     45:    [refuse(3)](http://netbsd.gw.com/cgi-bin/man-cgi?refuse+3+NetBSD-current+i386)
                     46:    manpage.
                     47: 
                     48: Examples for puffs and refuse filesystems can be found in the the NetBSD source
                     49: tree in `src/share/examples/{puffs,refuse}`.
                     50: 
                     51: ## Enabling puffs
                     52: 
                     53: puffs is enabled by default in GENERIC on major architectures starting from
                     54: NetBSD 5.0. For a custom kernel, the following are required in the kernel
                     55: configuration.
                     56: 
                     57:     file-system     PUFFS
                     58:     pseudo-device   putter
                     59: 
                     60: Alternatively, modules can be used.
                     61: 
                     62: ## Base system examples
                     63: 
                     64: This section presents usage examples for file systems shipped with the NetBSD
                     65: base system.
                     66: 
                     67: ### psshfs, The NetBSD sshfs
                     68: 
                     69: See the manpage for
                     70: [psshfs(8)](http://netbsd.gw.com/cgi-bin/man-cgi?mount_psshfs+8+NetBSD-5.1+i386).
                     71: 
                     72:     # mount_psshfs host.name.tld:/directory /puffs
                     73:     # ls /puffs
                     74:     AdobeFnt.lst    OS                  bin     public_html
                     75:     Desktop         OpenOffice.org1.1.0 in      tmp
                     76:     ...
                     77:     # cd /puffs
                     78:     # ls -l .cshrc
                     79:     -rw-r--r--  1 39068  2000  4706 Jun 16 01:01 .cshrc
                     80:     # head -2 .cshrc
                     81:     # Default .cshrc for Solaris, Irix, ...
                     82:     #
                     83:     # md5 .cshrc
                     84:     MD5 (.cshrc) = 2ad1d2606a5678f312709a388376c2e5
                     85:     # ls -l test
                     86:     ls: test: No such file or directory
                     87:     # date >test
                     88:     # ls -l test
                     89:     -rw-r--r--  1 39068  2000  29 Nov 23 01:19 test
                     90:     # cat test
                     91:     Thu Nov 23 01:19:36 MET 2006
                     92:     # umount /puffs
                     93: 
                     94: ### 9P file servers
                     95: 
                     96: See the [Bell manpage](http://www.cs.bell-labs.com/sys/man/5/INDEX.html) and
                     97: the [NetBSD manpage 9p(8)](http://netbsd.gw.com/cgi-bin/man-cgi?mount_9p+8+NetBSD-5.1+i386).
                     98: 
                     99:     # mount_9p nobody@192.168.1.2:/tmp /puffs
                    100:     # cd /puffs
                    101:     # echo 9puffs in action > msg_from_earth
                    102:     # ls -l msg_from_earth
                    103:     -rw-r--r--  1 nobody  wheel  17 Apr 25 23:24 msg_from_earth
                    104:     # rsh 192.168.1.2 cat /tmp/msg_from_earth
                    105:     9puffs in action
                    106: 
                    107: Since there is currently no support in the implementation for access control or
                    108: support for authentication, the account nobody was picked just to prove a point.
                    109: The NFS nobody user really does not have anything to do with 9P. Support for
                    110: access control and use of pre-established secure connections will be added to
                    111: `mount_9p` on a later date.
                    112: 
                    113: ## pkgsrc & FUSE
                    114: 
                    115: FUSE compatibility was added within pkgsrc, and besides the required
                    116: infrastructure work a number of FUSE packages were added to pkgsrc in the new
                    117: *filesystem* category. Example packages that are currently available include:
                    118: 
                    119:  * [fuse](ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse/README.html)
                    120:    -- filesystem in userspace (compat headers, pkg-config files, etc.), needed for
                    121:    pkgsrc on Linux
                    122: 
                    123:  * [fuse-archivemount](ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse-archivemount/README.html)
                    124:    -- FUSE gateway to libarchive
                    125: 
                    126:  * [fuse-cddfs](ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse-cddfs/README.html)
                    127:    -- FUSE filesystem that uses libparanoia for audio CDs
                    128: 
                    129:  * [fuse-ntfs-3g](ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse-ntfs-3g/README.html)
                    130:    -- a NTFS driver with read and write support
                    131: 
                    132:  * ...
                    133: 
                    134: More packages are currently being worked on, please see
                    135: [pkgsrc/filesystems](ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/filesystems/README.html)
                    136: for a full list.
                    137: 
                    138: Here is an example of installing and using the FUSE-enabled NTFS-3g
                    139: implementation:
                    140: 
                    141:     # cd pkgsrc/filesystems/fuse-ntfs-3g
                    142:     # make install
                    143:     # ntfs-3g ntfs.img /ntfs
                    144: 
                    145: Using FUSE file systems on NetBSD 4.0 is possible, but in addition to adding
                    146: puffs support support to the kernel, it requires fetching and manually
                    147: installing a backport of the ReFUSE library. The library is available
                    148: [here](http://www.cs.hut.fi/~pooka/NetBSD/librefuse_nb4-20080115.tar.gz)
                    149: and further instructions are available
                    150: [here](http://mail-index.NetBSD.org/netbsd-users/2008/01/15/msg000075.html).
                    151: 
                    152: ## Further Information
                    153: 
                    154: An in-depth technical description of puffs was presented at 
                    155: [AsiaBSDCon](http://www.asiabsdcon.org/) 2007 in a paper entitled
                    156: *puffs - Pass-to-Userspace Framework File System*. The
                    157: [paper](http://2007.asiabsdcon.org/papers/P04-paper.pdf) and
                    158: [slides](http://2007.asiabsdcon.org/papers/P04-slides.pdf) are available.
                    159: 
                    160: The ReFUSE emulation layer for FUSE file systems is described in
1.2       jdf       161: *[ReFUSE: Userspace FUSE Reimplementation Using puffs](/puffs/refuse.pdf)*
1.1       jdf       162: presented at EuroBSDCon 2007.
                    163: 
                    164: A [paper](http://2008.asiabsdcon.org/papers/P4B-paper.pdf) discussing the 
                    165: implementation of distributed file systems on top of puffs was presented at 
                    166: AsiaBSDCon 2008.
                    167: 
                    168: The [puffs(3)](http://man.NetBSD.org/man/puffs+3) manual pages provide further 
                    169: information from a development perspective.
                    170: 
                    171: The source code in browsable form:
                    172: 
                    173:  * [kernel code](http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/fs/puffs/)
                    174:  * [libpuffs](http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libpuffs/)
                    175:  * [librefuse](http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/librefuse/)
                    176:  * [puffs file systems in the base system](http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.sbin/puffs/)
                    177:  * [source code examples of other puffs file systems](http://cvsweb.netbsd.org/bsdweb.cgi/src/share/examples/puffs/)
                    178:  * [source code examples for refuse](http://cvsweb.netbsd.org/bsdweb.cgi/src/share/examples/refuse/)
                    179: 

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb