Annotation of wikisrc/puffs.mdwn, revision 1.1

1.1     ! jdf         1: **Contents**
        !             2: 
        !             3: [[!toc levels=2]]
        !             4: 
        !             5: # Filesystems in userspace: puffs, refuse, FUSE, and more
        !             6: 
        !             7: ## About
        !             8: 
        !             9: NetBSD now offers full support for running file systems in userspace.  Major
        !            10: components are "puffs", which is the kernel subsystem that realizes the
        !            11: pass-to-userspace framework file system, as well as the userland libraries that
        !            12: support constructing file system implementations. These are libpuffs and the
        !            13: FUSE compatibility library, librefuse.
        !            14: 
        !            15: This page describes the situation in NetBSD 5.0 and later.
        !            16: 
        !            17: ## Components
        !            18: 
        !            19: There are a number of components interacting to provide routines for userland
        !            20: file systems: puffs as the kernel part, and libpuffs and librefuse to provide
        !            21: functions for userland file systems to call. The following image gives an
        !            22: overview of the components and their connections:
        !            23: 
        !            24: ![/puffs/puffs.png](Overview about puffs)  
        !            25: **Overview about puffs**
        !            26: 
        !            27: The components themselves are:
        !            28: 
        !            29:  * **puffs** is the core component inside the kernel. It exposes a file system
        !            30:    interface towards the userland programs (accessible via `/dev/puffs` and
        !            31:    libpuffs), and communicates with the kernel's own idea of files, which are
        !            32:    represented as vnodes. The puffs kernel component and how to interface it is
        !            33:    described further in the MAN.PUFFS.4 manpage.
        !            34: 
        !            35:  * The **libpuffs** library is the interface between userland file systems
        !            36:    and the kernel component. It provides a number of convenience routines that
        !            37:    userland file systems can use, and is described in the MAN.PUFFS.3 manpage.
        !            38: 
        !            39:  * To facilitate running the huge amount of file systems already available for
        !            40:    the FUSE interface, but not dictate the capabilities of puffs by it, it was
        !            41:    decided that FUSE support should be provided as a compatibility layer on top
        !            42:    of the native puffs interface, which is offered by **librefuse**. The
        !            43:    re-implementation of the FUSE functionality is designed to be source code
        !            44:    compatible with FUSE, and it is further described in the
        !            45:    [refuse(3)](http://netbsd.gw.com/cgi-bin/man-cgi?refuse+3+NetBSD-current+i386)
        !            46:    manpage.
        !            47: 
        !            48: Examples for puffs and refuse filesystems can be found in the the NetBSD source
        !            49: tree in `src/share/examples/{puffs,refuse}`.
        !            50: 
        !            51: ## Enabling puffs
        !            52: 
        !            53: puffs is enabled by default in GENERIC on major architectures starting from
        !            54: NetBSD 5.0. For a custom kernel, the following are required in the kernel
        !            55: configuration.
        !            56: 
        !            57:     file-system     PUFFS
        !            58:     pseudo-device   putter
        !            59: 
        !            60: Alternatively, modules can be used.
        !            61: 
        !            62: ## Base system examples
        !            63: 
        !            64: This section presents usage examples for file systems shipped with the NetBSD
        !            65: base system.
        !            66: 
        !            67: ### psshfs, The NetBSD sshfs
        !            68: 
        !            69: See the manpage for
        !            70: [psshfs(8)](http://netbsd.gw.com/cgi-bin/man-cgi?mount_psshfs+8+NetBSD-5.1+i386).
        !            71: 
        !            72:     # mount_psshfs host.name.tld:/directory /puffs
        !            73:     # ls /puffs
        !            74:     AdobeFnt.lst    OS                  bin     public_html
        !            75:     Desktop         OpenOffice.org1.1.0 in      tmp
        !            76:     ...
        !            77:     # cd /puffs
        !            78:     # ls -l .cshrc
        !            79:     -rw-r--r--  1 39068  2000  4706 Jun 16 01:01 .cshrc
        !            80:     # head -2 .cshrc
        !            81:     # Default .cshrc for Solaris, Irix, ...
        !            82:     #
        !            83:     # md5 .cshrc
        !            84:     MD5 (.cshrc) = 2ad1d2606a5678f312709a388376c2e5
        !            85:     # ls -l test
        !            86:     ls: test: No such file or directory
        !            87:     # date >test
        !            88:     # ls -l test
        !            89:     -rw-r--r--  1 39068  2000  29 Nov 23 01:19 test
        !            90:     # cat test
        !            91:     Thu Nov 23 01:19:36 MET 2006
        !            92:     # umount /puffs
        !            93: 
        !            94: ### 9P file servers
        !            95: 
        !            96: See the [Bell manpage](http://www.cs.bell-labs.com/sys/man/5/INDEX.html) and
        !            97: the [NetBSD manpage 9p(8)](http://netbsd.gw.com/cgi-bin/man-cgi?mount_9p+8+NetBSD-5.1+i386).
        !            98: 
        !            99:     # mount_9p nobody@192.168.1.2:/tmp /puffs
        !           100:     # cd /puffs
        !           101:     # echo 9puffs in action > msg_from_earth
        !           102:     # ls -l msg_from_earth
        !           103:     -rw-r--r--  1 nobody  wheel  17 Apr 25 23:24 msg_from_earth
        !           104:     # rsh 192.168.1.2 cat /tmp/msg_from_earth
        !           105:     9puffs in action
        !           106: 
        !           107: Since there is currently no support in the implementation for access control or
        !           108: support for authentication, the account nobody was picked just to prove a point.
        !           109: The NFS nobody user really does not have anything to do with 9P. Support for
        !           110: access control and use of pre-established secure connections will be added to
        !           111: `mount_9p` on a later date.
        !           112: 
        !           113: ## pkgsrc & FUSE
        !           114: 
        !           115: FUSE compatibility was added within pkgsrc, and besides the required
        !           116: infrastructure work a number of FUSE packages were added to pkgsrc in the new
        !           117: *filesystem* category. Example packages that are currently available include:
        !           118: 
        !           119:  * [fuse](ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse/README.html)
        !           120:    -- filesystem in userspace (compat headers, pkg-config files, etc.), needed for
        !           121:    pkgsrc on Linux
        !           122: 
        !           123:  * [fuse-archivemount](ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse-archivemount/README.html)
        !           124:    -- FUSE gateway to libarchive
        !           125: 
        !           126:  * [fuse-cddfs](ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse-cddfs/README.html)
        !           127:    -- FUSE filesystem that uses libparanoia for audio CDs
        !           128: 
        !           129:  * [fuse-ntfs-3g](ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse-ntfs-3g/README.html)
        !           130:    -- a NTFS driver with read and write support
        !           131: 
        !           132:  * ...
        !           133: 
        !           134: More packages are currently being worked on, please see
        !           135: [pkgsrc/filesystems](ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/filesystems/README.html)
        !           136: for a full list.
        !           137: 
        !           138: Here is an example of installing and using the FUSE-enabled NTFS-3g
        !           139: implementation:
        !           140: 
        !           141:     # cd pkgsrc/filesystems/fuse-ntfs-3g
        !           142:     # make install
        !           143:     # ntfs-3g ntfs.img /ntfs
        !           144: 
        !           145: Using FUSE file systems on NetBSD 4.0 is possible, but in addition to adding
        !           146: puffs support support to the kernel, it requires fetching and manually
        !           147: installing a backport of the ReFUSE library. The library is available
        !           148: [here](http://www.cs.hut.fi/~pooka/NetBSD/librefuse_nb4-20080115.tar.gz)
        !           149: and further instructions are available
        !           150: [here](http://mail-index.NetBSD.org/netbsd-users/2008/01/15/msg000075.html).
        !           151: 
        !           152: ## Further Information
        !           153: 
        !           154: An in-depth technical description of puffs was presented at 
        !           155: [AsiaBSDCon](http://www.asiabsdcon.org/) 2007 in a paper entitled
        !           156: *puffs - Pass-to-Userspace Framework File System*. The
        !           157: [paper](http://2007.asiabsdcon.org/papers/P04-paper.pdf) and
        !           158: [slides](http://2007.asiabsdcon.org/papers/P04-slides.pdf) are available.
        !           159: 
        !           160: The ReFUSE emulation layer for FUSE file systems is described in
        !           161: *[ReFUSE: Userspace FUSE Reimplementation Using puffs](puffs/refuse.pdf)*
        !           162: presented at EuroBSDCon 2007.
        !           163: 
        !           164: A [paper](http://2008.asiabsdcon.org/papers/P4B-paper.pdf) discussing the 
        !           165: implementation of distributed file systems on top of puffs was presented at 
        !           166: AsiaBSDCon 2008.
        !           167: 
        !           168: The [puffs(3)](http://man.NetBSD.org/man/puffs+3) manual pages provide further 
        !           169: information from a development perspective.
        !           170: 
        !           171: The source code in browsable form:
        !           172: 
        !           173:  * [kernel code](http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/fs/puffs/)
        !           174:  * [libpuffs](http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libpuffs/)
        !           175:  * [librefuse](http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/librefuse/)
        !           176:  * [puffs file systems in the base system](http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.sbin/puffs/)
        !           177:  * [source code examples of other puffs file systems](http://cvsweb.netbsd.org/bsdweb.cgi/src/share/examples/puffs/)
        !           178:  * [source code examples for refuse](http://cvsweb.netbsd.org/bsdweb.cgi/src/share/examples/refuse/)
        !           179: 

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb