File:  [NetBSD Developer Wiki] / wikisrc / tutorials / bus_space_tutorial.mdwn
Revision 1.7: download - view: text, annotated - select for diffs
Sun Jun 23 15:07:22 2013 UTC (8 years, 5 months ago) by mspo
Branches: MAIN
CVS tags: HEAD
try startlevel=2 in the toc

    1: [[!toc startlevel=2]]
    2: 
    3: ## Introduction
    4: 
    5: ### Why was this tutorial created?
    6: 
    7: -   Introductory-level documentation is scarce
    8: -   Writing device drivers is often considered black magic
    9: -   Reading the man pages won’t give you the big picture
   10: -   BSD systems are always in need of new drivers
   11: -   Device drivers are fun
   12: 
   13: ### What won’t be covered here?
   14: 
   15: We don’t have much time, so several ~~advanced~~ topics were omitted:
   16: 
   17: -   Interrupt handling
   18: -   Direct Memory Access and the bus\_dma framework
   19: -   Power management
   20: -   Driver detachment
   21: -   Drivers as kernel modules
   22: -   Examples for buses other than PCI
   23: -   Pretty much everything else...
   24: 
   25: However, once you finish this tutorial, you should be able to pursue
   26: this knowledge yourself.
   27: 
   28: ### What is a driver anyway?
   29: 
   30: -   The interface between user space and hardware, implemented as a part
   31:     of the kernel
   32: -   The NetBSD drivers are written mostly in C
   33: -   Sometimes they have machine dependent assembler parts, but this is a
   34:     rare case
   35: 
   36: ### What do you need to write a driver?
   37: 
   38: -   C programming skills
   39: -   Hardware documentation (or the ability to reverse engineer the
   40:     hardware)
   41: -   A reference driver implementation will help but is not essential
   42: -   A NetBSD installation and kernel source, or a cross-build
   43:     environment (the latter is usually preferred for development of
   44:     drivers)
   45: -   A lot of time, coffee and patience
   46: 
   47: ### Why is writing the device drivers considered difficult?
   48: 
   49: -   It’s not as difficult as you may expect, in fact during this
   50:     tutorial we’ll prove that it’s quite easy
   51: -   You need to think on a very low level
   52:     -   Good understanding of computer architecture is a must
   53: -   Often documentation is the main problem – writing the driver is not
   54:     possible if you don’t understand how the device works
   55:     -   No access to documentation (uncooperative hardware vendors,
   56:         vendors out of business)
   57:     -   Documentation is incomplete or plain wrong
   58:     -   Reverse engineering can solve these problems but it’s a very
   59:         time consuming process
   60: 
   61: ## The NetBSD driver model
   62: 
   63: ### The NetBSD kernel basics
   64: 
   65: -   NetBSD has a classic monolithic UNIX-like kernel - all drivers are
   66:     running in the same address space
   67: -   Thanks to the above, communication between drivers and other kernel
   68:     layers is simple
   69: -   However, it also means that one badly written driver can affect the
   70:     whole kernel
   71: -   Numerous in-kernel frameworks standardise the way drivers are
   72:     written (bus\_space, autoconf, etc.)
   73: 
   74: ### The NetBSD source directory structure
   75: 
   76: -   We’ll only cover parts interesting for a device driver programmer
   77: -   src/sys/
   78:     - kernel source directory
   79: -   src/sys/dev/
   80:     - machine-independent device drivers
   81: -   src/sys/arch/
   82:     - port-specific or architecture-specific parts (such as the
   83:     low-level system initialisation procedures or machine-dependent
   84:     drivers)
   85: -   src/sys/arch/$PORTNAME/conf/
   86:     - kernel configuration files for a given port
   87: 
   88: ### Kernel autoconfiguration framework - autoconf(9)
   89: 
   90: -   Autoconfiguration is the process of matching hardware devices with
   91:     an appropriate device driver
   92: -   The kernel message buffer (dmesg) contains information about
   93:     autoconfiguration of devices
   94: -   driver0 at bus0: Foo hardware
   95:     -   Instance 0 of the driver has attached to instance 0 of the
   96:         particular bus
   97:     -   Such messages often carry additional bus-specific information
   98:         about the exact location of the device (like the device and
   99:         function number on the PCI bus)
  100: -   driver0: some message
  101:     -   Additional information about the driver state or device
  102:         configuration
  103: 
  104: ### Autoconfiguration as seen in the dmesg
  105: 
  106:     NetBSD 6.99.12 (GENERIC) #7: Fri Oct  5 18:43:21 CEST 2012
  107:             rkujawa@saiko.local:/Users/rkujawa/netbsd-eurobsdcon2012/src/sys/arch/cobalt/compile/obj/GENERIC
  108:     Cobalt Qube 2
  109:     total memory = 32768 KB
  110:     avail memory = 27380 KB
  111:     mainbus0 (root)
  112:     com0 at mainbus0 addr 0x1c800000 level 3: ns16550a, working fifo
  113:     com0: console
  114:     cpu0 at mainbus0: QED RM5200 CPU (0x28a0) Rev. 10.0 with built-in FPU Rev. 1.0
  115:     cpu0: 48 TLB entries, 256MB max page size
  116:     cpu0: 32KB/32B 2-way set-associative L1 instruction cache
  117:     cpu0: 32KB/32B 2-way set-associative write-back L1 data cache
  118:     mcclock0 at mainbus0 addr 0x10000070: mc146818 compatible time-of-day clock
  119:     panel0 at mainbus0 addr 0x1f000000
  120:     gt0 at mainbus0 addr 0x14000000
  121:     pci0 at gt0
  122:     pchb0 at pci0 dev 0 function 0: Galileo GT-64011 System Controller, rev 1
  123:     pcib0 at pci0 dev 9 function 0
  124:     pcib0: VIA Technologies VT82C586 PCI-ISA Bridge, rev 57
  125:     viaide0 at pci0 dev 9 function 1
  126:     viaide0: VIA Technologies VT82C586 (Apollo VP) ATA33 controller
  127:     viaide0: primary channel interrupting at irq 14
  128:     atabus0 at viaide0 channel 0
  129:     viaide0: secondary channel interrupting at irq 15
  130:     atabus1 at viaide0 channel 1
  131:     wd0 at atabus0 drive 0
  132:     wd0: <netbsd-cobalt.img>
  133:     wd0: 750 MB, 1524 cyl, 16 head, 63 sec, 512 bytes/sect x 1536192 sectors
  134: 
  135: ### Autoconfiguration as seen in the dmesg
  136: 
  137: ![image](img_cobaltdevices.png)
  138: 
  139: ### The bus\_space(9) framework
  140: 
  141: -   “The goal of the bus\_space functions is to allow a single driver
  142:     source file to manipulate a set of devices on different system
  143:     architectures, and to allow a single driver object file to
  144:     manipulate a set of devices on multiple bus types on a single
  145:     architecture.”
  146: -   Provides a set of functions implementing common operations on the
  147:     bus like mapping, reading, writing, copying, etc.
  148: -   The bus\_space(9) is implemented at the machine-dependent level
  149:     (typically it’s a part of architecture-specific code), but all
  150:     implementations present the same interface
  151: > At least they should, some functions are missing on less popular ports
  152: 
  153: ### Machine independent drivers
  154: 
  155: -   If possible drivers should work on any hardware platform
  156: -   High quality, machine-independent (MI) drivers are an important
  157:     factor that adds to NetBSD portability
  158: -   Some drivers are completely MI, some have MD or bus dependent
  159:     attachments and some are completely MD
  160:     -   A driver for a typical PCI card will be completely MI
  161:     -   A driver for the components of a SoC will usually be completely
  162:         MD
  163: -   The bus\_space abstraction helps to achieve portability,
  164:     transparently handling endianness issues and hiding bus
  165:     implementation details from the device driver
  166: -   Even if we have MI drivers, writing the drivers is always
  167:     significant part of effort needed to port NetBSD to new hardware
  168: 
  169: ## Example driver from scratch
  170: 
  171: ### Development environment
  172: 
  173: -   Out of scope of this course, but very well documented
  174: -   Cross compiling is an easy task with the build.sh script
  175: -   Described in [Part V of the NetBSD
  176:     Guide](http://www.netbsd.org/docs/guide/en/part-compile.html)
  177: -   Check out the NetBSD sources
  178: -   $ build.sh -m cobalt tools
  179:     will build compiler, assembler, linker, etc. for cobalt port
  180: -   $ build.sh -m cobalt kernel=GENERIC
  181:     will build the GENERIC kernel for cobalt
  182: -   Call build.sh with a -u parameter to update (won’t rebuilding
  183:     everything)
  184: -   build.sh
  185:     is calling nbconfig and nbmake tools, no magic involved
  186: 
  187: ### Quick introduction to GXemul
  188: 
  189: -   A framework for full-system computer architecture emulation,
  190:     excellent for educational purposes
  191: -   Capable of emulating several real machines supported by NetBSD
  192: -   We’ll emulate a [Cobalt](http://en.wikipedia.org/wiki/Cobalt_Qube),
  193:     MIPS-based micro server with PCI bus
  194: -   I’ve modified GXemul and implemented an emulation of an additional
  195:     PCI device
  196: -   It will be used to show (almost) a real-life example of the driver
  197:     development process
  198: 
  199: ### Our hardware - functional description
  200: 
  201: -   Business applications often use arithmetic operations like addition
  202: -   Fake Cards Inc. responded to market needs and created a new product,
  203:     Advanced Addition Accelerator
  204: -   Pointy Haired Bosses will certainly buy it to accelerate their
  205:     business applications, so let’s create a driver for NetBSD!
  206: 
  207: ### Our hardware - technical details
  208: 
  209: -   Overview
  210:     -   Implemented as a PCI device
  211:     -   Arithmetic unit capable of addition of two numbers
  212:     -   Four registers in the PCI memory space
  213: > Only three of these registers are of any importance for us at this moment
  214: -   PCI configuration space
  215:     -   Identified by the PCI vendor ID 0xfabc and product ID 0x0001
  216:     -   Base Address Register 0x10 used to configure the engine address
  217:     -   4 x 32-bit registers = 16 bytes
  218:     -   Other configuration registers irrelevant
  219: 
  220: ### Our hardware - technical details (memory mapped register set)
  221: 
  222: -   Advanced Addition Acceleration registers
  223: 
  224: <table>
  225: <tr><th>Register Name</th><th>Offset</th><th>Description</th></tr>
  226: <tr><td>COMMAND</td><td>0x4</td><td>Register used to issue commands to the engine</td></tr>
  227: <tr><td>DATA</td><td>0x8</td><td>Register used to load data to internal engine registers</td></tr>
  228: <tr><td>RESULT</td><td>0xC</td><td>Register used to store the result of arithmetic operation</td></tr>
  229: </table>
  230: 
  231: -   COMMAND register
  232: 
  233: <table>
  234: <tr><th>Bit</th><th>R/W</th><th>Description</th></tr>
  235: <tr><td>0</td><td>W</td><td>Execute ADD operation on values loaded into internal register A and B</td></tr>
  236: <tr><td>1</td><td>R/W</td><td>Select internal register A for access through DATA register</td></tr>
  237: <tr><td>2</td><td>R/W</td><td>Select internal register B for access through DATA register</td></tr>
  238: </table>
  239: 
  240: -   Selecting internal register A and B at the same time will lead to
  241:     undefined behaviour
  242: 
  243: ### Our hardware - technical details (memory mapped register set)
  244: 
  245: -   DATA register
  246: 
  247: <table>
  248: <tr><th>Bit</th><th>R/W</th><th>Description</th></tr>
  249: <tr><td>0:31</td><td>R/W</td><td>Read/write the value in internal engine register</td></tr>
  250: </table>
  251: 
  252: -   RESULT register
  253: 
  254: <table>
  255: <tr><th>Bit</th><th>R/W</th><th>Description</th></tr>
  256: <tr><td>0:31</td><td>R</td><td>Holds the result of last ADD operation</td></tr>
  257: </table>
  258: 
  259: ### Our hardware - technical details (operation algorithm)
  260: 
  261: -   Select the internal register A for access (write 0x2 into COMMAND
  262:     register)
  263: -   Write the first number into DATA register
  264: -   Select the internal register B for access (write 0x4 into COMMAND
  265:     register)
  266: -   Write the second number into DATA register
  267: -   Issue the ADD operation (write 0x1 into COMMAND register)
  268: -   Read the result from RESULT register
  269: 
  270: ### Adding a new driver to the NetBSD kernel
  271: 
  272: -   We’ll discuss the steps needed to add a new MI PCI device driver to
  273:     the NetBSD kernel
  274:     -   Add the vendor and device ID to the database of PCI IDs
  275:     -   Create a set of the driver source files in
  276:         src/sys/dev/$BUSNAME/
  277:     -   Add the new driver to src/sys/dev/$BUSNAME/$BUSNAME.files file
  278:     -   Add the new driver to DEVNAMES file
  279: > Required if you are NetBSD developer, optional otherwise.
  280: 
  281: ### Modifying the PCI device database
  282: 
  283:     unmatched vendor 0xfabc product 0x0001 (Co-processor 
  284:     processor, revision 0x01) at pci0 dev 12 function 0 
  285:     not configured
  286: 
  287: -   The kernel does not know anything about this vendor and device
  288: -   Add it to the PCI device database - src/sys/dev/pci/pcidevs
  289: -   vendor VENDORNAME 0xVENDORID Long Vendor Name
  290: -   product VENDORNAME PRODUCTNAME 0xPRODUCTID Long Product Name
  291: -   To regenerate pcidevs\*.h run awk -f devlist2h.awk pcidevs or
  292:     Makefile.pcidevs if you’re on NetBSD
  293: 
  294: ### Modifying the PCI device database - example
  295: 
  296:     --- pcidevs 29 Sep 2012 10:26:14 -0000  1.1139
  297:     +++ pcidevs 5 Oct 2012 08:52:59 -0000
  298:     @@ -669,6 +669,7 @@
  299:      vendor CHRYSALIS   0xcafe  Chrysalis-ITS
  300:      vendor MIDDLE_DIGITAL  0xdeaf  Middle Digital
  301:      vendor ARC     0xedd8  ARC Logic
  302:     +vendor FAKECARDS   0xfabc  Fake Cards
  303:      vendor INVALID     0xffff  INVALID VENDOR ID
  304:      
  305:      /*
  306:     @@ -2120,6 +2121,9 @@
  307:      /* Eumitcom products */
  308:      product EUMITCOM WL11000P  0x1100  WL11000P PCI WaveLAN/IEEE 802.11
  309:      
  310:     +/* FakeCards products */
  311:     +product FAKECARDS AAA      0x0001  Advanced Addition Accelerator
  312:     +
  313:      /* O2 Micro */
  314:      product O2MICRO 00F7       0x00f7  Integrated OHCI IEEE 1394 Host Controller
  315:      product O2MICRO OZ6729     0x6729  OZ6729 PCI-PCMCIA Bridge
  316: 
  317: ### Modifying the PCI device database - example
  318: 
  319:     Fake Cards Advanced Addition Accelerator (Co-processor 
  320:     processor, revision 0x01) at pci0 dev 12 function 0 
  321:     not configured
  322: 
  323: -   Now the kernel knows the vendor and product ID
  324: -   But there’s still no driver for this device
  325: 
  326: ### Adding the new PCI driver
  327: 
  328: -   Choose a name - short, easy to remember, avoid numbers
  329:     -   faa looks like a good name, but you can choose any name you like
  330: -   Create a set of new files in src/sys/dev/pci
  331:     -   faa.c
  332:         - main driver code
  333:     -   faareg.h
  334:         - register definitions
  335: > Might not exist if the driver is only a simple passthrough from a specific bus to another MI driver.
  336:     -   faavar.h
  337:         - driver structures and functions used in other parts of the
  338:         kernel
  339: > Omitted if not needed.
  340: -   Modify driver definitions
  341:     -   src/sys/dev/pci/files.pci
  342:     -   src/sys/dev/DEVNAMES
  343: -   Configure the kernel to use the newly added driver -
  344:     src/sys/arch/$PORTNAME/conf/GENERIC
  345: 
  346: ### Adding the new PCI driver - main driver
  347: 
  348: -   Kernel includes are at the beginning, followed by machine-specific
  349:     and bus-specific includes
  350: -   Should also include faareg.h and faavar.h files
  351: -   A minimal driver needs just two functions
  352:     -   faa\_match
  353:         (or faa\_probe for some buses)
  354:     -   faa\_attach
  355: -   The CFATTACH\_DECL\_NEW macro plugs the above functions into
  356:     autoconf(9) mechanism
  357: 
  358: ### Adding the new PCI driver - main driver
  359: 
  360: -   static int faa\_match(device\_t parent, cfdata\_t match, void
  361:     \*aux);
  362:     -   Check if the driver should attach to a given device (for example
  363:         in case of PCI bus, it will be used to check vendor and product
  364:         ID)
  365:     -   parent
  366:         - pointer to parent’s driver device structure
  367:     -   match
  368:         - pointer to autoconf(9) details structure
  369:     -   aux
  370:         - despite the name the most important argument, usually contains
  371:         bus-specific structure describing device details
  372: 
  373: -   static void faa\_attach(device\_t parent, device\_t self, void
  374:     \*aux);
  375:     -   Attach the driver to a given device
  376:     -   parent
  377:         - same as with match function
  378:     -   self
  379:         - pointer to driver’s device structure
  380:     -   aux
  381:         - same as with match function
  382: -   See definitions of these functions in the
  383:     [driver(9)](http://netbsd.gw.com/cgi-bin/man-cgi?driver+9+NetBSD-current)
  384:     man page.
  385: 
  386: ### Adding the new PCI driver - main driver cont’d
  387: 
  388: -   CFATTACH\_DECL\_NEW(faa, sizeof(struct faa\_softc), faa\_match,
  389:     faa\_attach, NULL, NULL);
  390:     -   driver name
  391:     -   size of softc structure containing state of driver’s instance
  392:     -   match/probe function
  393:     -   attach function
  394:     -   detach function
  395:     -   activate function
  396: -   The “\_NEW” name is unfortunate
  397: -   Pass NULL for unimplemented functions
  398: -   We won’t cover detach and activate now, as they are not needed for a
  399:     simple driver
  400: 
  401: ### Adding the new PCI driver - main driver example
  402: 
  403: -   src/sys/dev/pci/faa.c
  404: 
  405: <!-- -->
  406: 
  407:     #include <sys/cdefs.h>
  408:     __KERNEL_RCSID(0, "$NetBSD: bus_space_tutorial.mdwn,v 1.7 2013/06/23 15:07:22 mspo Exp $");
  409:     #include <sys/param.h>
  410:     #include <sys/device.h>
  411:     #include <dev/pci/pcivar.h>
  412:     #include <dev/pci/pcidevs.h>
  413:     #include <dev/pci/faareg.h>
  414:     #include <dev/pci/faavar.h>
  415: 
  416:     static int      faa_match(device_t, cfdata_t, void *);
  417:     static void     faa_attach(device_t, device_t, void *);
  418: 
  419:     CFATTACH_DECL_NEW(faa, sizeof(struct faa_softc),
  420:         faa_match, faa_attach, NULL, NULL);
  421: 
  422:     static int
  423:     faa_match(device_t parent, cfdata_t match, void *aux)
  424:     {
  425:             return 0;
  426:     }
  427: 
  428:     static void
  429:     faa_attach(device_t parent, device_t self, void *aux)
  430:     { 
  431:     }
  432: 
  433: ### Adding the new PCI driver - auxiliary includes
  434: 
  435: -   src/sys/dev/pci/faareg.h
  436: 
  437: <!-- -->
  438: 
  439:     #ifndef FAAREG_H
  440:     #define FAAREG_H
  441:     /* 
  442:      * Registers are defined using preprocessor:
  443:      * #define FAA_REGNAME  0x0
  444:      * We'll add them later, let's leave it empty for now.
  445:      */
  446:     #endif /* FAAREG_H */
  447: 
  448: -   src/sys/dev/pci/faavar.h
  449: 
  450: <!-- -->
  451: 
  452:     #ifndef FAAVAR_H
  453:     #define FAAVAR_H
  454: 
  455:     /* sc_dev is an absolute minimum, we'll add more later */
  456:     struct faa_softc {
  457:             device_t sc_dev;
  458:     };
  459:     #endif /* FAAVAR_H */
  460: 
  461: ### Adding the new PCI driver - registering the driver (courtesy)
  462: 
  463: -   src/sys/dev/DEVNAMES
  464: 
  465: <!-- -->
  466: 
  467:     --- DEVNAMES    1 Sep 2012 11:19:58 -0000   1.279
  468:     +++ DEVNAMES    6 Oct 2012 19:59:06 -0000
  469:     @@ -436,6 +436,7 @@
  470:      ex         MI
  471:      exphy          MI
  472:      ezload         MI      Attribute
  473:     +faa            MI
  474:      fb         luna68k
  475:      fb         news68k
  476:      fb         newsmips
  477: 
  478: ### Adding the new PCI driver - registering the driver
  479: 
  480: -   See config(5)
  481: -   src/sys/dev/pci/files.pci
  482: 
  483: <!-- -->
  484: 
  485:     --- pci/files.pci   2 Aug 2012 00:17:44 -0000   1.360
  486:     +++ pci/files.pci   6 Oct 2012 19:59:10 -0000
  487:     @@ -1122,3 +1122,9 @@
  488:      device tdvfb: wsemuldisplaydev, rasops8, vcons, videomode
  489:      attach tdvfb at pci
  490:      file   dev/pci/tdvfb.c     tdvfb   
  491:     +
  492:     +# FakeCards Advanced Addition Accelerator
  493:     +device faa
  494:     +attach faa at pci
  495:     +file   dev/pci/faa.c       faa 
  496:     +
  497: 
  498: ### Adding the new PCI driver to the kernel configuration
  499: 
  500: -   src/sys/arch/cobalt/conf/GENERIC
  501: 
  502: <!-- -->
  503: 
  504:     --- GENERIC 10 Mar 2012 21:51:50 -0000  1.134
  505:     +++ GENERIC 6 Oct 2012 20:12:37 -0000
  506:     @@ -302,6 +302,9 @@
  507:      #fms*      at pci? dev ? function ?    # Forte Media FM801
  508:      #sv*       at pci? dev ? function ?    # S3 SonicVibes
  509:      
  510:     +# Fake Cards Advanced Addition Accelerator
  511:     +faa*       at pci? dev ? function ?
  512:     +
  513:      # Audio support
  514:      #audio*        at audiobus?
  515: 
  516: -   The above definition means that an instance of faa may be attached
  517:     to any PCI bus, any device, any function
  518: -   The exact position of the rule in the configuration file is not
  519:     important in this case
  520: -   See
  521:     [config(5)](http://netbsd.gw.com/cgi-bin/man-cgi?config+5+NetBSD-current)
  522:     for a description of the device definition language
  523: 
  524: ### Adding the new PCI driver - example
  525: 
  526: -   The driver should compile now
  527: -   The driver’s match function will check if the driver is able to work
  528:     with a given device
  529: -   Since it is not implemented, the kernel will not attach the driver
  530: 
  531: ### Matching the PCI device
  532: 
  533: -   Modify the faa\_match function to match the specified PCI device
  534: -   Use PCI\_VENDOR and PCI\_PRODUCT macros to obtain the IDs
  535: 
  536: <!-- -->
  537: 
  538:     static int
  539:     faa_match(device_t parent, cfdata_t match, void *aux)
  540:     {
  541:             const struct pci_attach_args *pa = (const struct pci_attach_args *)aux;
  542: 
  543:             if ((PCI_VENDOR(pa->pa_id) == PCI_VENDOR_FAKECARDS) 
  544:                 && (PCI_PRODUCT(pa->pa_id) == PCI_PRODUCT_FAKECARDS_AAA))
  545:                     return 1;
  546: 
  547:             return 0;
  548:     }
  549: 
  550: ### Attaching to the PCI device
  551: 
  552:     faa0 at pci0 dev 12 function 0
  553: 
  554: -   The driver has successfully matched and attached to the PCI device
  555:     but still is not doing anything useful
  556: -   Let’s fill the attach function and actually program the hardware
  557: 
  558: ### Variable types used with bus\_space
  559: 
  560: -   bus\_space\_tag\_t
  561: 
  562:     – type used to describe a particular bus, usually passed to the
  563:     driver from MI bus structures
  564: 
  565: -   bus\_space\_handle\_t
  566: 
  567:     – used to describe a mapped range of bus space, usually created with
  568:     the bus\_space\_map() function
  569: 
  570: -   bus\_addr\_t
  571: 
  572:     – address on the bus
  573: 
  574: -   bus\_size\_t
  575: 
  576:     – an amount of space on the bus
  577: 
  578: -   Contents of these types are MD, so avoid modifying from within the
  579:     driver
  580: > although you’ll often have to use bus\_size\_t
  581: 
  582: ### Why do we need to “map” the resources?
  583: 
  584: -   “The bus space must be mapped before it can be used, and should be
  585:     unmapped when it is no longer needed”
  586: -   It’s a machine-dependent process but it’s also conveniently hidden
  587:     from the programmer by the bus\_space framework
  588: 
  589: ### Mapping the hardware resources
  590: 
  591: -   The generic bus\_space(9) way to map space
  592: 
  593: <!-- -->
  594: 
  595:     bus_space_map(bus_space_tag_t space, bus_addr_t address, 
  596:     bus_size_t size, int flags, bus_space_handle_t  *handlep);
  597: 
  598: -   bus\_space\_map
  599: 
  600:     creates a mapping from the physical address to a kernel virtual
  601:     address
  602: 
  603: -   space
  604: 
  605:     – represents the bus on which the mapping will be created
  606: 
  607: -   address
  608: 
  609:     – typically represents the physical address for which a mapping will
  610:     be created
  611: 
  612: -   size
  613: 
  614:     – describes the amount of bus space to be mapped
  615: 
  616: -   handlep
  617: 
  618:     – pointer to mapped space (filled after successful mapping)
  619: 
  620: -   Separate space and address
  621: 
  622: ### Mapping the hardware resources
  623: 
  624: -   The PCI-specific way to map space
  625: 
  626: <!-- -->
  627: 
  628:     pci_mapreg_map(const struct pci_attach_args *pa, int reg, pcireg_t type, 
  629:     int busflags, bus_space_tag_t *tagp, bus_space_handle_t *handlep, 
  630:     bus_addr_t *basep, bus_size_t *sizep);
  631: 
  632: -   pci\_mapreg\_map
  633: 
  634:     creates mapping from physical address present in specified BAR
  635:     register to kernel virtual address
  636: 
  637: -   pa
  638: 
  639:     – struct describing PCI attachment details (passed through aux)
  640: 
  641: -   reg
  642: 
  643:     – BAR register number
  644: 
  645: -   type
  646: 
  647:     – Select mapping type (I/O, memory)
  648: 
  649: -   busflags
  650: 
  651:     – Passed to bus\_space\_map flags argument
  652: 
  653: -   tagp
  654: 
  655:     – pointer to bus\_space\_tag
  656: 
  657: -   handlep
  658: 
  659:     – pointer to a mapped space
  660: 
  661: -   basep
  662: 
  663:     – address of a mapped space
  664: 
  665: -   sizep
  666: 
  667:     – size of mapped space (equivalent to BAR size)
  668: 
  669: -   The last four parameters are filled after successful mapping
  670: 
  671: ### Mapping the registers using BAR - adding auxiliary includes
  672: 
  673: -   src/sys/dev/pci/faareg.h
  674: 
  675: <!-- -->
  676: 
  677:     #define FAA_MMREG_BAR   0x10
  678: 
  679: -   src/sys/dev/pci/faavar.h
  680: 
  681: <!-- -->
  682: 
  683:     struct faa_softc {
  684:             device_t sc_dev;
  685: 
  686:             bus_space_tag_t sc_regt;
  687:             bus_space_handle_t sc_regh;
  688:             bus_addr_t sc_reg_pa;
  689: 
  690:     };
  691: 
  692: ### Mapping the registers using BAR - main driver code
  693: 
  694: -   src/sys/dev/pci/faa.c
  695: 
  696: <!-- -->
  697: 
  698:     static void
  699:     faa_attach(device_t parent, device_t self, void *aux)
  700:     {
  701:             struct faa_softc *sc = device_private(self);
  702:             const struct pci_attach_args *pa = aux;
  703: 
  704:             sc->sc_dev = self;
  705: 
  706:             pci_aprint_devinfo(pa, NULL);
  707: 
  708:             if (pci_mapreg_map(pa, FAA_MMREG_BAR, PCI_MAPREG_TYPE_MEM, 0, 
  709:                 &sc->sc_regt, &sc->sc_regh, &sc->sc_reg_pa, 0) != 0 ) {
  710:                 aprint_error_dev(sc->sc_dev, "can't map the BAR\n");
  711:                 return;
  712:             }
  713: 
  714:             aprint_normal_dev(sc->sc_dev, "regs at 0x%08x\n", (uint32_t) sc->sc_reg_pa);
  715:     }
  716: 
  717: ### Accessing the hardware registers
  718: 
  719: -   The bus\_space\_read\_ and bus\_space\_write\_ functions are basic
  720:     methods of reading and writing the hardware registers
  721: 
  722: -   uintX\_t bus\_space\_read\_X(bus\_space\_tag\_t space,
  723:     bus\_space\_handle\_t handle, bus\_size\_t offset);
  724: 
  725: -   void bus\_space\_write\_X(bus\_space\_tag\_t space,
  726:     bus\_space\_handle\_t handle, bus\_size\_t offset, uintX\_t value);
  727: 
  728:     -   space
  729: 
  730:         - tag describing the bus
  731: 
  732:     -   handle
  733: 
  734:         - describes the exact location on the bus where read/write
  735:         should occur, this handle is obtained by bus\_space\_map
  736: 
  737:     -   offset
  738: 
  739:         - offset from handle location
  740: 
  741:     -   The read function returns the data read from the specified
  742:         location, while write has an argument value which should be
  743:         filled with data to be written
  744: 
  745: ### Variants of bus\_space\_read and bus\_space\_write
  746: 
  747: <table>
  748: <tr><th>Data</th><th>Read function</th><th>Write function</th></tr>
  749: <tr><td>8-bit</td><td>bus\_space\_read\_1</td><td>bus\_space\_write\_1</td></tr>
  750: <tr><td>16-bit</td><td>bus\_space\_read\_2</td><td>bus\_space\_write\_2</td></tr>
  751: <tr><td>32-bit</td><td>bus\_space\_read\_4</td><td>bus\_space\_write\_4</td></tr>
  752: <tr><td>64-bit</td><td>bus\_space\_read\_8</td><td>bus\_space\_write\_8</td></tr>
  753: </table>
  754: 
  755: -   There are many more variants of read and write functions and they
  756:     are useful in certain situations, see the
  757:     [bus\_space(9)](http://netbsd.gw.com/cgi-bin/man-cgi?bus_space++NetBSD-current)
  758:     man page
  759: 
  760: ### Accessing the hardware registers - example
  761: 
  762: -   Create a function that will write a value into the DATA register of
  763:     our device, then read it back and check if the value is the same as
  764:     written
  765: 
  766: -   Define the DATA register in the driver
  767: 
  768: -   src/sys/dev/pci/faareg.h
  769: 
  770: <!-- -->
  771: 
  772:     #define FAA_DATA                0x8
  773:     #define FAA_COMMAND             0x4
  774:     #define FAA_COMMAND_STORE_A         __BIT(1)
  775: 
  776: -   Define the new function in main driver code
  777: 
  778: -   static bool faa\_check(struct faa\_softc \*sc);
  779: 
  780: ### Accessing the hardware registers - example
  781: 
  782: -   src/sys/dev/pci/faa.c
  783: 
  784: <!-- -->
  785: 
  786:     static void
  787:     faa_attach(device_t parent, device_t self, void *aux)
  788:     {
  789:        /* ... */
  790:        if (!faa_check(sc)) {
  791:             aprint_error_dev(sc->sc_dev, "hardware not responding\n");
  792:             return;
  793:        }
  794:     }
  795: 
  796:     static bool
  797:     faa_check(struct faa_softc *sc)
  798:     {
  799:             uint32_t testval = 0xff11ee22; 
  800:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_COMMAND, FAA_COMMAND_STORE_A);
  801:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_DATA, testval);
  802:             if (bus_space_read_4(sc->sc_regt, sc->sc_regh, FAA_DATA) == testval)
  803:                     return true;
  804: 
  805:             return false;
  806:     }
  807: 
  808: ### Accessing the hardware registers - running the example
  809: 
  810: -   Update the kernel binary and run it again
  811: 
  812: -   Check the GXemul log
  813: 
  814: <!-- -->
  815: 
  816:     [ faa: COMMAND register (0x4) WRITE value 0x2 ]
  817:     [ faa: DATA register (0x8) WRITE value 0xff11ee22 ]
  818:     [ faa: DATA register (0x8) READ value 0xff11ee22 ]
  819: 
  820: -   GXemul will conveniently display all accesses to our device
  821: 
  822: -   The faa driver still does attach without error, which means that the
  823:     check function is working properly
  824: 
  825: <!-- -->
  826: 
  827:     faa0 at pci0 dev 12 function 0: Fake Cards Advanced Addition Accelerator (rev. 0x01)
  828:     faa0: registers at 0x10110000
  829: 
  830: ### Implementing addition using the hardware
  831: 
  832: -   The basic principle of device operation should be laid out in the
  833:     data sheet
  834: 
  835: -   We need to implement an algorithm based on this description
  836: 
  837: -   Writing such an algorithm is often not needed, since the NetBSD
  838:     kernel already has frameworks for common device types (such as
  839:     atabus/wd for IDE and SATA hard disk controllers, wsdisplay/wscons
  840:     for frame buffers, etc.)
  841: 
  842: ### Implementing addition using the hardware
  843: 
  844: -   Define all registers
  845: 
  846: -   src/sys/dev/pci/faareg.h
  847: 
  848: <!-- -->
  849: 
  850:     #define FAA_STATUS              0x0
  851:     #define FAA_COMMAND             0x4
  852:     #define FAA_COMMAND_ADD             __BIT(0)        
  853:     #define FAA_COMMAND_STORE_A         __BIT(1)
  854:     #define FAA_COMMAND_STORE_B         __BIT(2)
  855:     #define FAA_DATA                0x8
  856:     #define FAA_RESULT              0xC
  857: 
  858: ### Implementing addition using the hardware
  859: 
  860: -   Add a new function to the main driver code
  861: 
  862: -   src/sys/dev/pci/faa.c
  863: 
  864: <!-- -->
  865: 
  866:     static void
  867:     faa_attach(device_t parent, device_t self, void *aux)
  868:     {
  869:             /* ... */
  870:             aprint_normal_dev(sc->sc_dev, "just checking: 1 + 2 = %d\n", faa_add(sc, 1, 2));
  871:     }
  872: 
  873:     static uint32_t
  874:     faa_add(struct faa_softc *sc, uint32_t a, uint32_t b)
  875:     {
  876:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_COMMAND, FAA_COMMAND_STORE_A);
  877:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_DATA, a);
  878:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_COMMAND, FAA_COMMAND_STORE_B);
  879:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_DATA, b);
  880:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_COMMAND, FAA_COMMAND_ADD);
  881:             return bus_space_read_4(sc->sc_regt, sc->sc_regh, FAA_RESULT);
  882:     }
  883: 
  884: ### Implementing addition using the hardware - running the example
  885: 
  886: -   Update the kernel binary and run it again
  887: 
  888: -   Check GXemul log
  889: 
  890: <!-- -->
  891: 
  892:     [ faa: COMMAND register (0x4) WRITE value 0x2 ]
  893:     [ faa: DATA register (0x8) WRITE value 0x1 ]
  894:     [ faa: COMMAND register (0x4) WRITE value 0x4 ]
  895:     [ faa: DATA register (0x8) WRITE value 0x2 ]
  896:     [ faa: COMMAND register (0x4) WRITE value 0x1 ]
  897:     [ faa: RESULT register (0xC) READ value 0x3 ]
  898: 
  899: -   Looks like it worked!
  900: 
  901: <!-- -->
  902: 
  903:     faa0 at pci0 dev 12 function 0: Fake Cards Advanced Addition Accelerator (rev. 0x01)
  904:     faa0: registers at 0x10110000
  905:     faa0: just checking: 1 + 2 = 3
  906: 
  907: ## Interacting with userspace
  908: 
  909: ### The kernel-user space interface
  910: 
  911: -   Now that the core functionality of the kernel driver is working, it
  912:     should be exposed to user space
  913: -   The interface between kernel driver and userspace can be designed in
  914:     many different ways
  915: -   The classic UNIX way of interfacing between the kernel and user
  916:     space is a device file
  917: -   Even when using device files there is no single interfacing method
  918:     that fits all use cases
  919: -   It’s up to the programmer to define the communication protocol
  920: 
  921: ### Device files
  922: 
  923: -   crw-r—– 1 root wheel 101, 1 Aug 12 21:53 /dev/file
  924: -   The kernel identifies which driver should service the request to
  925:     this file by using major and minor numbers (101 and 1 in the example
  926:     above)
  927: -   The major number identifies the driver
  928: -   The minor number usually identifies the driver instance, although
  929:     the driver is free to use it in any other way
  930: -   In NetBSD device files are created statically
  931:     -   By the MAKEDEV script during installation or boot
  932:     -   Manually by using the mknod utility
  933: 
  934: ### Operations on device files
  935: 
  936: -   [open(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)
  937:     and
  938:     [close(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)
  939: -   [read(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)
  940:     and
  941:     [write(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)
  942: -   [ioctl(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)
  943: -   [poll(2)](http://netbsd.gw.com/cgi-bin/man-cgi?poll++NetBSD-current)
  944: -   [mmap(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)
  945: -   and more…
  946: -   Any mix of the above system calls might be used to interface between
  947:     the kernel and user space
  948: -   We’ll implement an ioctl(2)-based communication mechanism
  949: 
  950: ### Adding cdevsw
  951: 
  952: -   cdevsw
  953: 
  954:     is used to decide which operation on the character device file calls
  955:     which driver function
  956: 
  957: -   Not all calls have to be implemented, although some device layers
  958:     define a set of calls that a driver must implement
  959: 
  960: -   For example disk drivers must implement open, close, read, write and
  961:     ioctl
  962: 
  963: -   src/sys/dev/pci/faa.c
  964: 
  965: <!-- -->
  966: 
  967:     dev_type_open(faaopen);
  968:     dev_type_close(faaclose);
  969:     dev_type_ioctl(faaioctl);
  970: 
  971:     const struct cdevsw faa_cdevsw = {
  972:             faaopen, faaclose, noread, nowrite, faaioctl,
  973:             nostop, notty, nopoll, nommap, nokqfilter, D_OTHER
  974:     };
  975: 
  976: ### Prototyping the cdevsw operations
  977: 
  978: -   The dev\_type\* macros are used to prototype the functions passed to
  979:     cdevsw
  980: -   Pass no followed by a function name to the appropriate cdevsw field
  981:     if it is not implemented
  982: -   There’s also bdevsw for block devices, but we won’t use it in this
  983:     example
  984: -   The last member of the cdevsw structure defines the device flags,
  985:     originally it was used to define the device type (still used for
  986:     disks, tape drives and ttys, for other devices pass D\_OTHER)
  987: 
  988: ### Implemeting the cdevsw operations - open / close
  989: 
  990: -   src/sys/dev/pci/faa.c
  991: 
  992: <!-- -->
  993: 
  994:     int
  995:     faaopen(dev_t dev, int flags, int mode, struct lwp *l)
  996:     {
  997:             struct faa_softc *sc;
  998:             sc = device_lookup_private(&faa_cd, minor(dev));
  999: 
 1000:             if (sc == NULL)
 1001:                     return ENXIO;
 1002:             if (sc->sc_flags & FAA_OPEN)
 1003:                     return EBUSY;
 1004: 
 1005:             sc->sc_flags |= FAA_OPEN;
 1006:             return 0;
 1007:     }
 1008:     int
 1009:     faaclose(dev_t dev, int flag, int mode, struct lwp *l)
 1010:     {
 1011:             struct faa_softc *sc;
 1012:             sc = device_lookup_private(&faa_cd, minor(dev));
 1013: 
 1014:             if (sc->sc_flags & FAA_OPEN)
 1015:                     sc->sc_flags =~ FAA_OPEN;
 1016: 
 1017:             return 0;
 1018:     }
 1019: 
 1020: ### Defining the ioctls
 1021: 
 1022: -   ioctl(2)
 1023: 
 1024:     can be used to call kernel-level functions and exchange data between
 1025:     the kernel and user space
 1026: 
 1027: -   The classic way of passing data is by using structures, their
 1028:     definitions are shared between the kernel and user space code
 1029: -   The driver might support more than one ioctl, the \_IO\* macros are
 1030:     used to define the operation and associated structure used to
 1031:     exchange data
 1032: 
 1033:     -   \_IO
 1034: 
 1035:         - just a kernel function call, no data exchange
 1036: 
 1037:     -   \_IOR
 1038: 
 1039:         - kernel function call and data pass from kernel to user space
 1040: 
 1041:     -   \_IOW
 1042: 
 1043:         - kernel function call and data pass from user space to kernel
 1044: 
 1045:     -   \_IOWR
 1046: 
 1047:         - kernel function call and data exchange in both directions
 1048: 
 1049:     -   \#define DRIVERIO\_IOCTLNAME \_IOXXX(group, ioctl\_number, data
 1050:         structure)
 1051: 
 1052: ## Using ioctls
 1053: 
 1054: ### Defining the ioctls
 1055: 
 1056: -   src/sys/dev/pci/faaio.h
 1057: 
 1058: <!-- -->
 1059: 
 1060:     #include <sys/ioccom.h>
 1061:                          
 1062:     #define FAAIO_ADD   _IOWR(0, 1, struct faaio_add)
 1063: 
 1064:     struct faaio_add {
 1065:         uint32_t a;
 1066:         uint32_t b;
 1067:         uint32_t *result;
 1068:     };
 1069: 
 1070: -   In the above example the ioctl group is not defined (0), but a
 1071:     single letter identifier could appear as first argument to \_IOWR
 1072: 
 1073: ### Implemeting the cdevsw operations - ioctl
 1074: 
 1075: -   src/sys/dev/pci/faa.c
 1076: 
 1077: <!-- -->
 1078: 
 1079:     int
 1080:     faaioctl(dev_t dev, u_long cmd, void *data, int flag, struct lwp *l)
 1081:     {
 1082:             struct faa_softc *sc = device_lookup_private(&faa_cd, minor(dev));
 1083:             int err;
 1084: 
 1085:             switch (cmd) {
 1086:             case FAAIO_ADD:
 1087:                     err = faaioctl_add(sc, (struct faaio_add *) data);
 1088:                     break;
 1089:             default:
 1090:                     err = EINVAL;
 1091:                     break;
 1092:             }
 1093:             return(err);
 1094:     }
 1095:     static int
 1096:     faaioctl_add(struct faa_softc *sc, struct faaio_add *data)
 1097:     {
 1098:             uint32_t result; int err;
 1099: 
 1100:             aprint_normal_dev(sc->sc_dev, "got ioctl with a %d, b %d\n",
 1101:                 data->a, data->b);
 1102: 
 1103:             result = faa_add(sc, data->a, data->b);
 1104:             err = copyout(&result, data->result, sizeof(uint32_t));
 1105:             return err;
 1106:     }
 1107: 
 1108: ### Using copyout to pass data to userspace
 1109: 
 1110: -   The copy(9) functions are used to copy kernel space data from/to
 1111:     user space
 1112: -   copyout(kernel\_address, user space\_address, size);
 1113: -   Actually on Cobalt we could just do data-\>result = faa\_add();
 1114:     instead of calling the copyout function, but that is a bad idea
 1115: -   Some architectures (such as sparc64) have totally separate kernel
 1116:     and user address spaces $ \implies $ user space addresses are
 1117:     meaningless in the kernel
 1118: 
 1119: ### Defining device major number
 1120: 
 1121: -   Device major numbers for hardware drivers are usually defined in a
 1122:     per-port manner
 1123: > It’s also possible to define a major in a machine-independent way in src/sys/conf/majors
 1124: -   src/sys/arch/$PORTNAME/conf/majors.$PORTNAME
 1125: -   src/sys/arch/cobalt/conf/majors.cobalt
 1126: -   The following defines a new character device file called /dev/faa\*
 1127:     with major number 101, but only if the faa driver is included in the
 1128:     kernel (last argument)
 1129: -   device-major faa char 101 faa
 1130: 
 1131: ### Creating the device node
 1132: 
 1133: -   The mknod utility can be used to create the device file manually
 1134: -   The driver name can be specified instead of the major number - it
 1135:     will be automatically resolved into the correct major number
 1136: -   mknod name [b | c] [major | driver] minor
 1137: -   mknod /dev/faa0 c faa 0
 1138: -   Created successfully
 1139: -   crw-r–r– 1 root wheel 101, 0 Oct 8 2012 /dev/faa0
 1140: 
 1141: ### An example user space program
 1142: 
 1143: -   The example program will open the device file and call ioctl(2) on
 1144:     it
 1145: -   As simple as possible, just to show how communication is done
 1146: -   Using ioctls from the user space
 1147:     -   Open the device file with O\_RDWR
 1148:     -   Call ioctl(2) with the operation number and structure as
 1149:         parameters
 1150: 
 1151: ### An example user space program - source
 1152: 
 1153:     void add(int, uint32_t, uint32_t);
 1154: 
 1155:     static const char* faa_device = "/dev/faa0";
 1156: 
 1157:     int
 1158:     main(int argc, char *argv[])
 1159:     {
 1160:             int devfd;
 1161: 
 1162:             if (argc != 3) {
 1163:                     printf("usage: %s a b\n", argv[0]);
 1164:                     return 1;
 1165:             }
 1166:             if ( (devfd = open(faa_device, O_RDWR)) == -1) {
 1167:                     perror("can't open device file");
 1168:                     return 1;
 1169:             }
 1170: 
 1171:             add(devfd, atoi(argv[1]), atoi(argv[2]));
 1172: 
 1173:             close(devfd);
 1174:             return 0;
 1175:     }
 1176: 
 1177: ### An example user space program - source
 1178: 
 1179:     void
 1180:     add(int devfd, uint32_t a, uint32_t b)
 1181:     {
 1182:             struct faaio_add faaio;
 1183:             uint32_t result = 0;
 1184: 
 1185:             faaio.result = &result;
 1186:             faaio.a = a;
 1187:             faaio.b = b;
 1188: 
 1189:             if (ioctl(devfd, FAAIO_ADD, &faaio) == -1) {
 1190:                     perror("ioctl failed");
 1191:             }
 1192:             printf("%d\n", result);
 1193:     }
 1194: 
 1195: ### An example user space program - running it
 1196: 
 1197:     # make
 1198:     cc -o aaa_add aaa_add.c
 1199:     # ./aaa_add 3 7
 1200:     faa0: got ioctl with a 3, b 7
 1201:     10
 1202: 
 1203: -   The program is successfully accessing the faa driver through the
 1204:     ioctl
 1205: -   The faa0:... line is a kernel message, normally only seen on the
 1206:     console terminal
 1207: 
 1208: ## A few tips
 1209: 
 1210: ### Avoiding common pitfalls
 1211: 
 1212: -   Always free resources allocated in the match or probe functions
 1213: -   Always use bus\_space methods, don’t access the hardware using a
 1214:     pointer dereference
 1215: -   If possible test on more than one hardware architecture, some bugs
 1216:     may surface
 1217: -   Don’t reinvent the wheel, try to use existing kernel frameworks as
 1218:     much as possible
 1219: -   Use copy(9) (or uiomove(9) or store(9)/fetch(9)) to move data
 1220:     between the kernel and user space
 1221: 
 1222: ### Basic driver debugging
 1223: 
 1224: -   Use aprint\_debug to print debug-level messages on console and log
 1225:     them (enabled by passing AB\_DEBUG from the boot loader)
 1226: -   Use the built-in DDB debugger
 1227:     -   Enabled by the kernel option DDB
 1228:     -   A kernel panic will start DDB if the DDB\_ONPANIC=1 kernel
 1229:         option is specified or the ddb.onpanic sysctl is set to 1.
 1230:     -   Run \# sysctl -w kern.panic\_now=1 to trigger a panic manually
 1231:         (DIAGNOSTIC option)
 1232: -   Remote debugging is possible on some ports
 1233:     -   With KGDB through the serial port
 1234:     -   With IPKDB through the network
 1235: 
 1236: ## Summary
 1237: 
 1238: ### Further reading
 1239: 
 1240: -   Documentation, articles:
 1241:     -   [A Machine-Independent DMA Framework for NetBSD, Jason R.
 1242:         Thorpe](http://www.netbsd.org/docs/kernel/bus_dma.pdf)
 1243:     -   [Writing Drivers for NetBSD, Jochen
 1244:         Kunz](ftp://ftp.netbsd.org/pub/NetBSD/misc/ddwg/NetBSD-driver_writing-1.0.1e.pdf)
 1245:     -   [NetBSD Documentation: Writing a pseudo
 1246:         device](http://www.netbsd.org/docs/kernel/pseudo/)
 1247:     -   [autoconf(9)](http://netbsd.gw.com/cgi-bin/man-cgi?autoconf+9+NetBSD-current),
 1248:         [bus\_space(9)](http://netbsd.gw.com/cgi-bin/man-cgi?bus_space+9+NetBSD-current)
 1249:         [bus\_dma(9)](http://netbsd.gw.com/cgi-bin/man-cgi?bus_dma+9+NetBSD-current)
 1250:         [driver(9)](http://netbsd.gw.com/cgi-bin/man-cgi?driver+9+NetBSD-current),
 1251:         [pci(9)](http://netbsd.gw.com/cgi-bin/man-cgi?pci+9+NetBSD-current)
 1252:         man pages, etc.
 1253: 
 1254: -   Example source code of drivers:
 1255:     -   tdvfb
 1256: 
 1257:         voodoofb are fairly good frame buffer driver examples with
 1258:         documentation publicly available.
 1259: 
 1260:     -   etsec
 1261: 
 1262:         is a nice example of a more complicated network interface driver
 1263: 
 1264: ### Get the source code
 1265: 
 1266: -   Download the source code and materials for this tutorial
 1267: -   <https://github.com/rkujawa/busspace-tutorial>
 1268: -   <https://github.com/rkujawa/gxemul-tutorial>
 1269: 
 1270: 
 1271: 
 1272: 
 1273: 
 1274: 
 1275: 

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb