Diff for /wikisrc/tutorials/bus_space_tutorial.mdwn between versions 1.2 and 1.13

version 1.2, 2013/06/23 14:10:19 version 1.13, 2016/03/17 11:38:49
Line 1 Line 1
 [allowframebreaks]  
   
 ### Table of Contents  
 [[!toc ]]  [[!toc ]]
   
 Introduction  ## Introduction
 ============  
   
 Why was this tutorial created?  
 ------------------------------  
   
 ### Why was this tutorial created?  ### Why was this tutorial created?
   
 -   Introductory-level documentation is scarce  -   Introductory-level documentation is scarce
   
 -   Writing device drivers is often considered black magic  -   Writing device drivers is often considered black magic
   -   Reading the man pages won't give you the big picture
 -   Reading the man pages won’t give you the big picture  
   
 -   BSD systems are always in need of new drivers  -   BSD systems are always in need of new drivers
   
 -   Device drivers are fun  -   Device drivers are fun
   
 What won’t be covered here?  ### What won't be covered here?
 ---------------------------  
   
 ### What won’t be covered here?  We don't have much time, so several ~~advanced~~ topics were omitted:
   
 We don’t have much time, so several ~~advanced~~ topics were omitted:  
   
 -   Interrupt handling  -   Interrupt handling
   
 -   Direct Memory Access and the bus\_dma framework  -   Direct Memory Access and the bus\_dma framework
   
 -   Power management  -   Power management
   
 -   Driver detachment  -   Driver detachment
   
 -   Drivers as kernel modules  -   Drivers as kernel modules
   
 -   Examples for buses other than PCI  -   Examples for buses other than PCI
   
 -   Pretty much everything else...  -   Pretty much everything else...
   
 However, once you finish this tutorial, you should be able to pursue  However, once you finish this tutorial, you should be able to pursue
 this knowledge yourself.  this knowledge yourself.
   
 What is a driver anyway?  
 ------------------------  
   
 ### What is a driver anyway?  ### What is a driver anyway?
   
 -   The interface between user space and hardware, implemented as a part  -   The interface between user space and hardware, implemented as a part
     of the kernel      of the kernel
   
 -   The NetBSD drivers are written mostly in C  -   The NetBSD drivers are written mostly in C
   
 -   Sometimes they have machine dependent assembler parts, but this is a  -   Sometimes they have machine dependent assembler parts, but this is a
     rare case      rare case
   
 What do you need to write a driver?  
 -----------------------------------  
   
 ### What do you need to write a driver?  ### What do you need to write a driver?
   
 -   C programming skills  -   C programming skills
   
 -   Hardware documentation (or the ability to reverse engineer the  -   Hardware documentation (or the ability to reverse engineer the
     hardware)      hardware)
   
 -   A reference driver implementation will help but is not essential  -   A reference driver implementation will help but is not essential
   
 -   A NetBSD installation and kernel source, or a cross-build  -   A NetBSD installation and kernel source, or a cross-build
     environment (the latter is usually preferred for development of      environment (the latter is usually preferred for development of
     drivers)      drivers)
   
 -   A lot of time, coffee and patience  -   A lot of time, coffee and patience
   
 ### Why is writing the device drivers considered difficult?  ### Why is writing the device drivers considered difficult?
   
 -   It’s not as difficult as you may expect, in fact during this  -   It's not as difficult as you may expect, in fact during this
     tutorial we’ll prove that it’s quite easy      tutorial we'll prove that it's quite easy
   
 -   You need to think on a very low level  -   You need to think on a very low level
   
     -   Good understanding of computer architecture is a must      -   Good understanding of computer architecture is a must
   -   Often documentation is the main problem - writing the driver is not
 -   Often documentation is the main problem – writing the driver is not      possible if you don't understand how the device works
     possible if you don’t understand how the device works  
   
     -   No access to documentation (uncooperative hardware vendors,      -   No access to documentation (uncooperative hardware vendors,
         vendors out of business)          vendors out of business)
   
     -   Documentation is incomplete or plain wrong      -   Documentation is incomplete or plain wrong
       -   Reverse engineering can solve these problems but it's a very
     -   Reverse engineering can solve these problems but it’s a very  
         time consuming process          time consuming process
   
 The NetBSD driver model  ## The NetBSD driver model
 =======================  
   
 The NetBSD kernel basics  
 ------------------------  
   
 ### The NetBSD kernel basics  ### The NetBSD kernel basics
   
 -   NetBSD has a classic monolithic UNIX-like kernel - all drivers are  -   NetBSD has a classic monolithic UNIX-like kernel - all drivers are
     running in the same address space      running in the same address space
   
 -   Thanks to the above, communication between drivers and other kernel  -   Thanks to the above, communication between drivers and other kernel
     layers is simple      layers is simple
   
 -   However, it also means that one badly written driver can affect the  -   However, it also means that one badly written driver can affect the
     whole kernel      whole kernel
   
 -   Numerous in-kernel frameworks standardise the way drivers are  -   Numerous in-kernel frameworks standardise the way drivers are
     written (bus\_space, autoconf, etc.)      written (bus\_space, autoconf, etc.)
   
 ### The NetBSD source directory structure  ### The NetBSD source directory structure
   
 -   We’ll only cover parts interesting for a device driver programmer  -   We'll only cover parts interesting for a device driver programmer
   
 -   src/sys/  -   src/sys/
   
     - kernel source directory      - kernel source directory
   
 -   src/sys/dev/  -   src/sys/dev/
   
     - machine-independent device drivers      - machine-independent device drivers
   
 -   src/sys/arch/  -   src/sys/arch/
   
     - port-specific or architecture-specific parts (such as the      - port-specific or architecture-specific parts (such as the
     low-level system initialisation procedures or machine-dependent      low-level system initialisation procedures or machine-dependent
     drivers)      drivers)
   -   src/sys/arch/$PORTNAME/conf/
 -   src/sys/arch/\$PORTNAME/conf/  
   
     - kernel configuration files for a given port      - kernel configuration files for a given port
   
 Kernel autoconfiguration framework  
 ----------------------------------  
   
 ### Kernel autoconfiguration framework - autoconf(9)  ### Kernel autoconfiguration framework - autoconf(9)
   
 -   Autoconfiguration is the process of matching hardware devices with  -   Autoconfiguration is the process of matching hardware devices with
     an appropriate device driver      an appropriate device driver
   
 -   The kernel message buffer (dmesg) contains information about  -   The kernel message buffer (dmesg) contains information about
     autoconfiguration of devices      autoconfiguration of devices
   
 -   driver0 at bus0: Foo hardware  -   driver0 at bus0: Foo hardware
   
     -   Instance 0 of the driver has attached to instance 0 of the      -   Instance 0 of the driver has attached to instance 0 of the
         particular bus          particular bus
   
     -   Such messages often carry additional bus-specific information      -   Such messages often carry additional bus-specific information
         about the exact location of the device (like the device and          about the exact location of the device (like the device and
         function number on the PCI bus)          function number on the PCI bus)
   
 -   driver0: some message  -   driver0: some message
   
     -   Additional information about the driver state or device      -   Additional information about the driver state or device
         configuration          configuration
   
Line 194  Kernel autoconfiguration framework Line 132  Kernel autoconfiguration framework
     wd0: <netbsd-cobalt.img>      wd0: <netbsd-cobalt.img>
     wd0: 750 MB, 1524 cyl, 16 head, 63 sec, 512 bytes/sect x 1536192 sectors      wd0: 750 MB, 1524 cyl, 16 head, 63 sec, 512 bytes/sect x 1536192 sectors
   
 ### Autoconfiguration as seen in the dmesg  
   
 ![image](img_cobaltdevices.png)  
   
 ### The bus\_space(9) framework  ### The bus\_space(9) framework
   
 -   “The goal of the bus\_space functions is to allow a single driver  -   "The goal of the bus\_space functions is to allow a single driver
     source file to manipulate a set of devices on different system      source file to manipulate a set of devices on different system
     architectures, and to allow a single driver object file to      architectures, and to allow a single driver object file to
     manipulate a set of devices on multiple bus types on a single      manipulate a set of devices on multiple bus types on a single
     architecture.”      architecture."
   
 -   Provides a set of functions implementing common operations on the  -   Provides a set of functions implementing common operations on the
     bus like mapping, reading, writing, copying, etc.      bus like mapping, reading, writing, copying, etc.
   
 -   The bus\_space(9) is implemented at the machine-dependent level  -   The bus\_space(9) is implemented at the machine-dependent level
     (typically it’s a part of architecture-specific code), but all      (typically it's a part of architecture-specific code), but all
     implementations present the same interface[^1]      implementations present the same interface
   > At least they should, some functions are missing on less popular ports
   
 ### Machine independent drivers  ### Machine independent drivers
   
 -   If possible drivers should work on any hardware platform  -   If possible drivers should work on any hardware platform
   
 -   High quality, machine-independent (MI) drivers are an important  -   High quality, machine-independent (MI) drivers are an important
     factor that adds to NetBSD portability      factor that adds to NetBSD portability
   
 -   Some drivers are completely MI, some have MD or bus dependent  -   Some drivers are completely MI, some have MD or bus dependent
     attachments and some are completely MD      attachments and some are completely MD
   
     -   A driver for a typical PCI card will be completely MI      -   A driver for a typical PCI card will be completely MI
   
     -   A driver for the components of a SoC will usually be completely      -   A driver for the components of a SoC will usually be completely
         MD          MD
   
 -   The bus\_space abstraction helps to achieve portability,  -   The bus\_space abstraction helps to achieve portability,
     transparently handling endianness issues and hiding bus      transparently handling endianness issues and hiding bus
     implementation details from the device driver      implementation details from the device driver
   
 -   Even if we have MI drivers, writing the drivers is always  -   Even if we have MI drivers, writing the drivers is always
     significant part of effort needed to port NetBSD to new hardware      significant part of effort needed to port NetBSD to new hardware
   
 Example driver from scratch  ## Example driver from scratch
 ===========================  
   
 Development environment  
 -----------------------  
   
 ### Development environment  ### Development environment
   
 -   Out of scope of this course, but very well documented  -   Out of scope of this course, but very well documented
   
 -   Cross compiling is an easy task with the build.sh script  -   Cross compiling is an easy task with the build.sh script
   
 -   Described in [Part V of the NetBSD  -   Described in [Part V of the NetBSD
     Guide](http://www.netbsd.org/docs/guide/en/part-compile.html)      Guide](http://www.netbsd.org/docs/guide/en/part-compile.html)
   
 -   Check out the NetBSD sources  -   Check out the NetBSD sources
   -   $ build.sh -m cobalt tools
 -   \$ build.sh -m cobalt tools  
   
     will build compiler, assembler, linker, etc. for cobalt port      will build compiler, assembler, linker, etc. for cobalt port
   -   $ build.sh -m cobalt kernel=GENERIC
 -   \$ build.sh -m cobalt kernel=GENERIC  
   
     will build the GENERIC kernel for cobalt      will build the GENERIC kernel for cobalt
   -   Call build.sh with a -u parameter to update (won't rebuilding
 -   Call build.sh with a -u parameter to update (won’t rebuilding  
     everything)      everything)
   
 -   build.sh  -   build.sh
   
     is calling nbconfig and nbmake tools, no magic involved      is calling nbconfig and nbmake tools, no magic involved
   
 Quick introduction to GXemul  
 ----------------------------  
   
 ### Quick introduction to GXemul  ### Quick introduction to GXemul
   
 -   A framework for full-system computer architecture emulation,  -   A framework for full-system computer architecture emulation,
     excellent for educational purposes      excellent for educational purposes
   
 -   Capable of emulating several real machines supported by NetBSD  -   Capable of emulating several real machines supported by NetBSD
   -   We'll emulate a [Cobalt](http://en.wikipedia.org/wiki/Cobalt_Qube),
 -   We’ll emulate a [Cobalt](http://en.wikipedia.org/wiki/Cobalt_Qube),  
     MIPS-based micro server with PCI bus      MIPS-based micro server with PCI bus
   -   I've modified GXemul and implemented an emulation of an additional
 -   I’ve modified GXemul and implemented an emulation of an additional  
     PCI device      PCI device
   
 -   It will be used to show (almost) a real-life example of the driver  -   It will be used to show (almost) a real-life example of the driver
     development process      development process
   
 Our hardware - a fake PCI card  
 ------------------------------  
   
 ### Our hardware - functional description  ### Our hardware - functional description
   
 -   Business applications often use arithmetic operations like addition  -   Business applications often use arithmetic operations like addition
   
 -   Fake Cards Inc. responded to market needs and created a new product,  -   Fake Cards Inc. responded to market needs and created a new product,
     Advanced Addition Accelerator      Advanced Addition Accelerator
   
 -   Pointy Haired Bosses will certainly buy it to accelerate their  -   Pointy Haired Bosses will certainly buy it to accelerate their
     business applications, so let’s create a driver for NetBSD!      business applications, so let's create a driver for NetBSD!
   
 ### Our hardware - technical details  ### Our hardware - technical details
   
 -   Overview  -   Overview
   
     -   Implemented as a PCI device      -   Implemented as a PCI device
   
     -   Arithmetic unit capable of addition of two numbers      -   Arithmetic unit capable of addition of two numbers
       -   Four registers in the PCI memory space
     -   Four[^2] registers in the PCI memory space  > Only three of these registers are of any importance for us at this moment
   
 -   PCI configuration space  -   PCI configuration space
   
     -   Identified by the PCI vendor ID 0xfabc and product ID 0x0001      -   Identified by the PCI vendor ID 0xfabc and product ID 0x0001
   
     -   Base Address Register 0x10 used to configure the engine address      -   Base Address Register 0x10 used to configure the engine address
   
     -   4 x 32-bit registers = 16 bytes      -   4 x 32-bit registers = 16 bytes
   
     -   Other configuration registers irrelevant      -   Other configuration registers irrelevant
   
 [faaop]  
   
 ### Our hardware - technical details (memory mapped register set)  ### Our hardware - technical details (memory mapped register set)
   
 -   Advanced Addition Acceleration registers  -   Advanced Addition Acceleration registers
   
    Register Name   Offset                          Description  [[!table data="""
   --------------- -------- -----------------------------------------------------------  Register Name   |Offset         |Description
       COMMAND       0x4           Register used to issue commands to the engine  COMMAND         |0x4            |Register used to issue commands to the engine
        DATA         0x8      Register used to load data to internal engine registers  DATA            |0x8            |Register used to load data to internal engine registers
       RESULT        0xC     Register used to store the result of arithmetic operation  RESULT          |0xC            |Register used to store the result of arithmetic operation
   """]]
   
 -   COMMAND register  -   COMMAND register
   
    Bit   R/W                                Description  [[!table data="""
   ----- ----- -----------------------------------------------------------------------  Bit     |R/W    |Description
     0     W    Execute ADD operation on values loaded into internal register A and B  0       |W      |Execute ADD operation on values loaded into internal register A and B
     1    R/W        Select internal register A for access through DATA register  1       |R/W    |Select internal register A for access through DATA register
     2    R/W        Select internal register B for access through DATA register  2       |R/W    |Select internal register B for access through DATA register
   """]]
   
 -   Selecting internal register A and B at the same time will lead to  -   Selecting internal register A and B at the same time will lead to
     undefined behaviour      undefined behaviour
Line 346  Our hardware - a fake PCI card Line 240  Our hardware - a fake PCI card
   
 -   DATA register  -   DATA register
   
    Bit    R/W                     Description  [[!table data="""
   ------ ----- --------------------------------------------------  Bit     |R/W    |Description
    0:31   R/W   Read/write the value in internal engine register  0:31    |R/W    |Read/write the value in internal engine register
   """]]
   
 -   RESULT register  -   RESULT register
   
    Bit    R/W                Description  [[!table data="""
   ------ ----- ----------------------------------------  Bit     |R/W    |Description
    0:31    R    Holds the result of last ADD operation  0:31    |R      |Holds the result of last ADD operation
   """]]
   
 ### Our hardware - technical details (operation algorithm)  ### Our hardware - technical details (operation algorithm)
   
 -   Select the internal register A for access (write 0x2 into COMMAND  -   Select the internal register A for access (write 0x2 into COMMAND
     register)      register)
   
 -   Write the first number into DATA register  -   Write the first number into DATA register
   
 -   Select the internal register B for access (write 0x4 into COMMAND  -   Select the internal register B for access (write 0x4 into COMMAND
     register)      register)
   
 -   Write the second number into DATA register  -   Write the second number into DATA register
   
 -   Issue the ADD operation (write 0x1 into COMMAND register)  -   Issue the ADD operation (write 0x1 into COMMAND register)
   
 -   Read the result from RESULT register  -   Read the result from RESULT register
   
 Adding a new driver to the NetBSD kernel  
 ----------------------------------------  
   
 ### Adding a new driver to the NetBSD kernel  ### Adding a new driver to the NetBSD kernel
   
 -   We’ll discuss the steps needed to add a new MI PCI device driver to  -   We'll discuss the steps needed to add a new MI PCI device driver to
     the NetBSD kernel      the NetBSD kernel
   
     -   Add the vendor and device ID to the database of PCI IDs      -   Add the vendor and device ID to the database of PCI IDs
   
     -   Create a set of the driver source files in      -   Create a set of the driver source files in
         src/sys/dev/\$BUSNAME/          src/sys/dev/$BUSNAME/
       -   Add the new driver to src/sys/dev/$BUSNAME/$BUSNAME.files file
     -   Add the new driver to src/sys/dev/\$BUSNAME/\$BUSNAME.files file      -   Add the new driver to DEVNAMES file
   > Required if you are NetBSD developer, optional otherwise.
     -   Add the new driver to DEVNAMES[^3] file  
   
 Matching the PCI device  
 -----------------------  
   
 ### Modifying the PCI device database  ### Modifying the PCI device database
   
Line 399  Matching the PCI device Line 281  Matching the PCI device
     not configured      not configured
   
 -   The kernel does not know anything about this vendor and device  -   The kernel does not know anything about this vendor and device
   
 -   Add it to the PCI device database - src/sys/dev/pci/pcidevs  -   Add it to the PCI device database - src/sys/dev/pci/pcidevs
   
 -   vendor VENDORNAME 0xVENDORID Long Vendor Name  -   vendor VENDORNAME 0xVENDORID Long Vendor Name
   
 -   product VENDORNAME PRODUCTNAME 0xPRODUCTID Long Product Name  -   product VENDORNAME PRODUCTNAME 0xPRODUCTID Long Product Name
   
 -   To regenerate pcidevs\*.h run awk -f devlist2h.awk pcidevs or  -   To regenerate pcidevs\*.h run awk -f devlist2h.awk pcidevs or
     Makefile.pcidevs if you’re on NetBSD      Makefile.pcidevs if you're on NetBSD
   
 ### Modifying the PCI device database - example  ### Modifying the PCI device database - example
   
Line 439  Matching the PCI device Line 317  Matching the PCI device
     not configured      not configured
   
 -   Now the kernel knows the vendor and product ID  -   Now the kernel knows the vendor and product ID
   -   But there's still no driver for this device
 -   But there’s still no driver for this device  
   
 ### Adding the new PCI driver  ### Adding the new PCI driver
   
 -   Choose a name - short, easy to remember, avoid numbers  -   Choose a name - short, easy to remember, avoid numbers
       -   faa looks like a good name, but you can choose any name you like
     -   faa  
   
         looks like a good name, but you can choose any name you like  
   
 -   Create a set of new files in src/sys/dev/pci  -   Create a set of new files in src/sys/dev/pci
   
     -   faa.c      -   faa.c
   
         - main driver code          - main driver code
   
     -   faareg.h      -   faareg.h
           - register definitions
         - register definitions[^4]  > Might not exist if the driver is only a simple passthrough from a specific bus to another MI driver.
   
     -   faavar.h      -   faavar.h
   
         - driver structures and functions used in other parts of the          - driver structures and functions used in other parts of the
         kernel[^5]          kernel
   > Omitted if not needed.
 -   Modify driver definitions  -   Modify driver definitions
   
     -   src/sys/dev/pci/files.pci      -   src/sys/dev/pci/files.pci
   
     -   src/sys/dev/DEVNAMES      -   src/sys/dev/DEVNAMES
   
 -   Configure the kernel to use the newly added driver -  -   Configure the kernel to use the newly added driver -
     src/sys/arch/\$PORTNAME/conf/GENERIC      src/sys/arch/$PORTNAME/conf/GENERIC
   
 ### Adding the new PCI driver - main driver  ### Adding the new PCI driver - main driver
   
 -   Kernel includes are at the beginning, followed by machine-specific  -   Kernel includes are at the beginning, followed by machine-specific
     and bus-specific includes      and bus-specific includes
   
 -   Should also include faareg.h and faavar.h files  -   Should also include faareg.h and faavar.h files
   
 -   A minimal driver needs just two functions  -   A minimal driver needs just two functions
   
     -   faa\_match      -   faa\_match
   
         (or faa\_probe for some buses)          (or faa\_probe for some buses)
   
     -   faa\_attach      -   faa\_attach
   
 -   The CFATTACH\_DECL\_NEW macro plugs the above functions into  -   The CFATTACH\_DECL\_NEW macro plugs the above functions into
     autoconf(9) mechanism      autoconf(9) mechanism
   
Line 496  Matching the PCI device Line 355  Matching the PCI device
   
 -   static int faa\_match(device\_t parent, cfdata\_t match, void  -   static int faa\_match(device\_t parent, cfdata\_t match, void
     \*aux);      \*aux);
   
     -   Check if the driver should attach to a given device (for example      -   Check if the driver should attach to a given device (for example
         in case of PCI bus, it will be used to check vendor and product          in case of PCI bus, it will be used to check vendor and product
         ID)          ID)
   
     -   parent      -   parent
           - pointer to parent's driver device structure
         - pointer to parent’s driver device structure  
   
     -   match      -   match
   
         - pointer to autoconf(9) details structure          - pointer to autoconf(9) details structure
   
     -   aux      -   aux
   
         - despite the name the most important argument, usually contains          - despite the name the most important argument, usually contains
         bus-specific structure describing device details          bus-specific structure describing device details
   
 -   static void faa\_attach(device\_t parent, device\_t self, void  -   static void faa\_attach(device\_t parent, device\_t self, void
     \*aux);      \*aux);
   
     -   Attach the driver to a given device      -   Attach the driver to a given device
   
     -   parent      -   parent
   
         - same as with match function          - same as with match function
   
     -   self      -   self
           - pointer to driver's device structure
         - pointer to driver’s device structure  
   
     -   aux      -   aux
   
         - same as with match function          - same as with match function
   
 -   See definitions of these functions in the  -   See definitions of these functions in the
     [driver(9)](http://netbsd.gw.com/cgi-bin/man-cgi?driver+9+NetBSD-current)      [driver(9)](http://netbsd.gw.com/cgi-bin/man-cgi?driver+9+NetBSD-current)
     man page.      man page.
   
 ### Adding the new PCI driver - main driver cont’d  ### Adding the new PCI driver - main driver cont'd
   
 -   CFATTACH\_DECL\_NEW(faa, sizeof(struct faa\_softc), faa\_match,  -   CFATTACH\_DECL\_NEW(faa, sizeof(struct faa\_softc), faa\_match,
     faa\_attach, NULL, NULL);      faa\_attach, NULL, NULL);
   
     -   driver name      -   driver name
       -   size of softc structure containing state of driver's instance
     -   size of softc structure containing state of driver’s instance  
   
     -   match/probe function      -   match/probe function
   
     -   attach function      -   attach function
   
     -   detach function      -   detach function
   
     -   activate function      -   activate function
   -   The "\_NEW" name is unfortunate
 -   The “\_NEW” name is unfortunate  
   
 -   Pass NULL for unimplemented functions  -   Pass NULL for unimplemented functions
   -   We won't cover detach and activate now, as they are not needed for a
 -   We won’t cover detach and activate now, as they are not needed for a  
     simple driver      simple driver
   
 ### Adding the new PCI driver - main driver example  ### Adding the new PCI driver - main driver example
Line 566  Matching the PCI device Line 401  Matching the PCI device
 <!-- -->  <!-- -->
   
     #include <sys/cdefs.h>      #include <sys/cdefs.h>
     __KERNEL_RCSID(0, "$NetBSD: bus_space_tutorial.mdwn,v 1.1 2013/06/23 13:59:13 mspo Exp $");      __KERNEL_RCSID(0, "$NetBSD: bus_space_tutorial.mdwn,v 1.12 2013/06/29 12:41:13 ryoon Exp $");
     #include <sys/param.h>      #include <sys/param.h>
     #include <sys/device.h>      #include <sys/device.h>
     #include <dev/pci/pcivar.h>      #include <dev/pci/pcivar.h>
Line 639  Matching the PCI device Line 474  Matching the PCI device
 ### Adding the new PCI driver - registering the driver  ### Adding the new PCI driver - registering the driver
   
 -   See config(5)  -   See config(5)
   
 -   src/sys/dev/pci/files.pci  -   src/sys/dev/pci/files.pci
   
 <!-- -->  <!-- -->
Line 677  Matching the PCI device Line 511  Matching the PCI device
   
 -   The above definition means that an instance of faa may be attached  -   The above definition means that an instance of faa may be attached
     to any PCI bus, any device, any function      to any PCI bus, any device, any function
   
 -   The exact position of the rule in the configuration file is not  -   The exact position of the rule in the configuration file is not
     important in this case      important in this case
   
 -   See  -   See
     [config(5)](http://netbsd.gw.com/cgi-bin/man-cgi?config+5+NetBSD-current)      [config(5)](http://netbsd.gw.com/cgi-bin/man-cgi?config+5+NetBSD-current)
     for a description of the device definition language      for a description of the device definition language
Line 688  Matching the PCI device Line 520  Matching the PCI device
 ### Adding the new PCI driver - example  ### Adding the new PCI driver - example
   
 -   The driver should compile now  -   The driver should compile now
   -   The driver's match function will check if the driver is able to work
 -   The driver’s match function will check if the driver is able to work  
     with a given device      with a given device
   
 -   Since it is not implemented, the kernel will not attach the driver  -   Since it is not implemented, the kernel will not attach the driver
   
 ### Matching the PCI device  ### Matching the PCI device
   
 -   Modify the faa\_match function to match the specified PCI device  -   Modify the faa\_match function to match the specified PCI device
   
 -   Use PCI\_VENDOR and PCI\_PRODUCT macros to obtain the IDs  -   Use PCI\_VENDOR and PCI\_PRODUCT macros to obtain the IDs
   
 <!-- -->  <!-- -->
Line 714  Matching the PCI device Line 543  Matching the PCI device
             return 0;              return 0;
     }      }
   
 Attaching to the PCI device  
 ---------------------------  
   
 ### Attaching to the PCI device  ### Attaching to the PCI device
   
     faa0 at pci0 dev 12 function 0      faa0 at pci0 dev 12 function 0
   
 -   The driver has successfully matched and attached to the PCI device  -   The driver has successfully matched and attached to the PCI device
     but still is not doing anything useful      but still is not doing anything useful
   -   Let's fill the attach function and actually program the hardware
 -   Let’s fill the attach function and actually program the hardware  
   
 Variable types used with bus\_space  
 -----------------------------------  
   
 ### Variable types used with bus\_space  ### Variable types used with bus\_space
   
 -   bus\_space\_tag\_t  -   bus\_space\_tag\_t
   
     – type used to describe a particular bus, usually passed to the      - type used to describe a particular bus, usually passed to the
     driver from MI bus structures         driver from MI bus structures
   
 -   bus\_space\_handle\_t  -   bus\_space\_handle\_t
   
     – used to describe a mapped range of bus space, usually created with      - used to describe a mapped range of bus space, usually created with
     the bus\_space\_map() function         the bus\_space\_map() function
   
 -   bus\_addr\_t  -   bus\_addr\_t
   
     – address on the bus      - address on the bus
   
 -   bus\_size\_t  -   bus\_size\_t
   
     – an amount of space on the bus      - an amount of space on the bus
   
 -   Contents of these types are MD, so avoid modifying from within the  -   Contents of these types are MD, so avoid modifying from within the
     driver[^6]      driver
   > although you'll often have to use bus\_size\_t
 Mapping the hardware resources  
 ------------------------------  
   
 ### Why do we need to “map” the resources?  
   
 -   “The bus space must be mapped before it can be used, and should be  ### Why do we need to "map" the resources?
     unmapped when it is no longer needed”  
   
 -   It’s a machine-dependent process but it’s also conveniently hidden  -   "The bus space must be mapped before it can be used, and should be
       unmapped when it is no longer needed"
   -   It's a machine-dependent process but it's also conveniently hidden
     from the programmer by the bus\_space framework      from the programmer by the bus\_space framework
   
 ### Mapping the hardware resources  ### Mapping the hardware resources
Line 774  Mapping the hardware resources Line 593  Mapping the hardware resources
   
 -   bus\_space\_map  -   bus\_space\_map
   
     creates a mapping from the physical address to a kernel virtual      - creates a mapping from the physical address to a kernel virtual
     address      address
   
 -   space  -   space
   
     – represents the bus on which the mapping will be created      - represents the bus on which the mapping will be created
   
 -   address  -   address
   
     – typically represents the physical address for which a mapping will      - typically represents the physical address for which a mapping will
     be created         be created
   
 -   size  -   size
   
     – describes the amount of bus space to be mapped      - describes the amount of bus space to be mapped
   
 -   handlep  -   handlep
   
     – pointer to mapped space (filled after successful mapping)      - pointer to mapped space (filled after successful mapping)
   
 -   Separate space and address  -   Separate space and address
   
Line 808  Mapping the hardware resources Line 627  Mapping the hardware resources
   
 -   pci\_mapreg\_map  -   pci\_mapreg\_map
   
     creates mapping from physical address present in specified BAR      - creates mapping from physical address present in specified BAR
     register to kernel virtual address      register to kernel virtual address
   
 -   pa  -   pa
   
     – struct describing PCI attachment details (passed through aux)      - struct describing PCI attachment details (passed through aux)
   
 -   reg  -   reg
   
     – BAR register number      - BAR register number
   
 -   type  -   type
   
     – Select mapping type (I/O, memory)      - Select mapping type (I/O, memory)
   
 -   busflags  -   busflags
   
     – Passed to bus\_space\_map flags argument      - Passed to bbus_space_map flags argument
   
 -   tagp  -   tagp
   
     – pointer to bus\_space\_tag      - pointer to bus_space_tag
   
 -   handlep  -   handlep
   
     – pointer to a mapped space      - pointer to a mapped space
   
 -   basep  -   basep
   
     – address of a mapped space      - address of a mapped space
   
 -   sizep  -   sizep
   
     – size of mapped space (equivalent to BAR size)      - size of mapped space (equivalent to BAR size)
   
 -   The last four parameters are filled after successful mapping  -   The last four parameters are filled after successful mapping
   
Line 891  Mapping the hardware resources Line 710  Mapping the hardware resources
             aprint_normal_dev(sc->sc_dev, "regs at 0x%08x\n", (uint32_t) sc->sc_reg_pa);              aprint_normal_dev(sc->sc_dev, "regs at 0x%08x\n", (uint32_t) sc->sc_reg_pa);
     }      }
   
 Accessing the hardware registers  
 --------------------------------  
   
 ### Accessing the hardware registers  ### Accessing the hardware registers
   
 -   The bus\_space\_read\_ and bus\_space\_write\_ functions are basic  -   The bus\_space\_read\_ and bus\_space\_write\_ functions are basic
Line 924  Accessing the hardware registers Line 740  Accessing the hardware registers
   
 ### Variants of bus\_space\_read and bus\_space\_write  ### Variants of bus\_space\_read and bus\_space\_write
   
     Data       Read function         Write function  [[!table data="""
   -------- --------------------- ----------------------  Data    |Read function          |Write function
    8-bit    bus\_space\_read\_1   bus\_space\_write\_1  8-bitb  |bus_space_read_1       |bus_space_write_1
    16-bit   bus\_space\_read\_2   bus\_space\_write\_2  16-bit  |bus_space_read_2       |bus_space_write_2
    32-bit   bus\_space\_read\_4   bus\_space\_write\_4  32-bit  |bus_space_read_4       |bus_space_write_4
    64-bit   bus\_space\_read\_8   bus\_space\_write\_8  64-bit  |bus_space_read_8       |bus_space_write_8
   """]]
   
 -   There are many more variants of read and write functions and they  -   There are many more variants of read and write functions and they
     are useful in certain situations, see the      are useful in certain situations, see the
Line 1083  Accessing the hardware registers Line 900  Accessing the hardware registers
     faa0: registers at 0x10110000      faa0: registers at 0x10110000
     faa0: just checking: 1 + 2 = 3      faa0: just checking: 1 + 2 = 3
   
 Interacting with userspace  ## Interacting with userspace
 ==========================  
   
 Device files  
 ------------  
   
 ### The kernel-user space interface  ### The kernel-user space interface
   
 -   Now that the core functionality of the kernel driver is working, it  -   Now that the core functionality of the kernel driver is working, it
     should be exposed to user space      should be exposed to user space
   
 -   The interface between kernel driver and userspace can be designed in  -   The interface between kernel driver and userspace can be designed in
     many different ways      many different ways
   
 -   The classic UNIX way of interfacing between the kernel and user  -   The classic UNIX way of interfacing between the kernel and user
     space is a device file      space is a device file
   
 -   Even when using device files there is no single interfacing method  -   Even when using device files there is no single interfacing method
     that fits all use cases      that fits all use cases
   -   It's up to the programmer to define the communication protocol
 -   It’s up to the programmer to define the communication protocol  
   
 ### Device files  ### Device files
   
 -   crw-r—– 1 root wheel 101, 1 Aug 12 21:53 /dev/file  -   crw-r-- 1 root wheel 101, 1 Aug 12 21:53 /dev/file
   
 -   The kernel identifies which driver should service the request to  -   The kernel identifies which driver should service the request to
     this file by using major and minor numbers (101 and 1 in the example      this file by using major and minor numbers (101 and 1 in the example
     above)      above)
   
 -   The major number identifies the driver  -   The major number identifies the driver
   
 -   The minor number usually identifies the driver instance, although  -   The minor number usually identifies the driver instance, although
     the driver is free to use it in any other way      the driver is free to use it in any other way
   
 -   In NetBSD device files are created statically  -   In NetBSD device files are created statically
   
     -   By the MAKEDEV script during installation or boot      -   By the MAKEDEV script during installation or boot
   
     -   Manually by using the mknod utility      -   Manually by using the mknod utility
   
 ### Operations on device files  ### Operations on device files
Line 1129  Device files Line 932  Device files
 -   [open(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)  -   [open(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)
     and      and
     [close(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)      [close(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)
   
 -   [read(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)  -   [read(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)
     and      and
     [write(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)      [write(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)
   
 -   [ioctl(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)  -   [ioctl(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)
   
 -   [poll(2)](http://netbsd.gw.com/cgi-bin/man-cgi?poll++NetBSD-current)  -   [poll(2)](http://netbsd.gw.com/cgi-bin/man-cgi?poll++NetBSD-current)
   
 -   [mmap(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)  -   [mmap(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)
   -   and more...
 -   and more…  
   
 -   Any mix of the above system calls might be used to interface between  -   Any mix of the above system calls might be used to interface between
     the kernel and user space      the kernel and user space
   -   We'll implement an ioctl(2)-based communication mechanism
 -   We’ll implement an ioctl(2)-based communication mechanism  
   
 ### Adding cdevsw  ### Adding cdevsw
   
Line 1177  Device files Line 973  Device files
   
 -   The dev\_type\* macros are used to prototype the functions passed to  -   The dev\_type\* macros are used to prototype the functions passed to
     cdevsw      cdevsw
   
 -   Pass no followed by a function name to the appropriate cdevsw field  -   Pass no followed by a function name to the appropriate cdevsw field
     if it is not implemented      if it is not implemented
   -   There's also bdevsw for block devices, but we won't use it in this
 -   There’s also bdevsw for block devices, but we won’t use it in this  
     example      example
   
 -   The last member of the cdevsw structure defines the device flags,  -   The last member of the cdevsw structure defines the device flags,
     originally it was used to define the device type (still used for      originally it was used to define the device type (still used for
     disks, tape drives and ttys, for other devices pass D\_OTHER)      disks, tape drives and ttys, for other devices pass D\_OTHER)
Line 1229  Device files Line 1022  Device files
   
 -   The classic way of passing data is by using structures, their  -   The classic way of passing data is by using structures, their
     definitions are shared between the kernel and user space code      definitions are shared between the kernel and user space code
   
 -   The driver might support more than one ioctl, the \_IO\* macros are  -   The driver might support more than one ioctl, the \_IO\* macros are
     used to define the operation and associated structure used to      used to define the operation and associated structure used to
     exchange data      exchange data
Line 1253  Device files Line 1045  Device files
     -   \#define DRIVERIO\_IOCTLNAME \_IOXXX(group, ioctl\_number, data      -   \#define DRIVERIO\_IOCTLNAME \_IOXXX(group, ioctl\_number, data
         structure)          structure)
   
 Using ioctls  ## Using ioctls
 ------------  
   
 ### Defining the ioctls  ### Defining the ioctls
   
Line 1314  Using ioctls Line 1105  Using ioctls
   
 -   The copy(9) functions are used to copy kernel space data from/to  -   The copy(9) functions are used to copy kernel space data from/to
     user space      user space
   
 -   copyout(kernel\_address, user space\_address, size);  -   copyout(kernel\_address, user space\_address, size);
   
 -   Actually on Cobalt we could just do data-\>result = faa\_add();  -   Actually on Cobalt we could just do data-\>result = faa\_add();
     instead of calling the copyout function, but that is a bad idea      instead of calling the copyout function, but that is a bad idea
   
 -   Some architectures (such as sparc64) have totally separate kernel  -   Some architectures (such as sparc64) have totally separate kernel
     and user address spaces $ \implies $ user space addresses are      and user address spaces $ \implies $ user space addresses are
     meaningless in the kernel      meaningless in the kernel
Line 1327  Using ioctls Line 1115  Using ioctls
 ### Defining device major number  ### Defining device major number
   
 -   Device major numbers for hardware drivers are usually defined in a  -   Device major numbers for hardware drivers are usually defined in a
     per-port manner[^7]      per-port manner
   > It's also possible to define a major in a machine-independent way in src/sys/conf/majors
 -   src/sys/arch/\$PORTNAME/conf/majors.\$PORTNAME  -   src/sys/arch/$PORTNAME/conf/majors.$PORTNAME
   
 -   src/sys/arch/cobalt/conf/majors.cobalt  -   src/sys/arch/cobalt/conf/majors.cobalt
   
 -   The following defines a new character device file called /dev/faa\*  -   The following defines a new character device file called /dev/faa\*
     with major number 101, but only if the faa driver is included in the      with major number 101, but only if the faa driver is included in the
     kernel (last argument)      kernel (last argument)
   
 -   device-major faa char 101 faa  -   device-major faa char 101 faa
   
 ### Creating the device node  ### Creating the device node
   
 -   The mknod utility can be used to create the device file manually  -   The mknod utility can be used to create the device file manually
   
 -   The driver name can be specified instead of the major number - it  -   The driver name can be specified instead of the major number - it
     will be automatically resolved into the correct major number      will be automatically resolved into the correct major number
   
 -   mknod name [b | c] [major | driver] minor  -   mknod name [b | c] [major | driver] minor
   
 -   mknod /dev/faa0 c faa 0  -   mknod /dev/faa0 c faa 0
   
 -   Created successfully  -   Created successfully
   -   crw-r-r- 1 root wheel 101, 0 Oct 8 2012 /dev/faa0
 -   crw-r–r– 1 root wheel 101, 0 Oct 8 2012 /dev/faa0  
   
 An example user space program  
 -----------------------------  
   
 ### An example user space program  ### An example user space program
   
 -   The example program will open the device file and call ioctl(2) on  -   The example program will open the device file and call ioctl(2) on
     it      it
   
 -   As simple as possible, just to show how communication is done  -   As simple as possible, just to show how communication is done
   
 -   Using ioctls from the user space  -   Using ioctls from the user space
   
     -   Open the device file with O\_RDWR      -   Open the device file with O\_RDWR
   
     -   Call ioctl(2) with the operation number and structure as      -   Call ioctl(2) with the operation number and structure as
         parameters          parameters
   
Line 1425  An example user space program Line 1198  An example user space program
   
 -   The program is successfully accessing the faa driver through the  -   The program is successfully accessing the faa driver through the
     ioctl      ioctl
   
 -   The faa0:... line is a kernel message, normally only seen on the  -   The faa0:... line is a kernel message, normally only seen on the
     console terminal      console terminal
   
 A few tips  ## A few tips
 ==========  
   
 Avoiding common pitfalls  
 ------------------------  
   
 ### Avoiding common pitfalls  ### Avoiding common pitfalls
   
 -   Always free resources allocated in the match or probe functions  -   Always free resources allocated in the match or probe functions
   -   Always use bus\_space methods, don't access the hardware using a
 -   Always use bus\_space methods, don’t access the hardware using a  
     pointer dereference      pointer dereference
   
 -   If possible test on more than one hardware architecture, some bugs  -   If possible test on more than one hardware architecture, some bugs
     may surface      may surface
   -   Don't reinvent the wheel, try to use existing kernel frameworks as
 -   Don’t reinvent the wheel, try to use existing kernel frameworks as  
     much as possible      much as possible
   
 -   Use copy(9) (or uiomove(9) or store(9)/fetch(9)) to move data  -   Use copy(9) (or uiomove(9) or store(9)/fetch(9)) to move data
     between the kernel and user space      between the kernel and user space
   
 Basic driver debugging  
 ----------------------  
   
 ### Basic driver debugging  ### Basic driver debugging
   
 -   Use aprint\_debug to print debug-level messages on console and log  -   Use aprint\_debug to print debug-level messages on console and log
     them (enabled by passing AB\_DEBUG from the boot loader)      them (enabled by passing AB\_DEBUG from the boot loader)
   
 -   Use the built-in DDB debugger  -   Use the built-in DDB debugger
   
     -   Enabled by the kernel option DDB      -   Enabled by the kernel option DDB
   
     -   A kernel panic will start DDB if the DDB\_ONPANIC=1 kernel      -   A kernel panic will start DDB if the DDB\_ONPANIC=1 kernel
         option is specified or the ddb.onpanic sysctl is set to 1.          option is specified or the ddb.onpanic sysctl is set to 1.
   
     -   Run \# sysctl -w kern.panic\_now=1 to trigger a panic manually      -   Run \# sysctl -w kern.panic\_now=1 to trigger a panic manually
         (DIAGNOSTIC option)          (DIAGNOSTIC option)
   
 -   Remote debugging is possible on some ports  -   Remote debugging is possible on some ports
   
     -   With KGDB through the serial port      -   With KGDB through the serial port
   
     -   With IPKDB through the network      -   With IPKDB through the network
   
 Summary  ## Summary
 =======  
   
 The end  
 -------  
   
 ### Further reading  ### Further reading
   
 -   Documentation, articles:  -   Documentation, articles:
   
     -   [A Machine-Independent DMA Framework for NetBSD, Jason R.      -   [A Machine-Independent DMA Framework for NetBSD, Jason R.
         Thorpe](http://www.netbsd.org/docs/kernel/bus_dma.pdf)          Thorpe](http://www.netbsd.org/docs/kernel/bus_dma.pdf)
   
     -   [Writing Drivers for NetBSD, Jochen      -   [Writing Drivers for NetBSD, Jochen
         Kunz](ftp://ftp.netbsd.org/pub/NetBSD/misc/ddwg/NetBSD-driver_writing-1.0.1e.pdf)          Kunz](ftp://ftp.netbsd.org/pub/NetBSD/misc/ddwg/NetBSD-driver_writing-1.0.1e.pdf)
   
     -   [NetBSD Documentation: Writing a pseudo      -   [NetBSD Documentation: Writing a pseudo
         device](http://www.netbsd.org/docs/kernel/pseudo/)          device](http://www.netbsd.org/docs/kernel/pseudo/)
   
     -   [autoconf(9)](http://netbsd.gw.com/cgi-bin/man-cgi?autoconf+9+NetBSD-current),      -   [autoconf(9)](http://netbsd.gw.com/cgi-bin/man-cgi?autoconf+9+NetBSD-current),
         [bus\_space(9)](http://netbsd.gw.com/cgi-bin/man-cgi?bus_space+9+NetBSD-current)          [bus\_space(9)](http://netbsd.gw.com/cgi-bin/man-cgi?bus_space+9+NetBSD-current)
         [bus\_dma(9)](http://netbsd.gw.com/cgi-bin/man-cgi?bus_dma+9+NetBSD-current)          [bus\_dma(9)](http://netbsd.gw.com/cgi-bin/man-cgi?bus_dma+9+NetBSD-current)
Line 1502  The end Line 1248  The end
         man pages, etc.          man pages, etc.
   
 -   Example source code of drivers:  -   Example source code of drivers:
   
     -   tdvfb      -   tdvfb
   
         , voodoofb are fairly good frame buffer driver examples with          voodoofb are fairly good frame buffer driver examples with
         documentation publicly available.          documentation publicly available.
   
     -   etsec      -   etsec
Line 1515  The end Line 1260  The end
 ### Get the source code  ### Get the source code
   
 -   Download the source code and materials for this tutorial  -   Download the source code and materials for this tutorial
   
 -   <https://github.com/rkujawa/busspace-tutorial>  -   <https://github.com/rkujawa/busspace-tutorial>
   
 -   <https://github.com/rkujawa/gxemul-tutorial>  -   <https://github.com/rkujawa/gxemul-tutorial>
   
 ### Questions?  
   
 -   Do you have any questions?  
   
 ### The End…  
   
 ![image](NetBSD.png)  
   
 Thank you!  
   
 [^1]: At least they should, some functions are missing on less popular  
     ports  
   
 [^2]: Only three of these registers are of any importance for us at this  
     moment  
   
 [^3]: Required if you are NetBSD developer, optional otherwise.  
   
 [^4]: Might not exist if the driver is only a simple passthrough from a  
     specific bus to another MI driver.  
   
 [^5]: Omitted if not needed.  
   
 [^6]: although you’ll often have to use bus\_size\_t  
   
 [^7]: It’s also possible to define a major in a machine-independent way  
     in src/sys/conf/majors  

Removed from v.1.2  
changed lines
  Added in v.1.13


CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb