Annotation of wikisrc/tutorials/bus_space_tutorial.mdwn, revision 1.2

1.1       mspo        1: [allowframebreaks]
                      2: 
                      3: ### Table of Contents
1.2     ! mspo        4: [[!toc ]]
1.1       mspo        5: 
                      6: Introduction
                      7: ============
                      8: 
                      9: Why was this tutorial created?
                     10: ------------------------------
                     11: 
                     12: ### Why was this tutorial created?
                     13: 
                     14: -   Introductory-level documentation is scarce
                     15: 
                     16: -   Writing device drivers is often considered black magic
                     17: 
                     18: -   Reading the man pages won’t give you the big picture
                     19: 
                     20: -   BSD systems are always in need of new drivers
                     21: 
                     22: -   Device drivers are fun
                     23: 
                     24: What won’t be covered here?
                     25: ---------------------------
                     26: 
                     27: ### What won’t be covered here?
                     28: 
                     29: We don’t have much time, so several ~~advanced~~ topics were omitted:
                     30: 
                     31: -   Interrupt handling
                     32: 
                     33: -   Direct Memory Access and the bus\_dma framework
                     34: 
                     35: -   Power management
                     36: 
                     37: -   Driver detachment
                     38: 
                     39: -   Drivers as kernel modules
                     40: 
                     41: -   Examples for buses other than PCI
                     42: 
                     43: -   Pretty much everything else...
                     44: 
                     45: However, once you finish this tutorial, you should be able to pursue
                     46: this knowledge yourself.
                     47: 
                     48: What is a driver anyway?
                     49: ------------------------
                     50: 
                     51: ### What is a driver anyway?
                     52: 
                     53: -   The interface between user space and hardware, implemented as a part
                     54:     of the kernel
                     55: 
                     56: -   The NetBSD drivers are written mostly in C
                     57: 
                     58: -   Sometimes they have machine dependent assembler parts, but this is a
                     59:     rare case
                     60: 
                     61: What do you need to write a driver?
                     62: -----------------------------------
                     63: 
                     64: ### What do you need to write a driver?
                     65: 
                     66: -   C programming skills
                     67: 
                     68: -   Hardware documentation (or the ability to reverse engineer the
                     69:     hardware)
                     70: 
                     71: -   A reference driver implementation will help but is not essential
                     72: 
                     73: -   A NetBSD installation and kernel source, or a cross-build
                     74:     environment (the latter is usually preferred for development of
                     75:     drivers)
                     76: 
                     77: -   A lot of time, coffee and patience
                     78: 
                     79: ### Why is writing the device drivers considered difficult?
                     80: 
                     81: -   It’s not as difficult as you may expect, in fact during this
                     82:     tutorial we’ll prove that it’s quite easy
                     83: 
                     84: -   You need to think on a very low level
                     85: 
                     86:     -   Good understanding of computer architecture is a must
                     87: 
                     88: -   Often documentation is the main problem – writing the driver is not
                     89:     possible if you don’t understand how the device works
                     90: 
                     91:     -   No access to documentation (uncooperative hardware vendors,
                     92:         vendors out of business)
                     93: 
                     94:     -   Documentation is incomplete or plain wrong
                     95: 
                     96:     -   Reverse engineering can solve these problems but it’s a very
                     97:         time consuming process
                     98: 
                     99: The NetBSD driver model
                    100: =======================
                    101: 
                    102: The NetBSD kernel basics
                    103: ------------------------
                    104: 
                    105: ### The NetBSD kernel basics
                    106: 
                    107: -   NetBSD has a classic monolithic UNIX-like kernel - all drivers are
                    108:     running in the same address space
                    109: 
                    110: -   Thanks to the above, communication between drivers and other kernel
                    111:     layers is simple
                    112: 
                    113: -   However, it also means that one badly written driver can affect the
                    114:     whole kernel
                    115: 
                    116: -   Numerous in-kernel frameworks standardise the way drivers are
                    117:     written (bus\_space, autoconf, etc.)
                    118: 
                    119: ### The NetBSD source directory structure
                    120: 
                    121: -   We’ll only cover parts interesting for a device driver programmer
                    122: 
                    123: -   src/sys/
                    124: 
                    125:     - kernel source directory
                    126: 
                    127: -   src/sys/dev/
                    128: 
                    129:     - machine-independent device drivers
                    130: 
                    131: -   src/sys/arch/
                    132: 
                    133:     - port-specific or architecture-specific parts (such as the
                    134:     low-level system initialisation procedures or machine-dependent
                    135:     drivers)
                    136: 
                    137: -   src/sys/arch/\$PORTNAME/conf/
                    138: 
                    139:     - kernel configuration files for a given port
                    140: 
                    141: Kernel autoconfiguration framework
                    142: ----------------------------------
                    143: 
                    144: ### Kernel autoconfiguration framework - autoconf(9)
                    145: 
                    146: -   Autoconfiguration is the process of matching hardware devices with
                    147:     an appropriate device driver
                    148: 
                    149: -   The kernel message buffer (dmesg) contains information about
                    150:     autoconfiguration of devices
                    151: 
                    152: -   driver0 at bus0: Foo hardware
                    153: 
                    154:     -   Instance 0 of the driver has attached to instance 0 of the
                    155:         particular bus
                    156: 
                    157:     -   Such messages often carry additional bus-specific information
                    158:         about the exact location of the device (like the device and
                    159:         function number on the PCI bus)
                    160: 
                    161: -   driver0: some message
                    162: 
                    163:     -   Additional information about the driver state or device
                    164:         configuration
                    165: 
                    166: ### Autoconfiguration as seen in the dmesg
                    167: 
                    168:     NetBSD 6.99.12 (GENERIC) #7: Fri Oct  5 18:43:21 CEST 2012
                    169:             rkujawa@saiko.local:/Users/rkujawa/netbsd-eurobsdcon2012/src/sys/arch/cobalt/compile/obj/GENERIC
                    170:     Cobalt Qube 2
                    171:     total memory = 32768 KB
                    172:     avail memory = 27380 KB
                    173:     mainbus0 (root)
                    174:     com0 at mainbus0 addr 0x1c800000 level 3: ns16550a, working fifo
                    175:     com0: console
                    176:     cpu0 at mainbus0: QED RM5200 CPU (0x28a0) Rev. 10.0 with built-in FPU Rev. 1.0
                    177:     cpu0: 48 TLB entries, 256MB max page size
                    178:     cpu0: 32KB/32B 2-way set-associative L1 instruction cache
                    179:     cpu0: 32KB/32B 2-way set-associative write-back L1 data cache
                    180:     mcclock0 at mainbus0 addr 0x10000070: mc146818 compatible time-of-day clock
                    181:     panel0 at mainbus0 addr 0x1f000000
                    182:     gt0 at mainbus0 addr 0x14000000
                    183:     pci0 at gt0
                    184:     pchb0 at pci0 dev 0 function 0: Galileo GT-64011 System Controller, rev 1
                    185:     pcib0 at pci0 dev 9 function 0
                    186:     pcib0: VIA Technologies VT82C586 PCI-ISA Bridge, rev 57
                    187:     viaide0 at pci0 dev 9 function 1
                    188:     viaide0: VIA Technologies VT82C586 (Apollo VP) ATA33 controller
                    189:     viaide0: primary channel interrupting at irq 14
                    190:     atabus0 at viaide0 channel 0
                    191:     viaide0: secondary channel interrupting at irq 15
                    192:     atabus1 at viaide0 channel 1
                    193:     wd0 at atabus0 drive 0
                    194:     wd0: <netbsd-cobalt.img>
                    195:     wd0: 750 MB, 1524 cyl, 16 head, 63 sec, 512 bytes/sect x 1536192 sectors
                    196: 
                    197: ### Autoconfiguration as seen in the dmesg
                    198: 
                    199: ![image](img_cobaltdevices.png)
                    200: 
                    201: ### The bus\_space(9) framework
                    202: 
                    203: -   “The goal of the bus\_space functions is to allow a single driver
                    204:     source file to manipulate a set of devices on different system
                    205:     architectures, and to allow a single driver object file to
                    206:     manipulate a set of devices on multiple bus types on a single
                    207:     architecture.”
                    208: 
                    209: -   Provides a set of functions implementing common operations on the
                    210:     bus like mapping, reading, writing, copying, etc.
                    211: 
                    212: -   The bus\_space(9) is implemented at the machine-dependent level
                    213:     (typically it’s a part of architecture-specific code), but all
                    214:     implementations present the same interface[^1]
                    215: 
                    216: ### Machine independent drivers
                    217: 
                    218: -   If possible drivers should work on any hardware platform
                    219: 
                    220: -   High quality, machine-independent (MI) drivers are an important
                    221:     factor that adds to NetBSD portability
                    222: 
                    223: -   Some drivers are completely MI, some have MD or bus dependent
                    224:     attachments and some are completely MD
                    225: 
                    226:     -   A driver for a typical PCI card will be completely MI
                    227: 
                    228:     -   A driver for the components of a SoC will usually be completely
                    229:         MD
                    230: 
                    231: -   The bus\_space abstraction helps to achieve portability,
                    232:     transparently handling endianness issues and hiding bus
                    233:     implementation details from the device driver
                    234: 
                    235: -   Even if we have MI drivers, writing the drivers is always
                    236:     significant part of effort needed to port NetBSD to new hardware
                    237: 
                    238: Example driver from scratch
                    239: ===========================
                    240: 
                    241: Development environment
                    242: -----------------------
                    243: 
                    244: ### Development environment
                    245: 
                    246: -   Out of scope of this course, but very well documented
                    247: 
                    248: -   Cross compiling is an easy task with the build.sh script
                    249: 
                    250: -   Described in [Part V of the NetBSD
                    251:     Guide](http://www.netbsd.org/docs/guide/en/part-compile.html)
                    252: 
                    253: -   Check out the NetBSD sources
                    254: 
                    255: -   \$ build.sh -m cobalt tools
                    256: 
                    257:     will build compiler, assembler, linker, etc. for cobalt port
                    258: 
                    259: -   \$ build.sh -m cobalt kernel=GENERIC
                    260: 
                    261:     will build the GENERIC kernel for cobalt
                    262: 
                    263: -   Call build.sh with a -u parameter to update (won’t rebuilding
                    264:     everything)
                    265: 
                    266: -   build.sh
                    267: 
                    268:     is calling nbconfig and nbmake tools, no magic involved
                    269: 
                    270: Quick introduction to GXemul
                    271: ----------------------------
                    272: 
                    273: ### Quick introduction to GXemul
                    274: 
                    275: -   A framework for full-system computer architecture emulation,
                    276:     excellent for educational purposes
                    277: 
                    278: -   Capable of emulating several real machines supported by NetBSD
                    279: 
                    280: -   We’ll emulate a [Cobalt](http://en.wikipedia.org/wiki/Cobalt_Qube),
                    281:     MIPS-based micro server with PCI bus
                    282: 
                    283: -   I’ve modified GXemul and implemented an emulation of an additional
                    284:     PCI device
                    285: 
                    286: -   It will be used to show (almost) a real-life example of the driver
                    287:     development process
                    288: 
                    289: Our hardware - a fake PCI card
                    290: ------------------------------
                    291: 
                    292: ### Our hardware - functional description
                    293: 
                    294: -   Business applications often use arithmetic operations like addition
                    295: 
                    296: -   Fake Cards Inc. responded to market needs and created a new product,
                    297:     Advanced Addition Accelerator
                    298: 
                    299: -   Pointy Haired Bosses will certainly buy it to accelerate their
                    300:     business applications, so let’s create a driver for NetBSD!
                    301: 
                    302: ### Our hardware - technical details
                    303: 
                    304: -   Overview
                    305: 
                    306:     -   Implemented as a PCI device
                    307: 
                    308:     -   Arithmetic unit capable of addition of two numbers
                    309: 
                    310:     -   Four[^2] registers in the PCI memory space
                    311: 
                    312: -   PCI configuration space
                    313: 
                    314:     -   Identified by the PCI vendor ID 0xfabc and product ID 0x0001
                    315: 
                    316:     -   Base Address Register 0x10 used to configure the engine address
                    317: 
                    318:     -   4 x 32-bit registers = 16 bytes
                    319: 
                    320:     -   Other configuration registers irrelevant
                    321: 
                    322: [faaop]
                    323: 
                    324: ### Our hardware - technical details (memory mapped register set)
                    325: 
                    326: -   Advanced Addition Acceleration registers
                    327: 
                    328:    Register Name   Offset                          Description
                    329:   --------------- -------- -----------------------------------------------------------
                    330:       COMMAND       0x4           Register used to issue commands to the engine
                    331:        DATA         0x8      Register used to load data to internal engine registers
                    332:       RESULT        0xC     Register used to store the result of arithmetic operation
                    333: 
                    334: -   COMMAND register
                    335: 
                    336:    Bit   R/W                                Description
                    337:   ----- ----- -----------------------------------------------------------------------
                    338:     0     W    Execute ADD operation on values loaded into internal register A and B
                    339:     1    R/W        Select internal register A for access through DATA register
                    340:     2    R/W        Select internal register B for access through DATA register
                    341: 
                    342: -   Selecting internal register A and B at the same time will lead to
                    343:     undefined behaviour
                    344: 
                    345: ### Our hardware - technical details (memory mapped register set)
                    346: 
                    347: -   DATA register
                    348: 
                    349:    Bit    R/W                     Description
                    350:   ------ ----- --------------------------------------------------
                    351:    0:31   R/W   Read/write the value in internal engine register
                    352: 
                    353: -   RESULT register
                    354: 
                    355:    Bit    R/W                Description
                    356:   ------ ----- ----------------------------------------
                    357:    0:31    R    Holds the result of last ADD operation
                    358: 
                    359: ### Our hardware - technical details (operation algorithm)
                    360: 
                    361: -   Select the internal register A for access (write 0x2 into COMMAND
                    362:     register)
                    363: 
                    364: -   Write the first number into DATA register
                    365: 
                    366: -   Select the internal register B for access (write 0x4 into COMMAND
                    367:     register)
                    368: 
                    369: -   Write the second number into DATA register
                    370: 
                    371: -   Issue the ADD operation (write 0x1 into COMMAND register)
                    372: 
                    373: -   Read the result from RESULT register
                    374: 
                    375: Adding a new driver to the NetBSD kernel
                    376: ----------------------------------------
                    377: 
                    378: ### Adding a new driver to the NetBSD kernel
                    379: 
                    380: -   We’ll discuss the steps needed to add a new MI PCI device driver to
                    381:     the NetBSD kernel
                    382: 
                    383:     -   Add the vendor and device ID to the database of PCI IDs
                    384: 
                    385:     -   Create a set of the driver source files in
                    386:         src/sys/dev/\$BUSNAME/
                    387: 
                    388:     -   Add the new driver to src/sys/dev/\$BUSNAME/\$BUSNAME.files file
                    389: 
                    390:     -   Add the new driver to DEVNAMES[^3] file
                    391: 
                    392: Matching the PCI device
                    393: -----------------------
                    394: 
                    395: ### Modifying the PCI device database
                    396: 
                    397:     unmatched vendor 0xfabc product 0x0001 (Co-processor 
                    398:     processor, revision 0x01) at pci0 dev 12 function 0 
                    399:     not configured
                    400: 
                    401: -   The kernel does not know anything about this vendor and device
                    402: 
                    403: -   Add it to the PCI device database - src/sys/dev/pci/pcidevs
                    404: 
                    405: -   vendor VENDORNAME 0xVENDORID Long Vendor Name
                    406: 
                    407: -   product VENDORNAME PRODUCTNAME 0xPRODUCTID Long Product Name
                    408: 
                    409: -   To regenerate pcidevs\*.h run awk -f devlist2h.awk pcidevs or
                    410:     Makefile.pcidevs if you’re on NetBSD
                    411: 
                    412: ### Modifying the PCI device database - example
                    413: 
                    414:     --- pcidevs 29 Sep 2012 10:26:14 -0000  1.1139
                    415:     +++ pcidevs 5 Oct 2012 08:52:59 -0000
                    416:     @@ -669,6 +669,7 @@
                    417:      vendor CHRYSALIS   0xcafe  Chrysalis-ITS
                    418:      vendor MIDDLE_DIGITAL  0xdeaf  Middle Digital
                    419:      vendor ARC     0xedd8  ARC Logic
                    420:     +vendor FAKECARDS   0xfabc  Fake Cards
                    421:      vendor INVALID     0xffff  INVALID VENDOR ID
                    422:      
                    423:      /*
                    424:     @@ -2120,6 +2121,9 @@
                    425:      /* Eumitcom products */
                    426:      product EUMITCOM WL11000P  0x1100  WL11000P PCI WaveLAN/IEEE 802.11
                    427:      
                    428:     +/* FakeCards products */
                    429:     +product FAKECARDS AAA      0x0001  Advanced Addition Accelerator
                    430:     +
                    431:      /* O2 Micro */
                    432:      product O2MICRO 00F7       0x00f7  Integrated OHCI IEEE 1394 Host Controller
                    433:      product O2MICRO OZ6729     0x6729  OZ6729 PCI-PCMCIA Bridge
                    434: 
                    435: ### Modifying the PCI device database - example
                    436: 
                    437:     Fake Cards Advanced Addition Accelerator (Co-processor 
                    438:     processor, revision 0x01) at pci0 dev 12 function 0 
                    439:     not configured
                    440: 
                    441: -   Now the kernel knows the vendor and product ID
                    442: 
                    443: -   But there’s still no driver for this device
                    444: 
                    445: ### Adding the new PCI driver
                    446: 
                    447: -   Choose a name - short, easy to remember, avoid numbers
                    448: 
                    449:     -   faa
                    450: 
                    451:         looks like a good name, but you can choose any name you like
                    452: 
                    453: -   Create a set of new files in src/sys/dev/pci
                    454: 
                    455:     -   faa.c
                    456: 
                    457:         - main driver code
                    458: 
                    459:     -   faareg.h
                    460: 
                    461:         - register definitions[^4]
                    462: 
                    463:     -   faavar.h
                    464: 
                    465:         - driver structures and functions used in other parts of the
                    466:         kernel[^5]
                    467: 
                    468: -   Modify driver definitions
                    469: 
                    470:     -   src/sys/dev/pci/files.pci
                    471: 
                    472:     -   src/sys/dev/DEVNAMES
                    473: 
                    474: -   Configure the kernel to use the newly added driver -
                    475:     src/sys/arch/\$PORTNAME/conf/GENERIC
                    476: 
                    477: ### Adding the new PCI driver - main driver
                    478: 
                    479: -   Kernel includes are at the beginning, followed by machine-specific
                    480:     and bus-specific includes
                    481: 
                    482: -   Should also include faareg.h and faavar.h files
                    483: 
                    484: -   A minimal driver needs just two functions
                    485: 
                    486:     -   faa\_match
                    487: 
                    488:         (or faa\_probe for some buses)
                    489: 
                    490:     -   faa\_attach
                    491: 
                    492: -   The CFATTACH\_DECL\_NEW macro plugs the above functions into
                    493:     autoconf(9) mechanism
                    494: 
                    495: ### Adding the new PCI driver - main driver
                    496: 
                    497: -   static int faa\_match(device\_t parent, cfdata\_t match, void
                    498:     \*aux);
                    499: 
                    500:     -   Check if the driver should attach to a given device (for example
                    501:         in case of PCI bus, it will be used to check vendor and product
                    502:         ID)
                    503: 
                    504:     -   parent
                    505: 
                    506:         - pointer to parent’s driver device structure
                    507: 
                    508:     -   match
                    509: 
                    510:         - pointer to autoconf(9) details structure
                    511: 
                    512:     -   aux
                    513: 
                    514:         - despite the name the most important argument, usually contains
                    515:         bus-specific structure describing device details
                    516: 
                    517: -   static void faa\_attach(device\_t parent, device\_t self, void
                    518:     \*aux);
                    519: 
                    520:     -   Attach the driver to a given device
                    521: 
                    522:     -   parent
                    523: 
                    524:         - same as with match function
                    525: 
                    526:     -   self
                    527: 
                    528:         - pointer to driver’s device structure
                    529: 
                    530:     -   aux
                    531: 
                    532:         - same as with match function
                    533: 
                    534: -   See definitions of these functions in the
                    535:     [driver(9)](http://netbsd.gw.com/cgi-bin/man-cgi?driver+9+NetBSD-current)
                    536:     man page.
                    537: 
                    538: ### Adding the new PCI driver - main driver cont’d
                    539: 
                    540: -   CFATTACH\_DECL\_NEW(faa, sizeof(struct faa\_softc), faa\_match,
                    541:     faa\_attach, NULL, NULL);
                    542: 
                    543:     -   driver name
                    544: 
                    545:     -   size of softc structure containing state of driver’s instance
                    546: 
                    547:     -   match/probe function
                    548: 
                    549:     -   attach function
                    550: 
                    551:     -   detach function
                    552: 
                    553:     -   activate function
                    554: 
                    555: -   The “\_NEW” name is unfortunate
                    556: 
                    557: -   Pass NULL for unimplemented functions
                    558: 
                    559: -   We won’t cover detach and activate now, as they are not needed for a
                    560:     simple driver
                    561: 
                    562: ### Adding the new PCI driver - main driver example
                    563: 
                    564: -   src/sys/dev/pci/faa.c
                    565: 
                    566: <!-- -->
                    567: 
                    568:     #include <sys/cdefs.h>
1.2     ! mspo      569:     __KERNEL_RCSID(0, "$NetBSD: bus_space_tutorial.mdwn,v 1.1 2013/06/23 13:59:13 mspo Exp $");
1.1       mspo      570:     #include <sys/param.h>
                    571:     #include <sys/device.h>
                    572:     #include <dev/pci/pcivar.h>
                    573:     #include <dev/pci/pcidevs.h>
                    574:     #include <dev/pci/faareg.h>
                    575:     #include <dev/pci/faavar.h>
                    576: 
                    577:     static int      faa_match(device_t, cfdata_t, void *);
                    578:     static void     faa_attach(device_t, device_t, void *);
                    579: 
                    580:     CFATTACH_DECL_NEW(faa, sizeof(struct faa_softc),
                    581:         faa_match, faa_attach, NULL, NULL);
                    582: 
                    583:     static int
                    584:     faa_match(device_t parent, cfdata_t match, void *aux)
                    585:     {
                    586:             return 0;
                    587:     }
                    588: 
                    589:     static void
                    590:     faa_attach(device_t parent, device_t self, void *aux)
                    591:     { 
                    592:     }
                    593: 
                    594: ### Adding the new PCI driver - auxiliary includes
                    595: 
                    596: -   src/sys/dev/pci/faareg.h
                    597: 
                    598: <!-- -->
                    599: 
                    600:     #ifndef FAAREG_H
                    601:     #define FAAREG_H
                    602:     /* 
                    603:      * Registers are defined using preprocessor:
                    604:      * #define FAA_REGNAME  0x0
                    605:      * We'll add them later, let's leave it empty for now.
                    606:      */
                    607:     #endif /* FAAREG_H */
                    608: 
                    609: -   src/sys/dev/pci/faavar.h
                    610: 
                    611: <!-- -->
                    612: 
                    613:     #ifndef FAAVAR_H
                    614:     #define FAAVAR_H
                    615: 
                    616:     /* sc_dev is an absolute minimum, we'll add more later */
                    617:     struct faa_softc {
                    618:             device_t sc_dev;
                    619:     };
                    620:     #endif /* FAAVAR_H */
                    621: 
                    622: ### Adding the new PCI driver - registering the driver (courtesy)
                    623: 
                    624: -   src/sys/dev/DEVNAMES
                    625: 
                    626: <!-- -->
                    627: 
                    628:     --- DEVNAMES    1 Sep 2012 11:19:58 -0000   1.279
                    629:     +++ DEVNAMES    6 Oct 2012 19:59:06 -0000
                    630:     @@ -436,6 +436,7 @@
                    631:      ex         MI
                    632:      exphy          MI
                    633:      ezload         MI      Attribute
                    634:     +faa            MI
                    635:      fb         luna68k
                    636:      fb         news68k
                    637:      fb         newsmips
                    638: 
                    639: ### Adding the new PCI driver - registering the driver
                    640: 
                    641: -   See config(5)
                    642: 
                    643: -   src/sys/dev/pci/files.pci
                    644: 
                    645: <!-- -->
                    646: 
                    647:     --- pci/files.pci   2 Aug 2012 00:17:44 -0000   1.360
                    648:     +++ pci/files.pci   6 Oct 2012 19:59:10 -0000
                    649:     @@ -1122,3 +1122,9 @@
                    650:      device tdvfb: wsemuldisplaydev, rasops8, vcons, videomode
                    651:      attach tdvfb at pci
                    652:      file   dev/pci/tdvfb.c     tdvfb   
                    653:     +
                    654:     +# FakeCards Advanced Addition Accelerator
                    655:     +device faa
                    656:     +attach faa at pci
                    657:     +file   dev/pci/faa.c       faa 
                    658:     +
                    659: 
                    660: ### Adding the new PCI driver to the kernel configuration
                    661: 
                    662: -   src/sys/arch/cobalt/conf/GENERIC
                    663: 
                    664: <!-- -->
                    665: 
                    666:     --- GENERIC 10 Mar 2012 21:51:50 -0000  1.134
                    667:     +++ GENERIC 6 Oct 2012 20:12:37 -0000
                    668:     @@ -302,6 +302,9 @@
                    669:      #fms*      at pci? dev ? function ?    # Forte Media FM801
                    670:      #sv*       at pci? dev ? function ?    # S3 SonicVibes
                    671:      
                    672:     +# Fake Cards Advanced Addition Accelerator
                    673:     +faa*       at pci? dev ? function ?
                    674:     +
                    675:      # Audio support
                    676:      #audio*        at audiobus?
                    677: 
                    678: -   The above definition means that an instance of faa may be attached
                    679:     to any PCI bus, any device, any function
                    680: 
                    681: -   The exact position of the rule in the configuration file is not
                    682:     important in this case
                    683: 
                    684: -   See
                    685:     [config(5)](http://netbsd.gw.com/cgi-bin/man-cgi?config+5+NetBSD-current)
                    686:     for a description of the device definition language
                    687: 
                    688: ### Adding the new PCI driver - example
                    689: 
                    690: -   The driver should compile now
                    691: 
                    692: -   The driver’s match function will check if the driver is able to work
                    693:     with a given device
                    694: 
                    695: -   Since it is not implemented, the kernel will not attach the driver
                    696: 
                    697: ### Matching the PCI device
                    698: 
                    699: -   Modify the faa\_match function to match the specified PCI device
                    700: 
                    701: -   Use PCI\_VENDOR and PCI\_PRODUCT macros to obtain the IDs
                    702: 
                    703: <!-- -->
                    704: 
                    705:     static int
                    706:     faa_match(device_t parent, cfdata_t match, void *aux)
                    707:     {
                    708:             const struct pci_attach_args *pa = (const struct pci_attach_args *)aux;
                    709: 
                    710:             if ((PCI_VENDOR(pa->pa_id) == PCI_VENDOR_FAKECARDS) 
                    711:                 && (PCI_PRODUCT(pa->pa_id) == PCI_PRODUCT_FAKECARDS_AAA))
                    712:                     return 1;
                    713: 
                    714:             return 0;
                    715:     }
                    716: 
                    717: Attaching to the PCI device
                    718: ---------------------------
                    719: 
                    720: ### Attaching to the PCI device
                    721: 
                    722:     faa0 at pci0 dev 12 function 0
                    723: 
                    724: -   The driver has successfully matched and attached to the PCI device
                    725:     but still is not doing anything useful
                    726: 
                    727: -   Let’s fill the attach function and actually program the hardware
                    728: 
                    729: Variable types used with bus\_space
                    730: -----------------------------------
                    731: 
                    732: ### Variable types used with bus\_space
                    733: 
                    734: -   bus\_space\_tag\_t
                    735: 
                    736:     – type used to describe a particular bus, usually passed to the
                    737:     driver from MI bus structures
                    738: 
                    739: -   bus\_space\_handle\_t
                    740: 
                    741:     – used to describe a mapped range of bus space, usually created with
                    742:     the bus\_space\_map() function
                    743: 
                    744: -   bus\_addr\_t
                    745: 
                    746:     – address on the bus
                    747: 
                    748: -   bus\_size\_t
                    749: 
                    750:     – an amount of space on the bus
                    751: 
                    752: -   Contents of these types are MD, so avoid modifying from within the
                    753:     driver[^6]
                    754: 
                    755: Mapping the hardware resources
                    756: ------------------------------
                    757: 
                    758: ### Why do we need to “map” the resources?
                    759: 
                    760: -   “The bus space must be mapped before it can be used, and should be
                    761:     unmapped when it is no longer needed”
                    762: 
                    763: -   It’s a machine-dependent process but it’s also conveniently hidden
                    764:     from the programmer by the bus\_space framework
                    765: 
                    766: ### Mapping the hardware resources
                    767: 
                    768: -   The generic bus\_space(9) way to map space
                    769: 
                    770: <!-- -->
                    771: 
                    772:     bus_space_map(bus_space_tag_t space, bus_addr_t address, 
                    773:     bus_size_t size, int flags, bus_space_handle_t  *handlep);
                    774: 
                    775: -   bus\_space\_map
                    776: 
                    777:     creates a mapping from the physical address to a kernel virtual
                    778:     address
                    779: 
                    780: -   space
                    781: 
                    782:     – represents the bus on which the mapping will be created
                    783: 
                    784: -   address
                    785: 
                    786:     – typically represents the physical address for which a mapping will
                    787:     be created
                    788: 
                    789: -   size
                    790: 
                    791:     – describes the amount of bus space to be mapped
                    792: 
                    793: -   handlep
                    794: 
                    795:     – pointer to mapped space (filled after successful mapping)
                    796: 
                    797: -   Separate space and address
                    798: 
                    799: ### Mapping the hardware resources
                    800: 
                    801: -   The PCI-specific way to map space
                    802: 
                    803: <!-- -->
                    804: 
                    805:     pci_mapreg_map(const struct pci_attach_args *pa, int reg, pcireg_t type, 
                    806:     int busflags, bus_space_tag_t *tagp, bus_space_handle_t *handlep, 
                    807:     bus_addr_t *basep, bus_size_t *sizep);
                    808: 
                    809: -   pci\_mapreg\_map
                    810: 
                    811:     creates mapping from physical address present in specified BAR
                    812:     register to kernel virtual address
                    813: 
                    814: -   pa
                    815: 
                    816:     – struct describing PCI attachment details (passed through aux)
                    817: 
                    818: -   reg
                    819: 
                    820:     – BAR register number
                    821: 
                    822: -   type
                    823: 
                    824:     – Select mapping type (I/O, memory)
                    825: 
                    826: -   busflags
                    827: 
                    828:     – Passed to bus\_space\_map flags argument
                    829: 
                    830: -   tagp
                    831: 
                    832:     – pointer to bus\_space\_tag
                    833: 
                    834: -   handlep
                    835: 
                    836:     – pointer to a mapped space
                    837: 
                    838: -   basep
                    839: 
                    840:     – address of a mapped space
                    841: 
                    842: -   sizep
                    843: 
                    844:     – size of mapped space (equivalent to BAR size)
                    845: 
                    846: -   The last four parameters are filled after successful mapping
                    847: 
                    848: ### Mapping the registers using BAR - adding auxiliary includes
                    849: 
                    850: -   src/sys/dev/pci/faareg.h
                    851: 
                    852: <!-- -->
                    853: 
                    854:     #define FAA_MMREG_BAR   0x10
                    855: 
                    856: -   src/sys/dev/pci/faavar.h
                    857: 
                    858: <!-- -->
                    859: 
                    860:     struct faa_softc {
                    861:             device_t sc_dev;
                    862: 
                    863:             bus_space_tag_t sc_regt;
                    864:             bus_space_handle_t sc_regh;
                    865:             bus_addr_t sc_reg_pa;
                    866: 
                    867:     };
                    868: 
                    869: ### Mapping the registers using BAR - main driver code
                    870: 
                    871: -   src/sys/dev/pci/faa.c
                    872: 
                    873: <!-- -->
                    874: 
                    875:     static void
                    876:     faa_attach(device_t parent, device_t self, void *aux)
                    877:     {
                    878:             struct faa_softc *sc = device_private(self);
                    879:             const struct pci_attach_args *pa = aux;
                    880: 
                    881:             sc->sc_dev = self;
                    882: 
                    883:             pci_aprint_devinfo(pa, NULL);
                    884: 
                    885:             if (pci_mapreg_map(pa, FAA_MMREG_BAR, PCI_MAPREG_TYPE_MEM, 0, 
                    886:                 &sc->sc_regt, &sc->sc_regh, &sc->sc_reg_pa, 0) != 0 ) {
                    887:                 aprint_error_dev(sc->sc_dev, "can't map the BAR\n");
                    888:                 return;
                    889:             }
                    890: 
                    891:             aprint_normal_dev(sc->sc_dev, "regs at 0x%08x\n", (uint32_t) sc->sc_reg_pa);
                    892:     }
                    893: 
                    894: Accessing the hardware registers
                    895: --------------------------------
                    896: 
                    897: ### Accessing the hardware registers
                    898: 
                    899: -   The bus\_space\_read\_ and bus\_space\_write\_ functions are basic
                    900:     methods of reading and writing the hardware registers
                    901: 
                    902: -   uintX\_t bus\_space\_read\_X(bus\_space\_tag\_t space,
                    903:     bus\_space\_handle\_t handle, bus\_size\_t offset);
                    904: 
                    905: -   void bus\_space\_write\_X(bus\_space\_tag\_t space,
                    906:     bus\_space\_handle\_t handle, bus\_size\_t offset, uintX\_t value);
                    907: 
                    908:     -   space
                    909: 
                    910:         - tag describing the bus
                    911: 
                    912:     -   handle
                    913: 
                    914:         - describes the exact location on the bus where read/write
                    915:         should occur, this handle is obtained by bus\_space\_map
                    916: 
                    917:     -   offset
                    918: 
                    919:         - offset from handle location
                    920: 
                    921:     -   The read function returns the data read from the specified
                    922:         location, while write has an argument value which should be
                    923:         filled with data to be written
                    924: 
                    925: ### Variants of bus\_space\_read and bus\_space\_write
                    926: 
                    927:     Data       Read function         Write function
                    928:   -------- --------------------- ----------------------
                    929:    8-bit    bus\_space\_read\_1   bus\_space\_write\_1
                    930:    16-bit   bus\_space\_read\_2   bus\_space\_write\_2
                    931:    32-bit   bus\_space\_read\_4   bus\_space\_write\_4
                    932:    64-bit   bus\_space\_read\_8   bus\_space\_write\_8
                    933: 
                    934: -   There are many more variants of read and write functions and they
                    935:     are useful in certain situations, see the
                    936:     [bus\_space(9)](http://netbsd.gw.com/cgi-bin/man-cgi?bus_space++NetBSD-current)
                    937:     man page
                    938: 
                    939: ### Accessing the hardware registers - example
                    940: 
                    941: -   Create a function that will write a value into the DATA register of
                    942:     our device, then read it back and check if the value is the same as
                    943:     written
                    944: 
                    945: -   Define the DATA register in the driver
                    946: 
                    947: -   src/sys/dev/pci/faareg.h
                    948: 
                    949: <!-- -->
                    950: 
                    951:     #define FAA_DATA                0x8
                    952:     #define FAA_COMMAND             0x4
                    953:     #define FAA_COMMAND_STORE_A         __BIT(1)
                    954: 
                    955: -   Define the new function in main driver code
                    956: 
                    957: -   static bool faa\_check(struct faa\_softc \*sc);
                    958: 
                    959: ### Accessing the hardware registers - example
                    960: 
                    961: -   src/sys/dev/pci/faa.c
                    962: 
                    963: <!-- -->
                    964: 
                    965:     static void
                    966:     faa_attach(device_t parent, device_t self, void *aux)
                    967:     {
                    968:        /* ... */
                    969:        if (!faa_check(sc)) {
                    970:             aprint_error_dev(sc->sc_dev, "hardware not responding\n");
                    971:             return;
                    972:        }
                    973:     }
                    974: 
                    975:     static bool
                    976:     faa_check(struct faa_softc *sc)
                    977:     {
                    978:             uint32_t testval = 0xff11ee22; 
                    979:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_COMMAND, FAA_COMMAND_STORE_A);
                    980:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_DATA, testval);
                    981:             if (bus_space_read_4(sc->sc_regt, sc->sc_regh, FAA_DATA) == testval)
                    982:                     return true;
                    983: 
                    984:             return false;
                    985:     }
                    986: 
                    987: ### Accessing the hardware registers - running the example
                    988: 
                    989: -   Update the kernel binary and run it again
                    990: 
                    991: -   Check the GXemul log
                    992: 
                    993: <!-- -->
                    994: 
                    995:     [ faa: COMMAND register (0x4) WRITE value 0x2 ]
                    996:     [ faa: DATA register (0x8) WRITE value 0xff11ee22 ]
                    997:     [ faa: DATA register (0x8) READ value 0xff11ee22 ]
                    998: 
                    999: -   GXemul will conveniently display all accesses to our device
                   1000: 
                   1001: -   The faa driver still does attach without error, which means that the
                   1002:     check function is working properly
                   1003: 
                   1004: <!-- -->
                   1005: 
                   1006:     faa0 at pci0 dev 12 function 0: Fake Cards Advanced Addition Accelerator (rev. 0x01)
                   1007:     faa0: registers at 0x10110000
                   1008: 
                   1009: ### Implementing addition using the hardware
                   1010: 
                   1011: -   The basic principle of device operation should be laid out in the
                   1012:     data sheet
                   1013: 
                   1014: -   We need to implement an algorithm based on this description
                   1015: 
                   1016: -   Writing such an algorithm is often not needed, since the NetBSD
                   1017:     kernel already has frameworks for common device types (such as
                   1018:     atabus/wd for IDE and SATA hard disk controllers, wsdisplay/wscons
                   1019:     for frame buffers, etc.)
                   1020: 
                   1021: ### Implementing addition using the hardware
                   1022: 
                   1023: -   Define all registers
                   1024: 
                   1025: -   src/sys/dev/pci/faareg.h
                   1026: 
                   1027: <!-- -->
                   1028: 
                   1029:     #define FAA_STATUS              0x0
                   1030:     #define FAA_COMMAND             0x4
                   1031:     #define FAA_COMMAND_ADD             __BIT(0)        
                   1032:     #define FAA_COMMAND_STORE_A         __BIT(1)
                   1033:     #define FAA_COMMAND_STORE_B         __BIT(2)
                   1034:     #define FAA_DATA                0x8
                   1035:     #define FAA_RESULT              0xC
                   1036: 
                   1037: ### Implementing addition using the hardware
                   1038: 
                   1039: -   Add a new function to the main driver code
                   1040: 
                   1041: -   src/sys/dev/pci/faa.c
                   1042: 
                   1043: <!-- -->
                   1044: 
                   1045:     static void
                   1046:     faa_attach(device_t parent, device_t self, void *aux)
                   1047:     {
                   1048:             /* ... */
                   1049:             aprint_normal_dev(sc->sc_dev, "just checking: 1 + 2 = %d\n", faa_add(sc, 1, 2));
                   1050:     }
                   1051: 
                   1052:     static uint32_t
                   1053:     faa_add(struct faa_softc *sc, uint32_t a, uint32_t b)
                   1054:     {
                   1055:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_COMMAND, FAA_COMMAND_STORE_A);
                   1056:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_DATA, a);
                   1057:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_COMMAND, FAA_COMMAND_STORE_B);
                   1058:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_DATA, b);
                   1059:             bus_space_write_4(sc->sc_regt, sc->sc_regh, FAA_COMMAND, FAA_COMMAND_ADD);
                   1060:             return bus_space_read_4(sc->sc_regt, sc->sc_regh, FAA_RESULT);
                   1061:     }
                   1062: 
                   1063: ### Implementing addition using the hardware - running the example
                   1064: 
                   1065: -   Update the kernel binary and run it again
                   1066: 
                   1067: -   Check GXemul log
                   1068: 
                   1069: <!-- -->
                   1070: 
                   1071:     [ faa: COMMAND register (0x4) WRITE value 0x2 ]
                   1072:     [ faa: DATA register (0x8) WRITE value 0x1 ]
                   1073:     [ faa: COMMAND register (0x4) WRITE value 0x4 ]
                   1074:     [ faa: DATA register (0x8) WRITE value 0x2 ]
                   1075:     [ faa: COMMAND register (0x4) WRITE value 0x1 ]
                   1076:     [ faa: RESULT register (0xC) READ value 0x3 ]
                   1077: 
                   1078: -   Looks like it worked!
                   1079: 
                   1080: <!-- -->
                   1081: 
                   1082:     faa0 at pci0 dev 12 function 0: Fake Cards Advanced Addition Accelerator (rev. 0x01)
                   1083:     faa0: registers at 0x10110000
                   1084:     faa0: just checking: 1 + 2 = 3
                   1085: 
                   1086: Interacting with userspace
                   1087: ==========================
                   1088: 
                   1089: Device files
                   1090: ------------
                   1091: 
                   1092: ### The kernel-user space interface
                   1093: 
                   1094: -   Now that the core functionality of the kernel driver is working, it
                   1095:     should be exposed to user space
                   1096: 
                   1097: -   The interface between kernel driver and userspace can be designed in
                   1098:     many different ways
                   1099: 
                   1100: -   The classic UNIX way of interfacing between the kernel and user
                   1101:     space is a device file
                   1102: 
                   1103: -   Even when using device files there is no single interfacing method
                   1104:     that fits all use cases
                   1105: 
                   1106: -   It’s up to the programmer to define the communication protocol
                   1107: 
                   1108: ### Device files
                   1109: 
                   1110: -   crw-r—– 1 root wheel 101, 1 Aug 12 21:53 /dev/file
                   1111: 
                   1112: -   The kernel identifies which driver should service the request to
                   1113:     this file by using major and minor numbers (101 and 1 in the example
                   1114:     above)
                   1115: 
                   1116: -   The major number identifies the driver
                   1117: 
                   1118: -   The minor number usually identifies the driver instance, although
                   1119:     the driver is free to use it in any other way
                   1120: 
                   1121: -   In NetBSD device files are created statically
                   1122: 
                   1123:     -   By the MAKEDEV script during installation or boot
                   1124: 
                   1125:     -   Manually by using the mknod utility
                   1126: 
                   1127: ### Operations on device files
                   1128: 
                   1129: -   [open(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)
                   1130:     and
                   1131:     [close(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)
                   1132: 
                   1133: -   [read(2)](http://netbsd.gw.com/cgi-bin/man-cgi?read++NetBSD-current)
                   1134:     and
                   1135:     [write(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)
                   1136: 
                   1137: -   [ioctl(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)
                   1138: 
                   1139: -   [poll(2)](http://netbsd.gw.com/cgi-bin/man-cgi?poll++NetBSD-current)
                   1140: 
                   1141: -   [mmap(2)](http://netbsd.gw.com/cgi-bin/man-cgi?write++NetBSD-current)
                   1142: 
                   1143: -   and more…
                   1144: 
                   1145: -   Any mix of the above system calls might be used to interface between
                   1146:     the kernel and user space
                   1147: 
                   1148: -   We’ll implement an ioctl(2)-based communication mechanism
                   1149: 
                   1150: ### Adding cdevsw
                   1151: 
                   1152: -   cdevsw
                   1153: 
                   1154:     is used to decide which operation on the character device file calls
                   1155:     which driver function
                   1156: 
                   1157: -   Not all calls have to be implemented, although some device layers
                   1158:     define a set of calls that a driver must implement
                   1159: 
                   1160: -   For example disk drivers must implement open, close, read, write and
                   1161:     ioctl
                   1162: 
                   1163: -   src/sys/dev/pci/faa.c
                   1164: 
                   1165: <!-- -->
                   1166: 
                   1167:     dev_type_open(faaopen);
                   1168:     dev_type_close(faaclose);
                   1169:     dev_type_ioctl(faaioctl);
                   1170: 
                   1171:     const struct cdevsw faa_cdevsw = {
                   1172:             faaopen, faaclose, noread, nowrite, faaioctl,
                   1173:             nostop, notty, nopoll, nommap, nokqfilter, D_OTHER
                   1174:     };
                   1175: 
                   1176: ### Prototyping the cdevsw operations
                   1177: 
                   1178: -   The dev\_type\* macros are used to prototype the functions passed to
                   1179:     cdevsw
                   1180: 
                   1181: -   Pass no followed by a function name to the appropriate cdevsw field
                   1182:     if it is not implemented
                   1183: 
                   1184: -   There’s also bdevsw for block devices, but we won’t use it in this
                   1185:     example
                   1186: 
                   1187: -   The last member of the cdevsw structure defines the device flags,
                   1188:     originally it was used to define the device type (still used for
                   1189:     disks, tape drives and ttys, for other devices pass D\_OTHER)
                   1190: 
                   1191: ### Implemeting the cdevsw operations - open / close
                   1192: 
                   1193: -   src/sys/dev/pci/faa.c
                   1194: 
                   1195: <!-- -->
                   1196: 
                   1197:     int
                   1198:     faaopen(dev_t dev, int flags, int mode, struct lwp *l)
                   1199:     {
                   1200:             struct faa_softc *sc;
                   1201:             sc = device_lookup_private(&faa_cd, minor(dev));
                   1202: 
                   1203:             if (sc == NULL)
                   1204:                     return ENXIO;
                   1205:             if (sc->sc_flags & FAA_OPEN)
                   1206:                     return EBUSY;
                   1207: 
                   1208:             sc->sc_flags |= FAA_OPEN;
                   1209:             return 0;
                   1210:     }
                   1211:     int
                   1212:     faaclose(dev_t dev, int flag, int mode, struct lwp *l)
                   1213:     {
                   1214:             struct faa_softc *sc;
                   1215:             sc = device_lookup_private(&faa_cd, minor(dev));
                   1216: 
                   1217:             if (sc->sc_flags & FAA_OPEN)
                   1218:                     sc->sc_flags =~ FAA_OPEN;
                   1219: 
                   1220:             return 0;
                   1221:     }
                   1222: 
                   1223: ### Defining the ioctls
                   1224: 
                   1225: -   ioctl(2)
                   1226: 
                   1227:     can be used to call kernel-level functions and exchange data between
                   1228:     the kernel and user space
                   1229: 
                   1230: -   The classic way of passing data is by using structures, their
                   1231:     definitions are shared between the kernel and user space code
                   1232: 
                   1233: -   The driver might support more than one ioctl, the \_IO\* macros are
                   1234:     used to define the operation and associated structure used to
                   1235:     exchange data
                   1236: 
                   1237:     -   \_IO
                   1238: 
                   1239:         - just a kernel function call, no data exchange
                   1240: 
                   1241:     -   \_IOR
                   1242: 
                   1243:         - kernel function call and data pass from kernel to user space
                   1244: 
                   1245:     -   \_IOW
                   1246: 
                   1247:         - kernel function call and data pass from user space to kernel
                   1248: 
                   1249:     -   \_IOWR
                   1250: 
                   1251:         - kernel function call and data exchange in both directions
                   1252: 
                   1253:     -   \#define DRIVERIO\_IOCTLNAME \_IOXXX(group, ioctl\_number, data
                   1254:         structure)
                   1255: 
                   1256: Using ioctls
                   1257: ------------
                   1258: 
                   1259: ### Defining the ioctls
                   1260: 
                   1261: -   src/sys/dev/pci/faaio.h
                   1262: 
                   1263: <!-- -->
                   1264: 
                   1265:     #include <sys/ioccom.h>
                   1266:                          
                   1267:     #define FAAIO_ADD   _IOWR(0, 1, struct faaio_add)
                   1268: 
                   1269:     struct faaio_add {
                   1270:         uint32_t a;
                   1271:         uint32_t b;
                   1272:         uint32_t *result;
                   1273:     };
                   1274: 
                   1275: -   In the above example the ioctl group is not defined (0), but a
                   1276:     single letter identifier could appear as first argument to \_IOWR
                   1277: 
                   1278: ### Implemeting the cdevsw operations - ioctl
                   1279: 
                   1280: -   src/sys/dev/pci/faa.c
                   1281: 
                   1282: <!-- -->
                   1283: 
                   1284:     int
                   1285:     faaioctl(dev_t dev, u_long cmd, void *data, int flag, struct lwp *l)
                   1286:     {
                   1287:             struct faa_softc *sc = device_lookup_private(&faa_cd, minor(dev));
                   1288:             int err;
                   1289: 
                   1290:             switch (cmd) {
                   1291:             case FAAIO_ADD:
                   1292:                     err = faaioctl_add(sc, (struct faaio_add *) data);
                   1293:                     break;
                   1294:             default:
                   1295:                     err = EINVAL;
                   1296:                     break;
                   1297:             }
                   1298:             return(err);
                   1299:     }
                   1300:     static int
                   1301:     faaioctl_add(struct faa_softc *sc, struct faaio_add *data)
                   1302:     {
                   1303:             uint32_t result; int err;
                   1304: 
                   1305:             aprint_normal_dev(sc->sc_dev, "got ioctl with a %d, b %d\n",
                   1306:                 data->a, data->b);
                   1307: 
                   1308:             result = faa_add(sc, data->a, data->b);
                   1309:             err = copyout(&result, data->result, sizeof(uint32_t));
                   1310:             return err;
                   1311:     }
                   1312: 
                   1313: ### Using copyout to pass data to userspace
                   1314: 
                   1315: -   The copy(9) functions are used to copy kernel space data from/to
                   1316:     user space
                   1317: 
                   1318: -   copyout(kernel\_address, user space\_address, size);
                   1319: 
                   1320: -   Actually on Cobalt we could just do data-\>result = faa\_add();
                   1321:     instead of calling the copyout function, but that is a bad idea
                   1322: 
                   1323: -   Some architectures (such as sparc64) have totally separate kernel
                   1324:     and user address spaces $ \implies $ user space addresses are
                   1325:     meaningless in the kernel
                   1326: 
                   1327: ### Defining device major number
                   1328: 
                   1329: -   Device major numbers for hardware drivers are usually defined in a
                   1330:     per-port manner[^7]
                   1331: 
                   1332: -   src/sys/arch/\$PORTNAME/conf/majors.\$PORTNAME
                   1333: 
                   1334: -   src/sys/arch/cobalt/conf/majors.cobalt
                   1335: 
                   1336: -   The following defines a new character device file called /dev/faa\*
                   1337:     with major number 101, but only if the faa driver is included in the
                   1338:     kernel (last argument)
                   1339: 
                   1340: -   device-major faa char 101 faa
                   1341: 
                   1342: ### Creating the device node
                   1343: 
                   1344: -   The mknod utility can be used to create the device file manually
                   1345: 
                   1346: -   The driver name can be specified instead of the major number - it
                   1347:     will be automatically resolved into the correct major number
                   1348: 
                   1349: -   mknod name [b | c] [major | driver] minor
                   1350: 
                   1351: -   mknod /dev/faa0 c faa 0
                   1352: 
                   1353: -   Created successfully
                   1354: 
                   1355: -   crw-r–r– 1 root wheel 101, 0 Oct 8 2012 /dev/faa0
                   1356: 
                   1357: An example user space program
                   1358: -----------------------------
                   1359: 
                   1360: ### An example user space program
                   1361: 
                   1362: -   The example program will open the device file and call ioctl(2) on
                   1363:     it
                   1364: 
                   1365: -   As simple as possible, just to show how communication is done
                   1366: 
                   1367: -   Using ioctls from the user space
                   1368: 
                   1369:     -   Open the device file with O\_RDWR
                   1370: 
                   1371:     -   Call ioctl(2) with the operation number and structure as
                   1372:         parameters
                   1373: 
                   1374: ### An example user space program - source
                   1375: 
                   1376:     void add(int, uint32_t, uint32_t);
                   1377: 
                   1378:     static const char* faa_device = "/dev/faa0";
                   1379: 
                   1380:     int
                   1381:     main(int argc, char *argv[])
                   1382:     {
                   1383:             int devfd;
                   1384: 
                   1385:             if (argc != 3) {
                   1386:                     printf("usage: %s a b\n", argv[0]);
                   1387:                     return 1;
                   1388:             }
                   1389:             if ( (devfd = open(faa_device, O_RDWR)) == -1) {
                   1390:                     perror("can't open device file");
                   1391:                     return 1;
                   1392:             }
                   1393: 
                   1394:             add(devfd, atoi(argv[1]), atoi(argv[2]));
                   1395: 
                   1396:             close(devfd);
                   1397:             return 0;
                   1398:     }
                   1399: 
                   1400: ### An example user space program - source
                   1401: 
                   1402:     void
                   1403:     add(int devfd, uint32_t a, uint32_t b)
                   1404:     {
                   1405:             struct faaio_add faaio;
                   1406:             uint32_t result = 0;
                   1407: 
                   1408:             faaio.result = &result;
                   1409:             faaio.a = a;
                   1410:             faaio.b = b;
                   1411: 
                   1412:             if (ioctl(devfd, FAAIO_ADD, &faaio) == -1) {
                   1413:                     perror("ioctl failed");
                   1414:             }
                   1415:             printf("%d\n", result);
                   1416:     }
                   1417: 
                   1418: ### An example user space program - running it
                   1419: 
                   1420:     # make
                   1421:     cc -o aaa_add aaa_add.c
                   1422:     # ./aaa_add 3 7
                   1423:     faa0: got ioctl with a 3, b 7
                   1424:     10
                   1425: 
                   1426: -   The program is successfully accessing the faa driver through the
                   1427:     ioctl
                   1428: 
                   1429: -   The faa0:... line is a kernel message, normally only seen on the
                   1430:     console terminal
                   1431: 
                   1432: A few tips
                   1433: ==========
                   1434: 
                   1435: Avoiding common pitfalls
                   1436: ------------------------
                   1437: 
                   1438: ### Avoiding common pitfalls
                   1439: 
                   1440: -   Always free resources allocated in the match or probe functions
                   1441: 
                   1442: -   Always use bus\_space methods, don’t access the hardware using a
                   1443:     pointer dereference
                   1444: 
                   1445: -   If possible test on more than one hardware architecture, some bugs
                   1446:     may surface
                   1447: 
                   1448: -   Don’t reinvent the wheel, try to use existing kernel frameworks as
                   1449:     much as possible
                   1450: 
                   1451: -   Use copy(9) (or uiomove(9) or store(9)/fetch(9)) to move data
                   1452:     between the kernel and user space
                   1453: 
                   1454: Basic driver debugging
                   1455: ----------------------
                   1456: 
                   1457: ### Basic driver debugging
                   1458: 
                   1459: -   Use aprint\_debug to print debug-level messages on console and log
                   1460:     them (enabled by passing AB\_DEBUG from the boot loader)
                   1461: 
                   1462: -   Use the built-in DDB debugger
                   1463: 
                   1464:     -   Enabled by the kernel option DDB
                   1465: 
                   1466:     -   A kernel panic will start DDB if the DDB\_ONPANIC=1 kernel
                   1467:         option is specified or the ddb.onpanic sysctl is set to 1.
                   1468: 
                   1469:     -   Run \# sysctl -w kern.panic\_now=1 to trigger a panic manually
                   1470:         (DIAGNOSTIC option)
                   1471: 
                   1472: -   Remote debugging is possible on some ports
                   1473: 
                   1474:     -   With KGDB through the serial port
                   1475: 
                   1476:     -   With IPKDB through the network
                   1477: 
                   1478: Summary
                   1479: =======
                   1480: 
                   1481: The end
                   1482: -------
                   1483: 
                   1484: ### Further reading
                   1485: 
                   1486: -   Documentation, articles:
                   1487: 
                   1488:     -   [A Machine-Independent DMA Framework for NetBSD, Jason R.
                   1489:         Thorpe](http://www.netbsd.org/docs/kernel/bus_dma.pdf)
                   1490: 
                   1491:     -   [Writing Drivers for NetBSD, Jochen
                   1492:         Kunz](ftp://ftp.netbsd.org/pub/NetBSD/misc/ddwg/NetBSD-driver_writing-1.0.1e.pdf)
                   1493: 
                   1494:     -   [NetBSD Documentation: Writing a pseudo
                   1495:         device](http://www.netbsd.org/docs/kernel/pseudo/)
                   1496: 
                   1497:     -   [autoconf(9)](http://netbsd.gw.com/cgi-bin/man-cgi?autoconf+9+NetBSD-current),
                   1498:         [bus\_space(9)](http://netbsd.gw.com/cgi-bin/man-cgi?bus_space+9+NetBSD-current)
                   1499:         [bus\_dma(9)](http://netbsd.gw.com/cgi-bin/man-cgi?bus_dma+9+NetBSD-current)
                   1500:         [driver(9)](http://netbsd.gw.com/cgi-bin/man-cgi?driver+9+NetBSD-current),
                   1501:         [pci(9)](http://netbsd.gw.com/cgi-bin/man-cgi?pci+9+NetBSD-current)
                   1502:         man pages, etc.
                   1503: 
                   1504: -   Example source code of drivers:
                   1505: 
                   1506:     -   tdvfb
                   1507: 
                   1508:         , voodoofb are fairly good frame buffer driver examples with
                   1509:         documentation publicly available.
                   1510: 
                   1511:     -   etsec
                   1512: 
                   1513:         is a nice example of a more complicated network interface driver
                   1514: 
                   1515: ### Get the source code
                   1516: 
                   1517: -   Download the source code and materials for this tutorial
                   1518: 
                   1519: -   <https://github.com/rkujawa/busspace-tutorial>
                   1520: 
                   1521: -   <https://github.com/rkujawa/gxemul-tutorial>
                   1522: 
                   1523: ### Questions?
                   1524: 
                   1525: -   Do you have any questions?
                   1526: 
                   1527: ### The End…
                   1528: 
                   1529: ![image](NetBSD.png)
                   1530: 
                   1531: Thank you!
                   1532: 
                   1533: [^1]: At least they should, some functions are missing on less popular
                   1534:     ports
                   1535: 
                   1536: [^2]: Only three of these registers are of any importance for us at this
                   1537:     moment
                   1538: 
                   1539: [^3]: Required if you are NetBSD developer, optional otherwise.
                   1540: 
                   1541: [^4]: Might not exist if the driver is only a simple passthrough from a
                   1542:     specific bus to another MI driver.
                   1543: 
                   1544: [^5]: Omitted if not needed.
                   1545: 
                   1546: [^6]: although you’ll often have to use bus\_size\_t
                   1547: 
                   1548: [^7]: It’s also possible to define a major in a machine-independent way
                   1549:     in src/sys/conf/majors

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb