File:  [NetBSD Developer Wiki] / wikisrc / tutorials / how_to_enable_and_run_dtrace.mdwn
Revision 1.29: download - view: text, annotated - select for diffs
Mon Mar 30 15:45:00 2020 UTC (5 months, 4 weeks ago) by sevan
Branches: MAIN
CVS tags: HEAD
formatting

    1: DTrace is a Dynamic Tracing framework developed by Sun and ported to NetBSD. It enables extensive instrumentation of the kernel and user space. See the [DTrace Community Page](http://dtrace.org) for more information.
    2: Also see [DTrace Introduction](http://dtrace.org/guide/preface.html), Brendan Gregg's [DTrace one liners](http://www.brendangregg.com/DTrace/dtrace_oneliners.txt) and his notes for [DTrace on FreeBSD](https://wiki.freebsd.org/DTrace/).
    3: 
    4: # Current status
    5: 
    6: ## Supported platforms
    7: 
    8: DTrace is a work-in-progress effort and it is for x86 systems and some arm boards.
    9: 
   10: * i386 and amd64
   11: * earm* (evbarm and armv4 based ports)
   12: 
   13: ## Supported providers
   14: 
   15: * DTrace: What to do when a script BEGINs, ENDs, ERRORs
   16: * FBT: Function Boundary Tracing
   17: * IO: Disk I/O
   18: * Lockstat: Kernel Lock Statistics
   19: * Proc: Process and thread related events
   20: * Profile: Time based interrupt event source for Profiling
   21: * SDT: Statically Defined Tracing
   22: * Syscall: System Calls
   23: * Syscall Linux (32bit & 64 bit): System calls via the Linux binary emulation layer
   24: * VFS: Filesystem operations (confined to namecache events at time of writing - 8.99.22)
   25: 
   26: ## TODO for netbsd-7
   27: 
   28: * Measure effect of `options KDTRACE_HOOKS` on system performance.
   29: * Determine whether the profile module works and list it here.
   30: * Integrate [[riz|users/riz]]'s syscall provider patch.
   31: 
   32: # How to use
   33: 
   34: ##  Building DTrace 
   35: 
   36: You need the following options in your kernel: 
   37:     
   38:     options         KDTRACE_HOOKS   # kernel DTrace hooks
   39:     options         MODULAR
   40: 
   41: Optionally:
   42: 
   43:     options         INSECURE   # permit modules to loaded from user space once system has gone multiuser and securelevel has been raised.
   44: 
   45: A Distribution needs to be built with the options `MKDTRACE=yes` and `MKCTF=yes`, this is taken care of automatically and doesn't need to be specified manually. The list of platforms it is applied to automatically is set in `src/share/mk/bsd.own.mk`
   46: 
   47: Set the system to load the solaris and dtrace related modules in `/etc/modules.conf`, for a list of available modules, see `/stand/$MACHINE/$VERSION/modules/`
   48: 
   49: For example, add the following to `/etc/modules.conf` (the file may not exist already on a system):
   50:     
   51: - `solaris`
   52: - `dtrace`
   53: - `dtrace_fbt`
   54: - `dtrace_lockstat`
   55: - `dtrace_profile`
   56: - `dtrace_sdt`
   57: - `dtrace_syscall`
   58: - `dtrace_syscall_linux`
   59:     
   60: A `dtrace` device node is created automatically in `/dev/dtrace` when the modules are loaded into place.
   61:     
   62: List the dtrace probes 
   63:     
   64:     dtrace -l
   65:     
   66:        ID   PROVIDER            MODULE                          FUNCTION NAME
   67:         1     dtrace                                                     BEGIN
   68:         2     dtrace                                                     END
   69:         3     dtrace                                                     ERROR
   70:         4        fbt            netbsd             AcpiAcquireGlobalLock entry
   71:         5        fbt            netbsd             AcpiAcquireGlobalLock return
   72:         6        fbt            netbsd             AcpiAllocateRootTable entry
   73:         7        fbt            netbsd                    AcpiAttachData entry
   74:         . 
   75:         .
   76:     29129        fbt           solaris                   zfs_vop_getattr entry 
   77:     29130        fbt           solaris                   zfs_vop_getattr return
   78:     29131       proc                                                     create
   79:     29132       proc                                                     exec 
   80:         .
   81:         .
   82:     29140       proc                                                     lwp_start
   83:     29141       proc                                                     lwp_exit
   84: 
   85: 
   86: ## Running hello world 
   87: 
   88: Put the following into the file hello.d:
   89:     
   90:     BEGIN
   91:     {
   92:         trace("Hello world");
   93:         exit(0);
   94:     }
   95:     
   96: 
   97: Run the hello world script: 
   98:     
   99:     dtrace -s hello.d
  100:     
  101:     dtrace: script './hello.d' matched 1 probe
  102:     CPU     ID                    FUNCTION:NAME
  103:       0      1                           :BEGIN   Hello world
  104: 
  105: The same script could be executed as a one liner on the shell, using
  106: 
  107:     dtrace -n 'BEGIN { trace("Hello world"); exit(0); }'
  108: 
  109: ## A more complex example
  110: 
  111: The following script traces the execution of a sleep operation
  112: in the kernel. Put it in sleep.d:
  113:     
  114:     #pragma D option flowindent
  115: 
  116:     syscall::nanosleep:entry
  117:     /execname == "sleep" && guard++ == 0/
  118:     {
  119:             self->traceme = 1;
  120:     }
  121: 
  122:     fbt:::
  123:     /self->traceme/
  124:     {}
  125: 
  126:     syscall::nanosleep:return
  127:     /self->traceme/
  128:     {
  129:             self->traceme = 0;
  130:             exit(0);
  131:     }
  132: 
  133: Start the script running:
  134: 
  135:     dtrace -s sleep.d
  136: 
  137: This will take a while as the script instruments every function in the
  138: kernel. When it's ready, it will print a message like "dtrace: script
  139: 'sleep.d' matched 59268 probes".  Then execute a "sleep 2" in another
  140: shell.
  141: 
  142: ## Tools included in base
  143: 
  144: Starting with NetBSD-8, on builds where `MKDTRACE=yes` is set, scripts from
  145: [Brendan Gregg's DTrace toolkit](https://github.com/opendtrace/toolkit/) are installed in base as standard.
  146: 
  147: At present, the following scripts are installed in `/usr/sbin`: 
  148: 
  149: - `dtruss` - An implementation of the truss utility in DTrace which traces the system calls
  150: made by a process
  151: - `execsnoop` - snoop on execution of processes as they occur
  152: - `opensnoop` - snoop on openning of files as they occur
  153: - `procsystime` -  print process system call time details.
  154: 
  155: ## Troubleshooting
  156: 
  157: The Compact C Type Format (CTF) has a 2^15 limit on types which can overflow, this prevents DTrace from
  158: working correctly.
  159: 
  160: Check the number of types using `ctfdump` e.g
  161: 
  162:     ctfdump -S /netbsd
  163: 
  164: Note the line which states `total number of types`, the value should by less than 32768.
  165: 
  166: If overflow is not an issue, `libdtrace(3)` can provide some insight into what is going on via an
  167: environment variable. Define `DTRACE_DEBUG` before tracing.
  168: 
  169:      env DTRACE_DEBUG= execsnoop
  170: 

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb