Feb 2017
S M T W T F S
     
20 21
       

Archives

This page is a blog mirror of sorts. It pulls in articles from blog's feed and publishes them here (with a feed, too).

The LLVM project is a quickly moving target, this also applies to the LLVM debugger -- LLDB. It's actively used in several first-class operating systems, while - thanks to my spare time dedication - NetBSD joined the LLDB club in 2014, only lately the native support has been substantially improved and the feature set is quickly approaching the support level of Linux and FreeBSD. During this work 12 patches were committed to upstream, 12 patches were submitted to review, 11 new ATF were tests added, 2 NetBSD bugs filed and several dozens of commits were introduced in pkgsrc-wip, reducing the local patch set to mostly Native Process Plugin for NetBSD.

What has been done in NetBSD

1. Triagged issues of ptrace(2) in the DTrace/NetBSD support

Chuck Silvers works on improving DTrace in NetBSD and he has detected an issue when tracer signals are being ignored in libproc. The libproc library is a compatibility layer for DTrace simulating /proc capabilities on the SunOS family of systems.

I've verified that the current behavior of signal routing is incorrect. The NetBSD kernel correctly masks signals emitted by a tracee, not routing them to its tracer. On the other hand the masking rules in the inferior process blacklists signals generated by the kernel, which is incorrect and turns a debugger into a deaf listener. This is the case for libproc as signals were masked and software breakpoints triggering INT3 on i386/amd64 CPUs and SIGTRAP with TRAP_BRKP si_code wasn't passed to the tracer.

This isn't limited to turning a debugger into a deaf listener, but also a regular execution of software breakpoints requires: rewinding the program counter register by a single instruction, removing trap instruction and restoring the original instruction. When an instruction isn't restored and further code execution is pretty randomly affected, it resulted in execution anomalies and breaking of tracee.

A workaround for this is to disable signal masking in tracee.

Another drawback inspired by the DTrace code is to enhance PT_SYSCALL handling by introducing a way to distinguish syscall entry and syscall exit events. I'm planning to add dedicated si_codes for these scenarios. While there, there are users requesting PT_STEP and PT_SYSCALL tracing at the same time in an efficient way without involving heuristcs.

I've filed the mentioned bug:

I've added new ATF tests:

  • Verify that masking single unrelated signal does not stop tracer from catching other signals
  • Verify that masking SIGTRAP in tracee stops tracer from catching this raised signal
  • Verify that masking SIGTRAP in tracee does not stop tracer from catching software breakpoints
  • Verify that masking SIGTRAP in tracee does not stop tracer from catching single step trap
  • Verify that masking SIGTRAP in tracee does not stop tracer from catching exec() breakpoint
  • Verify that masking SIGTRAP in tracee does not stop tracer from catching PTRACE_FORK breakpoint
  • Verify that masking SIGTRAP in tracee does not stop tracer from catching PTRACE_VFORK breakpoint
  • Verify that masking SIGTRAP in tracee does not stop tracer from catching PTRACE_VFORK_DONE breakpoint
  • Verify that masking SIGTRAP in tracee does not stop tracer from catching PTRACE_LWP_CREATE breakpoint
  • Verify that masking SIGTRAP in tracee does not stop tracer from catching PTRACE_LWP_EXIT breakpoint

2. ELF Auxiliary Vectors

The ELF file format permits to transfer additional information for a process with a dedicated container of properties, it's named ELF Auxilary Vector. Every system has its dedicated way to read this information in a debugger from a tracee. The NetBSD approach is to transfer this vector with a ptrace(2) API PIOD_READ_AUXV. Our interface shares the API with OpenBSD. I filed a bug that our interface returns vector size of 8496 bytes, while OpenBSD has constant 64 bytes. It was diagnosed and fixed by Christos Zoluas that we were incorrectly counting bits and bytes and this enlarged the data streamlined. The bug was harmless and had no known side-effects besides large chunk of zeroed data.

There is also a prepared local patch extending NetBSD platform support to read information for this vector, it's primarily required for correct handling of PIE binaries. At the moment there is no interface similar to "info auxv" to the one from GDB. Unfortunately at the current stage, this code is still unused by NetBSD. I will return to it once the Native Process Plugin is enhanced.

I've filed the mentioned bug:

I've added new ATF test:

  • Verify PT_READ_AUXV called for tracee.

What has been done in LLDB

1. Resolving executable's name with sysctl(7)

In the past the way to retrieve a specified process' executable path name was using Linux-compatibile feature in procfs (/proc). The canonical solution on Linux is to resolve path of /proc/$PID/exe. Christos Zoulas added in DTrace port enhancements a solution similar to FreeBSD to retrieve this property with sysctl(7).

This new approach removes dependency on /proc mounted and Linux compatibility functionality.

Support for this has been submitted to LLDB and merged upstream:

2. Real-Time Signals

The key feature of the POSIX standard with Asynchronous I/O is to support Real-Time Signals. One of their use-cases is in .NET debugging facilities. Support for this set of signals was developed during Google Summer of Code 2016 by Charles Cui and reviewed and committed by Christos Zoulas.

I've extended the LLDB capabilities for NetBSD to recognize these signals in the NetBSDSignals class.

Support for this has been submitted to LLDB and merged upstream:

3. Conflict removal with system-wide six.py

The transition from Python 2.x to 3.x is still ongoing and will take a while. The current deadline support for the 2.x generation has been extended to 2020. One of the ways to keep both generations supported in the same source-code is to use the six.py library (py2 x py3 = 6.py). It abstracts commonly used constructs to support both language families.

The issue for packaging LLDB in NetBSD was to install this tiny library unconditionally to a system-wide location. There were several solutions to this approach:

  • drop Python 2.x support,
  • install six.py into subdirectory,
  • make an installation of six.py conditional.

The first solution would turn discussion into flamewar, the second one happened to be too difficult to be properly implemented as the changes were invasive and Python is used in several places of the code-base (tests, bindings...). The final solution was to introduce a new CMake option LLDB_USE_SYSTEM_SIX - disabled by default to retain the current behavior.

To properly implement LLDB_USE_SYSTEM_SIX, I had to dig into installation scripts combined in CMake and Python files. It wasn't helping that Python scripts were reinventing getopt(3) functionality.. and I had to alter it in order to introduce a new option --useSystemSix.

Support for this has been submitted to LLDB and merged upstream:

4. Do not pass non-POD type variables through variadic function

There was a long standing local patch in pkgsrc, added by Tobias Nygren and detected with Clang.

According to the C++11 standard 5.2.2/7:

Passing a potentially-evaluated argument of class type having a non-trivial copy constructor, a non-trivial move constructor, or a non-trivial destructor, with no corresponding parameter, is conditionally-supported with implementation-defined semantics.

A short example to trigger similar warning was presented by Joerg Sonnenberg:

#include <string>
#include <cstdarg>

void f(std::string msg, ...) {
  va_list ap;
  va_start(ap, msg);
}

This code compiled against libc++ gives:

test.cc:6:3: error: cannot pass object of non-POD type 'std::string' (aka 'basic_string<char, char_traits<char>,

allocator<char> >') through variadic function; call will abort at runtime [-Wnon-pod-varargs]

Support for this has been submitted to LLDB and merged upstream:

5. Add NetBSD support in Host::GetCurrentThreadID

Linux has a very specific thread model, where process is mostly equivalent to native thread and POSIX thread - it's completely different on other mainstream general-purpose systems. That said fallback support to translate pthread_t on NetBSD to retrieve the native integer identifier was incorrect. The proper NetBSD function to retrieve light-weigth process identification is to call _lwp_self(2).

Support for this has been submitted to LLDB and merged upstream:

6. Synchronize PlatformNetBSD with Linux

The old PlatformNetBSD code was based on the FreeBSD version. While the FreeBSD current one is still similar to the one from a year ago, it's inappropriate to handle a remote process plugin approach. This forced me to base refreshed code on Linux.

After realizing that PlatformPlugin on POSIX platforms suffers from code duplication, Pavel Labath helped out to eliminate common functions shared by other systems. This resulted in a shorter patch synchronizing PlatformNetBSD with Linux, this step opened room for FreeBSD to catch up.

Support for this has been submitted to LLDB and merged upstream:

7. Transform ProcessLauncherLinux to ProcessLauncherPosixFork

It is UNIX specific that signal handlers are global per application. This introduces issues with wait(2)-like functions called in tracers, as these functions tend to conflict with real-life libraries, notably GUI toolkits (where SIGCHLD events are handled).

The current best approach to this limitation is to spawn a forkee and establish a remote connection over the GDB protocol with a debugger frontend. ProcessLauncherLinux was prepared with this design in mind and I have added support for NetBSD. Once FreeBSD will catch up, they might reuse the same code.

Support for this has been submitted to LLDB and merged upstream:

8. Document that LaunchProcessPosixSpawn is used on NetBSD

Host::GetPosixspawnFlags was built for most POSIX platforms - however only Apple, Linux, FreeBSD and other-GLIBC ones (I assume Debian/kFreeBSD to be GLIBC-like) were documented. I've included NetBSD to this list..

Support for this has been submitted to LLDB and merged upstream:

  • Document that LaunchProcessPosixSpawn is used on NetBSD committed r293770

9. Switch std::call_once to llvm::call_once

There is a long-standing bug in libstdc++ on several platforms that std::call_once is broken for cryptic reasons. This motivated me to follow the approach from LLVM and replace it with homegrown fallback implementation llvm::call_once.

This change wasn't that simple at first sight as the original LLVM version used different semantics that disallowed straight definition of non-static once_flag. Thanks to cooperation with upstream the proper solution was coined and LLDB now works without known regressions on libstdc++ out-of-the-box.

Support for this has been submitted to LLVM, LLDB and merged upstream:

10. Other enhancements

I a had plan to push more code in this milestone besides the mentioned above tasks. Unfortunately not everything was testable at this stage.

Among the rescheduled projects:

  • In the NetBSD platform code conflict removal in GetThreadName / SetThreadName between pthread_t and lwpid_t. It looks like another bite from the Linux thread model. Proper solution to this requires pushing forward the Process Plugin for NetBSD.
  • Host::LaunchProcessPosixSpawn proper setting ::posix_spawnattr_setsigdefault on NetBSD - currently untestable.
  • Fix false positives - premature before adding more functions in NetBSD Native Process Plugin.

On the other hand I've fixed a build issue of one test on NetBSD:

Plan for the next milestone

I've listed the following goals for the next milestone.

  • mark exect(3) obsolete in libc
  • remove libpthread_dbg(3) from the base distribution
  • add new API in ptrace(2) PT_SET_SIGMASK and PT_GET_SIGMASK
  • add new API in ptrace(2) to resume and suspend a specific thread
  • finish switch of the PT_WATCHPOINT API in ptrace(2) to PT_GETDBREGS & PT_SETDBREGS
  • validate i386, amd64 and Xen proper support of new interfaces
  • upstream to LLDB accessors for debug registers on NetBSD/amd64
  • validate PT_SYSCALL and add a functionality to detect and distinguish syscall-entry syscall-exit events
  • validate accessors for general purpose and floating point registers

Post mortem

FreeBSD is catching up after NetBSD changes, e.g. with the following commit:

This move allows to introduce further reduction of code-duplication. There still is a lot of room for improvement. Another benefit for other software distributions, is that they can now appropriately resolve the six.py conflict without local patches.

These examples clearly show that streamlining NetBSD code results in improved support for other systems and creates a cleaner environment for introducing new platforms.

A pure NetBSD-oriented gain is improvement of system interfaces in terms of quality and functionality, especially since DTrace/NetBSD is a quick adopter of new interfaces.. and indirectly a sandbox to sort out bugs in ptrace(2).

The tasks in the next milestone will turn NetBSD's ptrace(2) to be on par with Linux and FreeBSD, this time with marginal differences.

To render it more clearly NetBSD will have more interfaces in read/write mode than FreeBSD has (and be closer to Linux here), on the other hand not so many properites will be available in a thread specific field under the PT_LWPINFO operation that caused suspension of the process.

Another difference is that FreeBSD allows to trace only one type of syscall events: on entry or on exit. At the moment this is not needed in existing software, although it's on the longterm wishlist in the GDB project for Linux.

It turned out that, I was overly optimistic about the feature set in ptrace(2), while the basic ones from the first milestone were enough to implement basic support in LLDB.. it would require me adding major work in heuristics as modern tracers no longer want to perform guessing what might happened in the code and what was the source of signal interruption.

This was the final motivation to streamline the interfaces for monitoring capabilities and now I'm adding remaining interfaces as they are also needed, if not readily in LLDB, there is DTrace and other software that is waiting for them now. Somehow I suspect that I will need them in LLDB sooner than expected.

This work was sponsored by The NetBSD Foundation.

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue to fund projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can:

http://netbsd.org/donations/#how-to-donate

Posted in the wee hours of Monday night, February 14th, 2017 Tags: blog

Background

I am using a sparc64 Sun Blade 2500 (silver) as a desktop machine - for my pretty light "desktop" needs. Besides the usual developer tools (editors, compilers, subversion, hg, git) and admin stuff (all text based) I need mpg123 and mserv for music queues, Gimp for image manipulation and of course Firefox.

Recently I updated all my installed pkgs to pkgsrc-current and as usual the new Firefox version failed to build.

Fortunately the issues were minor, as they all had been handled upstream for Firefox 52 already, all I needed to do was back-porting a few fixes. This made the pkg build, but after a few minutes of test browsing, it crashed. Not surprisingly this was reproducible, any web site trying to play audio triggered it. A bit surprising though: the same happened on an amd64 machine I tried next. After a bit digging the bug was easy to fix, and upstream already took the fix and committed it to the libcubeb repository.

Success!

So I am now happily editing this post using Firefox 51 on the Blade 2500.

I saw one crash in two days of browsing, but unfortunately could not (yet) reproduce it (I have gdb attached now). There will be future pkg updates certainly.

Future Obstacles

You may have read elsewhere that Firefox will start to require a working Rust compiler to build.

This is a bit unfortunate, as Rust (while academically interesting) is right now not a very good implementation language if you care about portability. The only available compiler requires a working LLVM back end, which we are still debugging. Our auto-builds produce sparc sets with LLVM, but the result is not fully working (due to what we believe being code gen bugs in LLVM). It seems we need to fix this soon (which would be good anyway, independent of the Rust issue). Besides the back end, only very recently traces of sparc64 support popped up in Rust. However, we still have a few firefox versions time to get it all going. I am optimistic.

Another upcoming change is that Cairo (currently used as 2D graphics back end, at least on sparc64) will be phased out and Skia will be the only supported software rendering target. Unfortunately Skia does (as of now) not support any big endian machine at all. I am looking for help getting Skia to work on big endian hardware in general, and sparc64 in particular.

Alternatives

Just in case, I tested a few other browsers and (so far) they all failed:
  • NetSurf
    Nice, small, has a few "tweaks" and does not yet support JavaScript good enough for many sites
  • Midori
    They call it "lightweight" but it is based on WebKit, which alone is a few times more heavy than all of Firefox. It crashes immediately at startup on sparc64 (I am investigating, but with low priority - actually I had to replace the hard disk in my machine to make enough room for the debug object files for WebKit - it takes ~20GB)

So, while it is a bit of a struggle to keep a modern browser working on my favorite odd-ball architecture, it seems we will get at least to the Firefox 52 ESR release, and that should give us enough time to get Rust working and hopefully continue with Firefox.

Posted Wednesday afternoon, February 8th, 2017 Tags: blog
Operating systems can be called monitors as they handle system calls from userland processes. A similar task is performed by debuggers as they implement monitors for traced applications and interpret various events that occurred in tracees and are messaged usually with signals to their tracers. During this month I have started a new Process Plugin within LLDB to incept NativeProcessNetBSD - copied from NativeProcessLinux - implementing basic functionality and handling all the needed events in the MonitorCallback() function. To achieve these tasks, I had to add a bunch of new ptrace(2) interfaces in the kernel to cover all that is required by LLDB monitors. The current Process Plugin for NetBSD is capable to start a process, catch all the needed events correctly and if applicable resume or step the process.

What has been done in NetBSD

1. Verified the full matrix of combinations of wait(2) and ptrace(2) in the following test-cases

  • verify basic PT_GET_SIGINFO call for SIGTRAP from tracee
  • verify basic PT_GET_SIGINFO and PT_SET_SIGINFO calls without modification of SIGINT from tracee
  • verify basic PT_GET_SIGINFO and PT_SET_SIGINFO calls with setting signal to new value
  • detect SIGTRAP TRAP_EXEC from tracee
  • verify single PT_STEP call with signal information check
  • verify that fork(2) is intercepted by ptrace(2) with EVENT_MASK set to PTRACE_FORK and reports correct signal information
  • verify that vfork(2) is intercepted by ptrace(2) with EVENT_MASK set to PTRACE_VFORK_DONE
  • verify that vfork(2) is intercepted by ptrace(2) with EVENT_MASK set to PTRACE_FORK | PTRACE_VFORK_DONE
  • verify that PTRACE_VFORK in EVENT_MASK is preserved
  • verify that PTRACE_VFORK_DONE in EVENT_MASK is preserved
  • verify that PTRACE_LWP_CREATE in EVENT_MASK is preserved
  • verify that PTRACE_LWP_EXIT in EVENT_MASK is preserved
  • verify that thread creation is intercepted by ptrace(2) with EVENT_MASK set to PTRACE_LWP_CREATE
  • verify that thread termination is intercepted by ptrace(2) with EVENT_MASK set to PTRACE_LWP_EXIT

2. GNU libstdc++ std::call_once bug investigation

The LLVM toolchain internally uses several std::call_once calls, within LLVM and LLDB codebase. The GNU libstdc++ library implements C++11 std::call_once with wrapping pthread_once(3) and a function pointer passed as a TLS (Thread Local Storage) variable. Currently this setup isn't functional with shared linking on NetBSD and apparently on few other systems and hardware platforms. This issue is still unresolved and its status is traced in PR 51139. As of now, there is a walkaround originally prepared for NetBSD, and adapted for others - namely llvm::call_once.

3. Improving documentation and other minor system parts

  • rename the SIGPOLL signal to SIGIO in the documentation of siginfo(2)
  • replace SIGPOLL references with SIGIO in comments of <sys/siginfo.h>
  • document SI_NOINFO in siginfo(2)
  • document SI_LWP in siginfo(2)
  • document SI_QUEUE in siginfo(2)
  • document SI_MESGQ in siginfo(2)
  • document TRAP_EXEC in siginfo(2)
  • document TRAP_CHLD in siginfo(2)
  • document TRAP_LWP in siginfo(2)
  • reference siginfo(2) for a SIGCHLD signal in ptrace(2)
  • remove unused macro for ATF_TP_ADD_TC_HAVE_DBREGS in src/tests/kernel/t_ptrace_wait.h
  • cleanup commented out code after revert of racy vfork(2) commit from year 2012
  • remove stub code to redefine PTRACE_FORK in ATF tests
  • ... and several microimprovements in the codebase

4. Documentation of ptrace(2) and explanation how debuggers work

  • document addr and data argument usage for PT_GET_PROCESS_STATE
  • document PT_SET_SIGINFO and PT_GET_SIGINFO
  • explain execve(2) handling and behavior, SIGTRAP & TRAP_EXEC
  • reference PaX MPROTECT restrictions for debuggers
  • explain software breakpoints handling and behavior, SIGTRAP & TRAP_BKPT
  • explain single step behavior, SIGTRAP & TRAP_TRACE
  • explain that PT_TRACE_ME does not send a SIGSTOP signal
  • list predefined MI symbols for help debuggers in port specific headers
  • document the current behavior of TRAP_CHLD
  • add more notes on PTRACE_FORK events
  • document PTRACE_VFORK and PTRACE_VFORK_DONE
  • document PTRACE_LWP_CREATE and PTRACE_LWP_EXIT
  • import HISTORY notes from FreeBSD

5. Introduction of new siginfo(2) codes for SIGTRAP

  • TRAP_EXEC - catch exec() events transforming the calling process into a new process
  • TRAP_CHLD - process child trap (fork, vfork and vfork done events)
  • TRAP_LWP - process thread traps (birth, termination)
  • TRAP_HWWPT - process hardware assisted watchpoints

6. New ptrace(2) interfaces

There were added new interfaces to the native ptrace(2) NetBSD API.

Interface to introspect and fake signal information

The PT_GET_SIGINFO interface is designed to read signal information sent to tracee. A program can also fake a signal to be passed to a debugged process, with included destination thread or the whole process. A debugger requires detailed signal information in order to distinguish the exact trap reason easily and precisely, for example whether a program stopped on a software defined breakpoint or a single step trap.

This interface is modeled after the Linux specific calls: PTRACE_GETSIGINFO and PTRACE_SETSIGINFO, but adapted for the NetBSD use-case. FreeBSD currently has no way to fake signal information.

I have modeled this interface to be most efficient in terms of determination what exact thread received the signal. Thanks to it, a process does not need to examine each thread separately and performs only a single ptrace(2) call. This makes a significant difference in a massively threaded software.

The signal information accessors introduce a new structure ptrace_siginfo:

/*
 * Signal Information structure
 */
typedef struct ptrace_siginfo {
       siginfo_t       psi_siginfo;    /* signal information structure */
       lwpid_t         psi_lwpid;      /* destination LWP of the signal
                                        * value 0 means the whole process
                                        * (route signal to all LWPs) */
} ptrace_siginfo_t;
I've included in the <sys/ptrace.h> header required file to define the siginfo_t type - <sys/siginfo.h>. This makes sure that no existing software will break during build due to a missing type definition.

Interface to monitor vfork(2) operations

Forking a process is one of the fundamental design features of a UNIX operating system. There are two basic types of forks: fork(2) and vfork(2) ones. The fork(2) one is designed to spawn a mostly independent child from parent's memory layout, however this operation is costy. In order to address it and optimize the forking operation there is vfork(2) which shares address space with its parent. From the ptrace(2) point of view, the difference between fork(2) and vfork(2) calls is whether a parent giving birth to its child is suspended waiting on child termination or exec() operation or not.

Currently there is an interface to monitor fork(2) operations - PTRACE_FORK - however NetBSD missed the vfork(2) ones. I've added two new functions: PTRACE_VFORK and PTRACE_VFORK_DONE. The former is supposed to notify why a parent gives a vfork(2)-like birth to its child and later when a child is born with vfork(2) from its parent - perfoming a handshake with two SIGTRAP signals emitted from the kernel. Once the parent is resuming it gets a notification for vfork(2) done.

These events throw SIGTRAP signal with the TRAP_CHLD property. Currently PTRACE_VFORK is a stub and it's planned to be fully implemented later.

Interface to monitor thread operations

A debugger requires an interface to monitor thread birth and termination. This is needed in order to properly track the thread list of a process and ensure that every thread has for example applied watchpoints.

I've added two events:

  • PTRACE_LWP_CREATE - to report thread births,
  • PTRACE_LWP_EXIT - to report thread termination.

This interface reuses the EVENT_MASK and PROCESS_STATE interface. It means that it shares these calls with PTRACE_FORK, PTRACE_VFORK and PTRACE_VFORK_DONE.

To achieve this goal, I've changed the following structure:

typedef struct ptrace_state {
        int     pe_report_event;
        pid_t   pe_other_pid;
} ptrace_state_t;

to

typedef struct ptrace_state {
        int     pe_report_event;
        union {
                pid_t   _pe_other_pid;
                lwpid_t _pe_lwp;
        } _option;
} ptrace_state_t;

#define pe_other_pid    _option._pe_other_pid
#define pe_lwp          _option._pe_lwp
This change keeps the size of ptrace_state_t unchanged as both pid_t and lwpid_t are defined as an int32_t-like integer. New struct form should not break existing software and be source and binary compatible with it.

I've introduced a new SIGTAP type for thread events: TRAP_LWP.

Hardware assisted watchpoints

I've introduced a few changes to the current interface. One of them is allowing to mix single-step operation with enabled hardware assisted watchpoints. The other one was added new extension pw_type to the ptrace_watchpoint structure.

6. Updated doc/TODO.ptrace entries

The current state of TODO.ptrace - after several updates - is as follows:
  • verify ppid of core dump generated with PT_DUMPCORE it must point to the real parent, not tracer
  • adapt OpenBSD regress test (regress/sys/ptrace/ptrace.c) for the ATF context
  • add new ptrace(2) calls to lock (suspend) and unlock LWP within a process
  • add PT_DUMPCORE tests in the ATF framework
  • add ATF tests for PT_WRITE_I and PIOD_WRITE_I - test mprotect restrictions
  • add ATF tests for PIOD_READ_AUXV
  • once the API for hardware watchpoints will stabilize, document it
  • add tests for the procfs interface covering all functions available on the same level as ptrace(2)
  • add support for PT_STEP, PT_GETREGS, PT_SETREGS, PT_GETFPREGS, PT_SETFPREGS in all ports
  • integrate all ptrace(2) features in gdb
  • add ptrace(2) NetBSD support in LLDB
  • add proper implementation of PTRACE_VFORK for vfork(2)-like events
  • remove exect(3) - there is no usecase for it
  • refactor pthread_dbg(3) to only query private pthread_t data, otherwise it duplicates ptrace(2) interface and cannot cover all types of threads
  • add ATF tests for SIGCHLD
  • add ATF tests for PT_SYSCALL and PT_SYSCALLEMU

Features in ELF, DWARF, CTF, DTrace are out of scope for the above list.

7. Future directions

After research and testing the current watchpoint interface, I've realized that it's impossible (impractically complicated) to pretend to have a "safe" watchpoint interface inside the kernel, as the current one isn't safe from undefined behavior even on stock amd64. I've decided to revert this code and introduce PT_GETDBREGS and PT_SETDBREGS restricted to INSECURE secure level mode. The good side of this change is that there is already part of the code needed in the kernel, I have a local draft introducing this interface and it will be easier to integrate with LLDB, as Linux and FreeBSD keep having the same interface.

What has been done in LLDB

I work on the LLDB port inside the pkgsrc-wip repository, in the lldb-netbsd package.

To summarize the changes, there were so far 84 commits in this directory. The overall result is a list of 26 patched or added files. The overall diff's length is 3539 lines.

$ wc -l patches/patch-*                                                               
      12 patches/patch-cmake_LLDBDependencies.cmake
      17 patches/patch-cmake_modules_AddLLDB.cmake
      14 patches/patch-include_lldb_Host_netbsd_HostThreadNetBSD.h
      30 patches/patch-include_lldb_Host_netbsd_ProcessLauncherNetBSD.h
      12 patches/patch-source_CMakeLists.txt
      12 patches/patch-source_Host_CMakeLists.txt
      60 patches/patch-source_Host_common_Host.cpp
      13 patches/patch-source_Host_common_NativeProcessProtocol.cpp
      21 patches/patch-source_Host_netbsd_HostThreadNetBSD.cpp
     175 patches/patch-source_Host_netbsd_ProcessLauncherNetBSD.cpp
      23 patches/patch-source_Host_netbsd_ThisThread.cpp
      24 patches/patch-source_Initialization_SystemInitializerCommon.cpp
     803 patches/patch-source_Plugins_Platform_NetBSD_PlatformNetBSD.cpp
     141 patches/patch-source_Plugins_Platform_NetBSD_PlatformNetBSD.h
      12 patches/patch-source_Plugins_Process_CMakeLists.txt
      13 patches/patch-source_Plugins_Process_NetBSD_CMakeLists.txt
    1392 patches/patch-source_Plugins_Process_NetBSD_NativeProcessNetBSD.cpp
     188 patches/patch-source_Plugins_Process_NetBSD_NativeProcessNetBSD.h
     393 patches/patch-source_Plugins_Process_NetBSD_NativeThreadNetBSD.cpp
      92 patches/patch-source_Plugins_Process_NetBSD_NativeThreadNetBSD.h
      13 patches/patch-tools_lldb-mi_MICmnBase.cpp
      13 patches/patch-tools_lldb-mi_MICmnBase.h
      13 patches/patch-tools_lldb-mi_MIDriver.cpp
      13 patches/patch-tools_lldb-mi_MIUtilString.cpp
      13 patches/patch-tools_lldb-mi_MIUtilString.h
      27 patches/patch-tools_lldb-server_CMakeLists.txt
    3539 total

1. Native Process NetBSD Plugin

I've created the initial code for the Native Process NetBSD Plugin.

  • process resume support (PT_CONTINUE)
  • process step support (PT_STEP) - currently not fully functional
  • functional callback monitor
  • functional process launch operation
  • appropriate ptrace(2) wrapper with logging capabilities
  • initial NativeThreadNetBSD code
  • setup for EVENT_MASK of a traced process (we monitor thread events)
  • preliminary code for other functions, like attach to a process and reading/writing memory of a tracee

2. The MonitorCallback function

The MonitorCallback function supports now the following events:

  • process exit and retrieve status code
  • software breakpoint (TRAP_BRKPT)
  • single step (TRAP_TRACE)
  • process image swich trap (TRAP_EXEC)
  • child traps (TRAP_CHLD) - currently detectable but disabled
  • thread trap (TRAP_LWP)
  • hardware assisted watchpoint trap (TRAP_HWWPT)

3. Other LLDB code, out of the NativeProcessNetBSD Plugin

During this work segment I've completed the following tasks:

  • created initial Native Process Plugin for NetBSD with remote debugging facilities, with skeleton copied from Linux but almost every function used so far had to be rewritten for NetBSD.
  • attached the Native Process Plugin for NetBSD to the build infrastructure.
  • disabled existing code for retrieving and updating Thread Name in NatBSD host plugin as currently not compatible with the LLDB codebase.
  • added NetBSD Process Launcher.
  • support proper native NetBSD Thread ID in Host::GetCurrentThreadID.
  • upgraded Platform NetBSD Plugin for new remote debugging plugin.

4. Automated LLDB Test Results Summary

The number of passing tests increased by 45% between devel/lldb 3.9.1 and lldb-netbsd 2017-01-21.

The above graphs renders test results for:

Example LLDB sessions

It's demo time!

Breakpoint interrupt

In this example, I'm calling a hello world application that is triggering a software breakpoint. It's implemented by embedding int3 call on amd64. The debugger is capable of catching this and resuming till correct process termination.

$ lldb ./int3 
(lldb) target create "./int3"
Current executable set to './int3' (x86_64).
(lldb) r
Hello world!
Process 29578 launched: './int3' (x86_64)
Process 29578 stopped
* thread #1, stop reason = signal SIGTRAP
    frame #0:
(lldb) c
Process 29578 resuming
Process 29578 exited with status = 0 (0x00000000)
(lldb)

Thread monitor interrupt

In this example we set trap on thread events - creation and termination. The executed program incepts a thread and terminates afterwards, this is caught by the MonitorCallback with appropriate thread list update. After the end, program terminates correctly and passes proper exit status to the debugger.

$ lldb ./lwp_create
(lldb) target create "./lwp_create"
Current executable set to './lwp_create' (x86_64).
(lldb) r
Process 27331 launched: './lwp_create' (x86_64)
Hello world!
Process 27331 stopped
* thread #1, stop reason = SIGTRAP has been caught with Process LWP Trap type
    frame #0:
  thread #2, stop reason = SIGTRAP has been caught with Process LWP Trap type
    frame #0:
(lldb) thread list 
Process 27331 stopped
* thread #1: tid = 0x0001, stop reason = SIGTRAP has been caught with Process LWP Trap type
  thread #2: tid = 0x0002, stop reason = SIGTRAP has been caught with Process LWP Trap type
(lldb) c
Process 27331 resuming
Process 27331 stopped
* thread #1, stop reason = SIGTRAP has been caught with Process LWP Trap type
    frame #0:
(lldb) thread list
Process 27331 stopped
* thread #1: tid = 0x0001, stop reason = SIGTRAP has been caught with Process LWP Trap type
(lldb) c
Process 27331 resuming
It works
Process 27331 exited with status = 0 (0x00000000)
(lldb)

Plan for the next milestone

I've listed the following goals for the next milestone.

  • fix conflict with system-wide py-six
  • add support for auxv read operation
  • switch resolution of pid -> path to executable from /proc to sysctl(7)
  • recognize Real-Time Signals (SIGRTMIN-SIGRTMAX)
  • upstream !NetBSDProcessPlugin code
  • switch std::call_once to llvm::call_once
  • add new ptrace(2) interface to lock and unlock threads from execution
  • switch the current PT_WATCHPOINT interface to PT_GETDBREGS and PT_SETDBREGS

This work was sponsored by The NetBSD Foundation.

Previous report "Summary of the ptrace(2) project".

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue to fund projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can:

http://netbsd.org/donations/#how-to-donate

Posted at noon on Monday, January 23rd, 2017 Tags: blog

[Update, 2017-01-16 20:48 UTC: All issues have been resolved and NetBSD.org should be functioning normally now.]

As some of you may have noticed, the netbsd.org DNS records are broken right now. Two independent incidents caused this to happen:

  1. UC Berkeley stopped providing secondary DNS service for netbsd.org at adns1.berkeley.edu and adns2.berkeley.edu on a few hours notice, before we had time to update the NS delegations in the ORG. zone. [Update, 2017-01-16 20:48 UTC: The Berkeley network administrators have graciously restored service until we can update our NS delegations. Thanks, Berkeley—for doing this on a US holiday, no less!]
  2. The network hosting our primary nameserver ns.netbsd.de went offline at around 14:00 UTC for reasons unknown. [Update, 2017-01-16 20:48 UTC: The network and ns.netbsd.de are back up.]

We have sent a request to our registrar to update the delegations to make ns.netbsd.org primary and ns.netbsd.de secondary. We asked them to expedite the process, but owing to the weekend, when the humans who process these requests are off work, it may take another day for that to take effect, and another day after that for the cached TTLs on the old NS delegations to Berkeley's nameservers to expire.

As a provisional stop gap, if you need access to the netbsd.org zone and you can prime your DNS cache with manually entered records, you can prime it with the following records to restore service at your site:

netbsd.org.     86400   IN      NS      ns.netbsd.org.
netbsd.org.     86400   IN      NS      ns.netbsd.de.
ns.netbsd.org.  86400   IN      A       199.233.217.200
ns.netbsd.org.  86400   IN      AAAA    2001:470:a085:999::53
  

(Presumably if you are reading this you have some DNS records in the netbsd.org zone cached, but they will eventually expire and you may need some others.)

Posted late Monday afternoon, January 16th, 2017 Tags: blog
The NetBSD Foundation took part in the 2016 Google Summer of Code.

Hrishikesh Goyal worked on the project "Implement Ext4fs support in ReadOnly mode". He tackled two features of an ext4fs implementation: extents and HTree DIR read/write support. His work was committed into the NetBSD source tree in multiple commits.

charles cui worked on adding tests for better "POSIX Test Suite Compliance". This involved porting the Open Posix benchmark suite to NetBSD. Many of the tests showed missing features, and Charles worked with his mentors to improve the results. See his summary for details.

Leonardo Taccari worked on "Split debug symbols for pkgsrc builds". He already blogged about this in much detail on this blog.

A big thank you to Google for sponsoring the students to work on NetBSD, the students for working on the projects, and the mentors that were helping them along!

Posted Saturday afternoon, January 14th, 2017 Tags: blog

The first release candidate of NetBSD 7.1 is now available for download at:

http://cdn.NetBSD.org/pub/NetBSD/NetBSD-7.1_RC1/

Those of you who prefer to build from source can continue to follow the netbsd-7 branch or use the netbsd-7-1-RC1 tag.

There have been quite a lot of changes since 7.0. See src/doc/CHANGES-7.1 for the full list.

Please help us out by testing 7.1_RC1. We love any and all feedback. Report problems through the usual channels (submit a PR or write to the appropriate list). More general feedback is welcome at releng@NetBSD.org.

Posted early Monday morning, January 9th, 2017 Tags: blog
In short, we are already in a good state with the existing ptrace(2) interfaces, as most necessary functions in LLDB are representable by existing NetBSD specific interfaces. We can fully implement core LLDB functionality without further extensions in ptrace(2). During this project dozen of bugs were investigated & fixed and several hundreds of ATF tests added. The major addition is newly added support for hardware assisted watchpoints API for ptrace(2) on amd64 and preliminary code for i386 and XEN.

What has been done

1. Verified basic ptrace(2) functionality:

  • debugger cannot attach to PID0 (as root and user)
  • debugger cannot attach to PID1 (as user)
  • debugger cannot attach to PID1 with sercurelevel >= 1 (as root and user)
  • debugger cannot attach to self
  • debugger cannot attach to another process unless the process's root directory is at or below the tracing process's root

2. Verified the full matrix of combinations of wait(2) and ptrace(2) in the following test-cases

  • tracee can emit PT_TRACE_ME for parent and raise SIGSTOP followed by _exit(2)
  • tracee can emit PT_TRACE_ME for parent and raise SIGSTOP followed by _exit(2), with perent sending SIGINT and catching this singal only once with a signal handler and without termination of tracee
  • tracee can emit PT_TRACE_ME for parent and raise SIGSTOP, with perent sending SIGINT and terminating the child without signal handler
  • tracee can emit PT_TRACE_ME for parent and raise SIGSTOP, with perent sending SIGINT and terminating the child without signal handler
  • tracee can emit PT_TRACE_ME for parent and raise SIGCONT, and parent reports it as process stopped
  • assert that tracer sees process termination earlier than the parent
  • assert that any tracer sees process termination earlier than its parent
  • assert that tracer parent can PT_ATTACH to its child
  • assert that tracer child can PT_ATTACH to its parent
  • assert that tracer sees its parent when attached to tracer (check getppid(2))
  • assert that tracer sees its parent when attached to tracer (check sysctl(7) and struct kinfo_proc2)
  • assert that tracer sees its parent when attached to tracer (check /proc/curproc/status 3rd column)
  • verify that empty EVENT_MASK is preserved
  • verify that PTRACE_FORK in EVENT_MASK is preserved
  • verify that fork(2) is intercepted by ptrace(2) with EVENT_MASK set to PTRACE_FORK
  • verify that fork(2) is not intercepted by ptrace(2) with empty EVENT_MASK
  • verify that vfork(2) is intercepted by ptrace(2) with EVENT_MASK set to PTRACE_VFORK [currently EVENT_VFORK not implemented]
  • verify that vfork(2) is not intercepted by ptrace(2) with empty EVENT_MASK [currently failing as EVENT_VFORK not implemented]
  • verify PT_IO with PIOD_READ_D and len = sizeof(uint8_t)
  • verify PT_IO with PIOD_READ_D and len = sizeof(uint16_t)
  • verify PT_IO with PIOD_READ_D and len = sizeof(uint32_t)
  • verify PT_IO with PIOD_READ_D and len = sizeof(uint64_t)
  • verify PT_IO with PIOD_WRITE_D and len = sizeof(uint8_t)
  • verify PT_IO with PIOD_WRITE_D and len = sizeof(uint16_t)
  • verify PT_IO with PIOD_WRITE_D and len = sizeof(uint32_t)
  • verify PT_IO with PIOD_WRITE_D and len = sizeof(uint64_t)
  • verify PT_READ_D called once
  • verify PT_READ_D called twice
  • verify PT_READ_D called three times
  • verify PT_READ_D called four times
  • verify PT_WRITE_D called once
  • verify PT_WRITE_D called twice
  • verify PT_WRITE_D called three times
  • verify PT_WRITE_D called four times
  • verify PT_IO with PIOD_READ_D and PIOD_WRITE_D handshake
  • verify PT_IO with PIOD_WRITE_D and PIOD_READ_D handshake
  • verify PT_READ_D with PT_WRITE_D handshake
  • verify PT_WRITE_D with PT_READ_D handshake
  • verify PT_IO with PIOD_READ_I and len = sizeof(uint8_t)
  • verify PT_IO with PIOD_READ_I and len = sizeof(uint16_t)
  • verify PT_IO with PIOD_READ_I and len = sizeof(uint32_t)
  • verify PT_IO with PIOD_READ_I and len = sizeof(uint64_t)
  • verify PT_READ_I called once
  • verify PT_READ_I called twice
  • verify PT_READ_I called three times
  • verify PT_READ_I called four times
  • verify plain PT_GETREGS call without further steps
  • verify plain PT_GETREGS call and retrieve PC
  • verify plain PT_GETREGS call and retrieve SP
  • verify plain PT_GETREGS call and retrieve INTRV
  • verify PT_GETREGS and PT_SETREGS calls without changing regs
  • verify plain PT_GETFPREGS call without further steps
  • verify PT_GETFPREGS and PT_SETFPREGS calls without changing regs
  • verify single PT_STEP call
  • verify PT_STEP called twice
  • verify PT_STEP called three times
  • verify PT_STEP called four times
  • verify that PT_CONTINUE with SIGKILL terminates child
  • verify that PT_KILL terminates child
  • verify basic LWPINFO call for single thread (PT_TRACE_ME)
  • verify basic LWPINFO call for single thread (PT_ATTACH from tracer)

3. Documentation of ptrace(2)

  • documented PT_SET_EVENT_MASK, PT_GET_EVENT_MASK and PT_GET_PROCESS_STATE
  • updated and fixed documentation of PT_DUMPCORE
  • documented PT_GETXMMREGS and PT_SETXMMREGS (i386 port specific)
  • documented PT_GETVECREGS and PT_SETVECREGS (ppc ports specific)
  • other tweaks and cleanups in the documentation

4. exect(3) - execve(2) wrapper with tracing capabilities

Researched its usability and added ATF test, it's close to be marked for removal - it's marked as broken as it is on all ports... this call was inherited from BSD4.2 (VAX) and was never useful since the inception as it is enabling singlestepping before calling execve(2) and tracing libc calls before switching to new process image.

5. pthread_dbg(3) - POSIX threads debugging library documentation

  • added documentation for the library in man-page
  • upstreamed to the mandoc project to recognize the pthread_dbg(3) library
  • document td_close(3) close connection to a threaded process
  • document td_map_pth2thr(3) convert a pthread_t to a thread handle
  • document td_open(3) make connection to a threaded process
  • document td_thr_getname(3) get the user-assigned name of a thread
  • document td_thr_info(3) get information on a thread
  • document td_thr_iter(3) iterate over the threads in the process

6. pthread_dbg(3) - pthread debug library - t_dummy tests

  • assert that dummy lookup functions stops td_open(3)
  • assert that td_open(3) for basic proc_{read,write,lookup} works
  • asserts that calling td_open(3) twice for the same process fails

7. pthread_dbg(3) - pthread debug library - test of features

  • assert that td_thr_iter(3) call without extra logic works
  • assert that td_thr_iter(3) call is executed for each thread once
  • assert that for each td_thr_iter(3) call td_thr_info(3) is valid
  • assert that for each td_thr_iter(3) call td_thr_getname(3) is valid
  • assert that td_thr_getname(3) handles shorter buffer parameter and the result is properly truncated
  • assert that pthread_t can be translated with td_map_pth2thr(3) to td_thread_t -- and assert earlier that td_thr_iter(3) call is valid
  • assert that pthread_t can be translated with td_map_pth2thr(3) to td_thread_t -- and assert later that td_thr_iter() call is valid
  • assert that pthread_t can be translated with td_map_pth2thr(3) to td_thread_t -- compare thread's name of pthread_t and td_thread_t
  • assert that pthread_t can be translated with td_map_pth2thr(3) to td_thread_t -- assert that thread is in the TD_STATE_RUNNING state

8. pthread_dbg(3) - pthread debug library - code fixes

  • fix pt_magic (part of pthread_t) read in td_thr_info(3)
  • correct pt_magic reads in td_thr_{getname,suspend,resume}(3)
  • fix pt_magic read in td_map_pth2thr(3)
  • always set trailing '\0' in td_thr_getname(3) to compose valid ASCIIZ string
  • kill SA thread states (TD_STATE_*) in pthread_dbg and add TD_STATE_DEAD
  • obsolete thread_type in td_thread_info_st in pthread_dbg.h

This library was designed for Scheduler Activation, and this feature was removed in NetBSD 5.0.

9. wait(2) family tests

  • test that wait6(2) handled stopped/continued process loop
  • test whether wait(2)-family of functions return error and set ECHILD for lack of children
  • test whether wait(2)-family of functions return error and set ECHILD for lack of children, with WNOHANG option verifying that error is still signaled and errno set

10. Debug Registers assisted watchpoints:

  • fix rdr6() function on amd64
  • add accessors for available x86 Debug Registers (DR0-DR3, DR6, DR7)
  • switch x86 CPU Debug Register types from vaddr_t to register_t
  • torn down KSTACK_CHECK_DR0, i386-only feature to detect stack overflow

12. i386 port tests

  • call PT_GETREGS and iterate over General Purpose registers

13. amd64 port tests

  • call PT_GETREGS and iterate over General Purpose registers
  • call PT_COUNT_WATCHPOINTS and assert four available watchpoints
  • call PT_COUNT_WATCHPOINTS and assert four available watchpoints, verify that we can read these watchpoints
  • call PT_COUNT_WATCHPOINTS and assert four available watchpoints, verify that we can read and write unmodified these watchpoints
  • call PT_COUNT_WATCHPOINTS and test code trap with watchpoint 0
  • call PT_COUNT_WATCHPOINTS and test code trap with watchpoint 1
  • call PT_COUNT_WATCHPOINTS and test code trap with watchpoint 2
  • call PT_COUNT_WATCHPOINTS and test code trap with watchpoint 3
  • call PT_COUNT_WATCHPOINTS and test write trap with watchpoint 0
  • call PT_COUNT_WATCHPOINTS and test write trap with watchpoint 1
  • call PT_COUNT_WATCHPOINTS and test write trap with watchpoint 2
  • call PT_COUNT_WATCHPOINTS and test write trap with watchpoint 3
  • call PT_COUNT_WATCHPOINTS and test write trap with watchpoint 0
  • call PT_COUNT_WATCHPOINTS and test rw trap with watchpoint 0 on data read
  • call PT_COUNT_WATCHPOINTS and test rw trap with watchpoint 1 on data read
  • call PT_COUNT_WATCHPOINTS and test rw trap with watchpoint 2 on data read
  • call PT_COUNT_WATCHPOINTS and test rw trap with watchpoint 3 on data read

14. Other minor improvements

  • document in wtf(7) PCB process control block
  • fix cpu_switchto(9) prototype in a comment in RICSV

15. Add support for hardware assisted watchpoints/breakpoints API in ptrace(2)

Add new ptrace(2) calls:
  1. PT_COUNT_WATCHPOINTS - count the number of available hardware watchpoints
  2. PT_READ_WATCHPOINT - read struct ptrace_watchpoint from the kernel state
  3. PT_WRITE_WATCHPOINT - write new struct ptrace_watchpoint state, this includes enabling and disabling watchpoints
The ptrace_watchpoint structure contains MI and MD parts:
typedef struct ptrace_watchpoint {
	int		pw_index;	/* HW Watchpoint ID (count from 0) */
	lwpid_t		pw_lwpid;	/* LWP described */
	struct mdpw	pw_md;		/* MD fields */
} ptrace_watchpoint_t;
For example amd64 defines MD as follows:
struct mdpw {
	void	*md_address;
	int	 md_condition;
	int	 md_length;
};

These calls are protected with the __HAVE_PTRACE_WATCHPOINTS guard.

Tested on amd64, initial support added for i386 and XEN.

16. Reported bugs:

  • PR kern/51596 (ptrace(2): raising SIGCONT in a child results with WIFCONTINUED and WIFSTOPPED true in the parent)
  • PR kern/51600 (Tracer must detect zombie before child's parent)
  • PR standards/51603 WIFCONTINUED()=true always implies WIFSTOPPED()=true
  • PR standards/51606 wait(2) with WNOHANG does not return errno ECHILD for lack of children
  • PR kern/51621: PT_ATTACH from a parent is unreliable [false positive]
  • PR kern/51624: Tracee process cannot see its appropriate parent when debugged by a tracer
  • PR kern/51630 ptrace(2) command PT_SET_EVENT_MASK: option PTRACE_VFORK unsupported
  • PR lib/51633 tests/lib/libpthread_dbg/t_dummy unreliable [false positive]
  • PR lib/51635 td_thr_iter in seems broken [false positive]
  • PR lib/51636: It's not possible to run under gdb(1) programs using the pthread_dbg library
  • PR kern/51649: PRIxREGISTER and PTRACE_REG_* mismatch
  • PR kern/51685 (ptrace(2): Signal does not set PL_EVENT_SIGNAL in (struct ptrace_lwpinfo.)pl_event)
  • PR port-amd64/51700 exect(3) misdesigned and hangs

.... and several critical ones not reported and fixed directly by the NetBSD team.

Credit for Christos Zoulas and K. Robert Elz for helping with the mentioned bugs.

17. Added doc/TODO.ptrace entries

  • verify ppid of core dump generated with PT_DUMPCORE it must point to the real parent, not tracer
  • adapt OpenBSD regress test (regress/sys/ptrace/ptrace.c) for the ATF context
  • add new ptrace(2) calls to lock (suspend) and unlock LWP within a process
  • add PT_DUMPCORE tests in the ATF framework
  • add ATF tests for PT_WRITE_I and PIOD_WRITE_I - test mprotect restrictions
  • add ATF tests for PIOD_READ_AUXV
  • add tests for the procfs interface covering all functions available on the same level as ptrace(2)
  • add support for PT_STEP, PT_GETREGS, PT_SETREGS, PT_GETFPREGS, PT_SETFPREGS in all ports
  • integrate all ptrace(2) features in gdb
  • add ptrace(2) NetBSD support in LLDB
  • add support for detecting equivalent events to PTRACE_O_TRACEEXEC, PTRACE_O_TRACECLONE, PTRACE_O_TRACEEXIT from Linux
  • exect(3) rething or remove -- maybe PT_TRACE_ME + PTRACE_O_TRACEEXEC?
  • refactor pthread_dbg(3) to only query private pthread_t data, otherwise it duplicates ptrace(2) interface and cannot cover all types of threads

Features in ELF, DWARF, CTF, DTrace are out of scope of the above list.

Plan for the coming weeks

My initial goal is to copy the Linux Process Plugin and add minimal functional support for NetBSD in LLDB. It will be followed with running and passing some tests from the lldb-server test-suite.

This is a shift from the original plan about porting FreeBSD Process Plugin to NetBSD, as the FreeBSD one is lacking remote debugging support and it needs to be redone from scratch.

I'm going to fork wip/lldb-git (as for git snapshot from 16 Dec 2016) for the purpose of this task to wip/lldb-netbsd and work there.

Next steps after finishing this task are to sync up with Pavel from the LLDB team after New Year. The NetBSD Process Plugin will be used as a reference to create new Common Process Plugin shared between Linux and (Net)BSD.

This work was sponsored by The NetBSD Foundation.

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue to fund projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can:

http://netbsd.org/donations/#how-to-donate

Posted Sunday afternoon, December 18th, 2016 Tags: blog

The NetBSD Project is pleased to announce NetBSD 7.0.2, the second security/bugfix update of the NetBSD 7.0 release branch. It represents a selected subset of fixes deemed important for security or stability reasons. If you are running an earlier release of NetBSD, we strongly suggest updating to 7.0.2.

For more details, please see the release notes.

Complete source and binaries for NetBSD are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, SUP, and other services may be found at http://www.NetBSD.org/mirrors/.

Posted in the wee hours of Wednesday night, October 27th, 2016 Tags: blog
Greetings,


The NetBSD Foundation has initiated a contract with Kamil Rytarowski
 to complete the following during it:

   1.  Add missing interfaces in ptrace(2), mostly sync it with
       the FreeBSD capabilities, add ATF tests, add documentation.

   2.  Develop process plugin in LLDB based on the FreeBSD code in
       LLDB and make it functional (start passing at least some tests).

   3.  Revamp the process plugin in LLDB for new remote debugging
       capabilities (in order to get it accepted and merged upstream),
       pass more tests.

   4.  LLDB x86 support, pass more of the standard LLDB tests
       and import LLDB to the NetBSD base. Add some ATF LLDB basic
       functionality tests to the tree. The original tests are
       unreliable and generate false positives.

   5.  Develop missing features needed for .NET (POSIX robust
       mutexes), add ATF tests.

   6.  Develop missing features for VirtualBox as host, including
       needing sigevent(2) on par with POSIX and SIGEV_KEVENT, and
       other real-time AIO related interfaces as needed.

   7.  Port Apple's Swift programming language. Enhance .NET port to
       validate new interface and correct more issues as needed.

   8.  Improve VirtualBox host support. Make it build first by disabling
       missing features of providing empty facades for them.

   9.  Implement CDROM, floppy, NIC support for NetBSD in VBox as host.

  10.  Make VirtualBox runnable at least with a restricted feature set, ship
        it in pkgsrc, and submit it upstream.


The NetBSD Foundation will continue to work diligently with the
community to fund projects furthering specific key and quality
improvements to the NetBSD projects.  We have a list of projects at
http://wiki.netbsd.org/projects/ as well as welcome other proposals
to move our flag forward to next releases!

Thank you to Kamil for committing to it and we all look forward to it!

The NetBSD Foundation is a non-profit organization and welcomes any
donations to help us continue to fund projects and services to the
open-source community.  Please consider visiting the following URL,
and chip in what you can:

		http://netbsd.org/donations/#how-to-donate





			Submitted for The NetBSD Foundation,
				William J. Coldwell (billc@)
				President, Chairperson

20161012: edited to fix #10 to be the correct wording.
Posted Wednesday night, October 12th, 2016 Tags: blog
Please join us in welcoming the new Directors to the NetBSD Foundation Board:

Michael van Elst, Taylor R. Campbell, Thomas Klausner

We'd like to sincerely thank the departing board members for their service during their term:

Matthew Sporleder, SAITOH Masanobu, Christos Zoulas

Christos will be remaining as Secretary and Treasurer for the Foundation.


Your 2016-2017 Directors of the Foundation are:
Erik Berls
Taylor R. Campbell
William J. Coldwell
Michael van Elst
Thomas Klausner
Jeremy C. Reed
S. P. Zeidler

The current office holders of the Foundation are:
President: William J. Coldwell
Vice President: Jeremy C. Reed
Secretary: Christos Zoulas
Treasurer: Christos Zoulas

Thank you to all of the developers that nominated and voted, the NomCom, the Voting Administrator and Voting Validator.

Respectfully submitted for the Board of Directors,
William J. Coldwell

Posted late Thursday afternoon, October 6th, 2016 Tags: blog
Add a comment
Contact | Disclaimer | Copyright © 1994-2017 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
NetBSD® is a registered trademark of The NetBSD Foundation, Inc.