I have introduced changes that make debuggers more reliable in threaded scenarios. Additionally, I have enhanced Leak Sanitizer support and introduced various improvements in the basesystem.

Threading support

Threads and synchronization in the kernel, in general, is an evergreen task of the kernel developers. The process of enhancing support for tracing multiple threads has been documented by Michal Gorny in his LLDB entry Threading support in LLDB continued.

Overall I have introduced these changes:

• Separate suspend from userland (_lwp_suspend(2)) flag from suspend by a debugger (PT_SUSPEND). This removes one of the underlying problems of threading stability as a debuggee was able to accidentally unstop suspended thread. This property is needed whenever we want to trace a selection (typically single entity) of threads.
• Store SIGTRAP event information inside siginfo_t, rather than in struct proc. A single signal can only be reported at the time to the debugger, and its context is no longer prone to be overwritten by concurrent threads.
• Change that introduces restarts in functions notifying events for debuggers. There was a time window between registering an event by a thread, stopping the process and unlocking mutexes of the process; as another process could take the mutexes before being stopped and overwrite the event with its own data. Now each event routine for debugger checks whether a process is already stopping (or demising or no longer being tracked) and preserves the signal to be emitted locally in the context of the lwp local variable on the stack and continues stopping self as requested by the other LWP. Once the thread is awaken, it retries to emit the signal and deliver the event signal to the debugger.
• Introduce PT_STOP, that combines kill(SIGSTOP) and ptrace(PT_CONTINUE,SIGSTOP) semantics in a single call. It works like:
  • kill(SIGSTOP) for unstopped tracee
  • ptrace(PT_CONTINUE,SIGSTOP) for stopped tracee
  The child will be stopped and always possible to be waited (with wait(2) like calls).

  For stopped tracee kill(SIGSTOP) has no effect. PT_CONTINUE+SIGSTOP cannot be used on an unstopped process (EBUSY).

  This operation is modeled after PT_KILL that is similar for the SIGKILL call. While there, allow PT_KILL on unstopped traced child.

  This operation is useful in an abnormal exit of a debugger from a signal handler, usually followed by waitpid(2) and ptrace(PT_DETACH).

For the sake of tracking the missed in action signals emitted by tracee, I have introduced the feature in NetBSD truss (as part of the picotrace repository) to register syscall entry (SCE) and syscall exit (SCX) calls and track missing SCE/SCX events that were never delivered. Unfortunately, the number of missing events was huge, even for simple 2-threaded applications.

    truss[2585] running for 22.205305922 seconds
    truss[2585] attached to child=759 ('firefox') for 22.204289369 seconds
    syscall                     seconds      calls     errors missed-sce missed-scx
    read                    0.048522952        609          0         54         76
    write                   0.044693735        487          0         35         66
    open                    0.002516815         18          0          5          5
    close                   0.001015263         17          0          9          6
    unlink                  0.001375463         13          0          3          0
    getpid                  0.093458089       1993          0         16         56
    geteuid                 0.000049301          1          0          0          1
    recvmsg                 0.343353019       4828       3685         90        112
    access                  0.001450653         12          3          5          4
    dup                     0.000570904         10          0          0          1
    munmap                  0.010375949         88          0          6          3
    mprotect                0.196781932       2251          0         11         62
    madvise                 0.049820002        430          0         11         18
    writev                  0.237488362       1507          0         76         67
    rename                  0.000379918          2          0          1          0
    mkdir                   0.000283846          2          2          1          2
    mmap                    0.033342935        481          0         15         40
    lseek                   0.003341775         62          0         25         24
    ftruncate               0.000507707          9          0          1          0
    __sysctl                0.000144506          2          0          0          0
    poll                   18.694195617       4531          0        106        191
    __sigprocmask14         0.001585329         20          0          0          2
    getcontext              0.000083238          1          0          0          0
    _lwp_create             0.000104646          1          0          0          0
    _lwp_self               0.001456718         22          0         24         79
    _lwp_unpark             0.035319633        607          0         14         39
    _lwp_unpark_all         0.020660377        250          0         38         50
    _lwp_setname            0.000118418          2          0          0          0
    __select50             15.125525493        637          0         82        125
    __gettimeofday50        3.279021049       2930          0         40        135
    __clock_gettime50      10.673311747      33132          0       1418       3003
    __stat50                0.006375356         52          3         12          5
    __fstat50               0.001490944         17          0          3          2
    __lstat50               0.000110906          1          0          1          0
    __getrusage50           0.008863815        109          0          7          1
    ___lwp_park60          62.720893458        964        251        454        453
                          -------------    -------    -------    -------    -------
                          111.638589870      56098       3944       2563       4628

With my kernel changes landed, the number of missed sce/scx events is down to zero (with exceptions to signals that e.g. never return such as the exit(2) call).

Once these changes settle in HEAD, I plan to backport them to NetBSD-9. I have already received feedback that GDB works much better now.

The kernel also has now more runtime asserts that validate correctness of the code paths.

Sanitizers

I've introduced a special preprocessor macro to detect LSan (__SANITIZE_LEAK__) and UBSan (__SANITIZE_UNDEFINED__) in GCC. The patches were submitted upstream to the GCC mailing list, in two patches (LSan + UBSan). Unfortunately, GCC does not see value in feature parity with LLVM and for the time being it will be a local NetBSD specific GCC extension. These macros are now integrated into the NetBSD public system headers, for use by the basesystem software.

The LSan macro is now used inside the LLVM codebase and the ps(1) program is the first user of it. The UBSan macro is now used to disable relaxed alignment on x86. While such code is still functional, it is not clean from undefined behavior as specified by C. This is especially needed in the kernel fuzzing process, as we can reduce noise from less interesting reports.

During the previous month a number of reports from kernel fuzzing were fixed. There is still more to go.

Almost all local patches needed for LSan were merged upstream. The last remaining local patch is scheduled for later as it is very invasive for all platforms and sanitizers. In the worst case we just have more false negatives in detection of leaks in specific scenarios.

Miscellaneous changes

I have fixed a regression in upstream GDB with SIGTTOU handling. This was an upstream bug fixed by Alan Hayward and cherry-picked by me. As a side effect, a certain environment setup would cause the tracer to sleep.

I have reverted the regression in changed in6_addr change. It appeased UBSan, but broke at least qemu networking. The regression was tracked down by Andreas Gustafsson and reported in the NetBSD's bug tracking system.

I have landed a patch that returns ELF loader dl_phdr_info information for dl_iterate_phdr(3). This synchronized the behavior with Linux, FreeBSD and OpenBSD and is used by sanitizers.

I have passed through core@ the patch to change the kevent::udata type from intptr_t to void*. The former is slightly more pedantic, but the latter is what is in all other kevent users and this mismatch of types affected specifically C++ users that needed special NetBSD-only workarounds.

I have marked git and hg meta files as ones to be ignored by cvs import. This was causing problems among people repackaging the NetBSD source code with other VCS software than CVS.

I keep working on getting GDB test-suite to run on NetBSD, I spent some time on getting fluent in the TCL programming language (as GDB uses dejagnu and TCL scripting). I have already fixed two bugs that affected NetBSD users in the TCL runtime: getaddrbyname_r and gethostbyaddr_r were falsely reported as available and picked on NetBSD, causing damage in operation. Fluency in TCL will allow me to be more efficient in addressing and debugging failing tests in GDB and likely reuse this knowledge in other fields useful for the project.

I made __CTASSERT a static assert again. Previously, this homegrown check for compile-time checks silently stopped working for C99 compilers supporting VLA (variable length array). It was caught by kUBSan that detected VLA of dynamic size of -1, that is still compatible but has unspecified runtime semantics. The new form is inspired by the Perl ctassert code and uses bit-field constant that enforces the assert to be effective again. Few misuses __CTASSERT, mostly in the Linux DRMKMS code, were fixed.

I have submitted a proposal to the C Working Group a proposal to add new methods for setting and getting the thread name.

Plan for the next milestone

Keep stabilizing the reliability debugging interfaces and get ATF and LLDB threading code reliably pass tests. Cover more scenarios with ptrace(2) in the ATF regression test-suite.

This work was sponsored by The NetBSD Foundation.

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can:

http://netbsd.org/donations/#how-to-donate

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support. You can read more about that in my July 2019 report.

I've been on vacation in August, and in September I've resumed the work on LLDB. I've started by fixing new regressions in LLVM suite, then improved my previous patches and continued debugging test failures and timeouts resulting from my patches.

LLVM 8 and 9 in NetBSD

Updates to LLVM 8 src branch

I have been asked to rebase my llvm8 branch of NetBSD src tree. I've done that, and updated it to LLVM 8.0.1 while at it.

LLVM 9 release

The LLVM 9.0.0 final has been tagged in September. I have been doing the pre-release testing for it, and discovered that the following tests were hanging:

LLVM :: ExecutionEngine/MCJIT/eh-lg-pic.ll
LLVM :: ExecutionEngine/MCJIT/eh.ll
LLVM :: ExecutionEngine/MCJIT/multi-module-eh-a.ll
LLVM :: ExecutionEngine/OrcMCJIT/eh-lg-pic.ll
LLVM :: ExecutionEngine/OrcMCJIT/eh.ll
LLVM :: ExecutionEngine/OrcMCJIT/multi-module-eh-a.ll

I couldn't reproduce the problem with LLVM trunk, so I've instead focused on looking for a fix. I've came to the conclusion that the problem was fixed through adding missing linked library. I've requested backport in bug 43196 and it has been merged in r371042.

I didn't put more effort into figuring out why the lack of this linkage caused issues for us. However, as Lang Hames said on the bug, ‘adding the dependency was the right thing to do’.

LLVM 9 for NetBSD src

Afterwards, I have started working on updating my NetBSD src branch for LLVM 9. However, in middle of that I've been informed that Joerg has already finished doing that independently, so I've stopped.

Furthermore, I was informed that LLVM 9.0.0 will not make it to src, since it still lacks some fixes (most notably, adding a pass to lower is.constant and objectsize intrinsics). Joerg plans to import some revision of the trunk instead.

Buildbot regressions

Initial regressions

The first problem that needed solving was LLDB build failure caused by replacing std::once_flag with llvm::once_flag. I've came to the conclusion that the build fails because the call site in LLDB combined std::call_once with llvm::once_flag. The solution was to replace the former with llvm::call_once.

After fixing the build failure, we had a bunch of test failures on buildbot to address. Kamil helped me and tracked one of them down to a new test for stack exhaustion handling. The test author decided that it ‘is only a best-effort mitigation for the case where things have already gone wrong’, and marked it unsupported on NetBSD.

On the plus side, two of the tests previously failing on NetBSD have been fixed upstream. I've un-XFAIL-ed them appropriately. Five new test failures in LLDB were related to those tests being unconditionally skipped before — I've marked them XFAIL pending further investigation in the future.

Another set of issues was caused by enabling -fvisibility=hidden for libc++ which caused problems when building with GCC. After being pinged, the author decided to enable it only for builds done using clang.

New issues through September

During September, two new issues arose. The first one was my fault, so I'm going to cover it in appropriate section below. The second one was new thread_local test failing. Since it was a newly added test that failed on most of the supported platforms, I've just added NetBSD to the list of failing platforms.

Current buildbot status

After fixing the immediate issues, the buildbot returned to previous status. The majority of tests pass, with one flaky test repeatedly timing out. Normally, I would skip this specific test in order to have buildbot report only fresh failures. However, since it is threading-related I'm waiting to finish my threading update and reassert afterwards.

Furthermore, I have added --shuffle to lit arguments in order to randomize the order in which the tests are run. According to upstream, this reduces the chance of load-intensive tests being run simultaneously and therefore causing timeouts.

The buildbot host seems to have started crashing recently. OpenMP tests were causing similar issues in the past, and I'm currently trying to figure out whether they are the culprit again.

__has_feature(leak_sanitizer)

Kamil asked me to implement a feature check for leak sanitizer being used. The __has_feature(leak_sanitizer) preprocessor macro is complementary to __SANITIZE_LEAK__ used in NetBSD gcc and is used to avoid reports when leaks are known but the cost of fixing them exceeds the gain.

Progress in threading support

Fixing LLDB bugs

In the course of previous work, I had a patch for threading support in LLDB partially ready. However, the improvements have also resulted in some of the tests starting to hang. The main focus of my late work as investigating those problems.

The first issue that I've discovered was inconsistency in expressing no signal sent. In some places, LLDB used LLDB_INVALID_SIGNAL (-1) to express that, in others it used 0. So far this went unnoticed since the end result in ptrace calls was the same. However, the reworked NetBSD threading support used explicit PT_SET_SIGINFO which — combined with wrong signal parameter — wiped previously queued signal.

I've fixed C packet handler, then fixed c, vCont and s handlers to use LLDB_INVALID_SIGNAL correctly. However, I've only tested the fixes with my updated thread support, causing regression in the old code. Therefore, I've also had to fix LLDB_INVALID_SIGNAL handling in NetBSD plugin for the time being.

Thread suspend/resume kernel problem

Sadly, further investigation of hanging tests led me to the conclusion that they are caused by kernel bugs. The first bug I've noticed is that PT_SUSPEND/PT_RESUME do not cause the thread to be resumed correctly. I have written the following reproducer for it:

#include <assert.h>
#include <lwp.h>
#include <pthread.h>
#include <signal.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/ptrace.h>
#include <sys/wait.h>

void* thread_func(void* foo) {
    int i;
    printf("in thread_func, lwp = %d\n", _lwp_self());
    for (i = 0; i < 100; ++i) {
        printf("t2 %d\n", i);
        sleep(2);
    }
    printf("out thread_func\n");
    return NULL;
}

int main() {
    int ret;
    int pid = fork();
    assert(pid != -1);
    if (pid == 0) {
        int i;
        pthread_t t2;

        ret = ptrace(PT_TRACE_ME, 0, NULL, 0);
        assert(ret != -1);
        printf("in main, lwp = %d\n", _lwp_self());
        ret = pthread_create(&t2, NULL, thread_func, NULL);
        assert(ret == 0);
        printf("thread started\n");

        for (i = 0; i < 100; ++i) {
            printf("t1 %d\n", i);
            sleep(2);
        }

        ret = pthread_join(t2, NULL);
        assert(ret == 0);
        printf("thread joined\n");
    }

    sleep(1);
    ret = kill(pid, SIGSTOP);
    assert(ret == 0);
    printf("stopped\n");

    pid_t waited = waitpid(pid, &ret, 0);
    assert(waited == pid);
    printf("wait: %d\n", ret);

    printf("t2 suspend\n");
    ret = ptrace(PT_SUSPEND, pid, NULL, 2);
    assert(ret == 0);
    ret = ptrace(PT_CONTINUE, pid, (void*)1, 0);
    assert(ret == 0);

    sleep(3);
    ret = kill(pid, SIGSTOP);
    assert(ret == 0);
    printf("stopped\n");

    waited = waitpid(pid, &ret, 0);
    assert(waited == pid);
    printf("wait: %d\n", ret);

    printf("t2 resume\n");
    ret = ptrace(PT_RESUME, pid, NULL, 2);
    assert(ret == 0);
    ret = ptrace(PT_CONTINUE, pid, (void*)1, 0);
    assert(ret == 0);

    sleep(5);
    ret = kill(pid, SIGTERM);
    assert(ret == 0);

    waited = waitpid(pid, &ret, 0);
    assert(waited == pid);
    printf("wait: %d\n", ret);

    return 0;
}

The program should run a two-threaded subprocess, with both threads outputting successive numbers. The second thread should be suspended shortly, then resumed. However, currently it does not resume.

I believe that this caused by ptrace_startstop() altering process flags without reimplementing the complete logic as used by lwp_suspend() and lwp_continue(). I've been able to move forward by calling the two latter functions from ptrace_startstop(). However, Kamil has indicated that he'd like to make those routines use separate bits (to distinguish LWPs stopped by process from LWPs stopped by debugger), so I haven't pushed my patch forward.

Multiple thread reporting kernel problem

The second and more important problem is related to how new LWPs are reported to the debugger. Or rather, that they are not reported reliably. When many threads are started by the process in a short time (e.g. in a loop), the debugger receives reports only for some of them.

This can be reproduced using the following program:

#include <assert.h>
#include <lwp.h>
#include <pthread.h>
#include <signal.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/ptrace.h>
#include <sys/wait.h>

void* thread_func(void* foo) {
    printf("in thread, lwp = %d\n", _lwp_self());
    sleep(10);
    return NULL;
}

int main() {
    int ret;
    int pid = fork();
    assert(pid != -1);
    if (pid == 0) {
        int i;
        pthread_t t[10];

        ret = ptrace(PT_TRACE_ME, 0, NULL, 0);
        assert(ret != -1);
        printf("in main, lwp = %d\n", _lwp_self());
        raise(SIGSTOP);
        printf("main resumed\n");

        for (i = 0; i < 10; i++) {
            ret = pthread_create(&t[i], NULL, thread_func, NULL);
            assert(ret == 0);
            printf("thread %d started\n", i);
        }

        for (i = 0; i < 10; i++) {
            ret = pthread_join(t[i], NULL);
            assert(ret == 0);
            printf("thread %d joined\n", i);
        }

        return 0;
    }

    pid_t waited = waitpid(pid, &ret, 0);
    assert(waited == pid);
    printf("wait: %d\n", ret);
    assert(WSTOPSIG(ret) == SIGSTOP);

    struct ptrace_event ev;
    ev.pe_set_event = PTRACE_LWP_CREATE | PTRACE_LWP_EXIT;

    ret = ptrace(PT_SET_EVENT_MASK, pid, &ev, sizeof(ev));
    assert(ret == 0);

    ret = ptrace(PT_CONTINUE, pid, (void*)1, 0);
    assert(ret == 0);

    while (1) {
        waited = waitpid(pid, &ret, 0);
        assert(waited == pid);
        printf("wait: %d\n", ret);
        if (WIFSTOPPED(ret)) {
            assert(WSTOPSIG(ret) == SIGTRAP);

            ptrace_siginfo_t info;
            ret = ptrace(PT_GET_SIGINFO, pid, &info, sizeof(info));
            assert(ret == 0);

            struct ptrace_state pst;
            ret = ptrace(PT_GET_PROCESS_STATE, pid, &pst, sizeof(pst));
            assert(ret == 0);
            printf("SIGTRAP: si_code = %d, ev = %d, lwp = %d\n",
                    info.psi_siginfo.si_code, pst.pe_report_event, pst.pe_lwp);

            ret = ptrace(PT_CONTINUE, pid, (void*)1, 0);
            assert(ret == 0);
        } else
            break;
    }

    return 0;
}

The program starts 10 threads, and the debugger should report 10 SIGTRAP events for LWPs being started (ev = 8) and the same number for LWPs exiting (ev = 16). However, initially I've been getting as many as 4 SIGTRAPs, and the remaining 6 threads went unnoticed.

The issue is that do_lwp_create() does not raise SIGTRAP directly but defers that to mi_startlwp() that is called asynchronously as the LWP starts. This means that the former function can return before SIGTRAP is emitted, and the program can start another LWP. Since signals are not properly queued, multiple SIGTRAPs can end up being issued simultaneously and lost.

Kamil has already worked on making simultaneous signal deliver more reliable. However, he reverted his commit as it caused regressions. Nevertheless, applying it made it possible for the test program to get all SIGTRAPs at least most of the time.

The ‘repeated’ SIGTRAPs did not include correct LWP information, though. Kamil has recently fixed that by moving the relevant data from process information to signal information struct. Combined with his earlier patch, this makes my test program pass most of the time (sadly, there seem to be some more race conditions involved).

Summary of threading work

My current work-in-progress patch can be found on Differential as D64647. However, it is currently unsuitable for merging as some tests start failing or hanging as a side effect of the changes. I'd like to try to get as many of them fixed as possible before pushing the changes to trunk, in order to avoid causing harm to the build bot.

The status with the current set of Kamil's work-in-progress patches applied to the kernel includes approximately 4 failing tests and 10 hanging tests.

Other LLVM news

Manikishan Ghantasala has been working on NetBSD-specific clang-format improvements in this year's Google Summer of Code. He is continuing to work on clang-format, and has recently been given commit access to the LLVM project!

Besides NetBSD-specific work, I've been trying to improve a few other areas of LLVM. I've been working on fixing regressions in stand-alone build support and regressions in support for BUILD_SHARED_LIBS=ON builds. I have to admit that while a year ago I was the only person fixing those issues, nowadays I see more contributions submitting patches for breakages specific to those builds.

I have recently worked on fixing bad assumptions in LLDB's Python support. However, it seems that Haibo Huang has taken it from me and is doing a great job.

My most recent endeavor was fixing LLVM_DISTRIBUTION_COMPONENTS support in LLVM projects. This is going to make it possible to precisely fine-tune which components are installed, both in combined tree and stand-alone builds.

Future plans

My first goal right now is to assert what is causing the test host to crash, and restore buildbot stability. Afterwards, I'd like to continue investigating threading problems and provide more reproducers for any kernel issues we may be having. Once this is done, I'd like to finally push my LLDB patch.

Since threading is not the only goal left in the TODO, I may switch between working on it and on the remaining TODO items. Those are:

1. Add support to backtrace through signal trampoline and extend the support to libexecinfo, unwind implementations (LLVM, nongnu). Examine adding CFI support to interfaces that need it to provide more stable backtraces (both kernel and userland).

2. Add support for i386 and aarch64 targets.

3. Stabilize LLDB and address breaking tests from the test suite.

4. Merge LLDB with the base system (under LLVM-style distribution).

This work is sponsored by The NetBSD Foundation

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can:

https://netbsd.org/donations/#how-to-donate

Submitted by Maciej Grochowski.

This year EuroBSDCon took place in Lillehammer Norway. I had the pleasure to attend as a speaker with my talk about fuzzing the NetBSD filesystems.

Venue

Lillehammer is a ski resort, nestled amid very beautiful scenery between mountains and lakes, just two hours from Oslo. The conference took place in the Scandic Lillehammer Hotel, a little bit away from the downtown of Lillehammer, close to the Olympic Ski Jumps.

View from the Olympic Ski Jump

Talks

Every year, EuroBSDCon has a lot of interesting talks. Unfortunately, it is hard to attend all the interesting seminars, as many of them take place at the same time, so I won't be able to highlight all of them; accordingly, I gratefully acknowledge several organizations for handling the live streaming from every session.

Keynote: Embedded Ethics

The conference started with an excellent Keynote from Patricia Aas (ex. Opera/Cisco/Vivaldi, cur Turtlesec), about the Ethics in the IT industry. As a person who is familiar with the issues with the privacy and many different threads of abusing user data by the company, I have to say that this talk started the avalanche of different thoughts and reflections in my mind. To my surprise, I was not the only one to have such thoughts. This topic arose quite often during the rest of the conference through many conversations between different people. For those of you who didn't see it yet, I highly recommend that you do. The key takeaway is that we, the people who are building today's digital world, need to think about the implications of our work and decisions upon the users of our services. This topic is getting more complicated even as we think about it. However, Patricia come here with the strategy "Annoying as a Service" that can be simply used in every situation to at least not makes things worse...

Conference Talks

During the first day, there were a couple of interesting talks about NetBSD: "Improving modularity of NetBSD compat code", and mine, on "Fuzzing NetBSD Filesystems" [+ Taking NetBSD kernel bug roast to the next level: Kernel Fuzzers (quick A.D. 2019 overview) by Kamil Rytarowski]. As it turns out, there was another interesting talk about foundations of kernel fuzzing by Andrew Turner, in which he presented the connection between sanitizers, tracing modes and fuzzers. After the break, I attended the excellent talk "7th Edition Unix at 40" by Warner Losh -- if you love the history of Unix, this is a must-see. The first day finished with the social mixer. The second day started with one of my favourites of the entire conference: "Kernel TLS and TLS hardware offload" via Drew Gallatin and Hans Petter Selasky. In another room was also a very interesting seminar on Rust for System Programmers. The next session via Netflix folks was about NUMA optimizations in the FreeBSD Network stack, another interesting talk about the usage of BSD as a high-speed CDN serving about 200Gbps Video content(!). After that, I attended the session on The Future of OpenZFS via Allan Jude, where he showed the progress done in the collaboration of different OSes on ZFS Filesystem. The last sessions I attended were the "23 years of software side-channel attacks" by Colin, and the last one before the closing notes: "Unbound & FreeBSD: A true love story", by Pablo Carboni.

Highlights

• Security: We can see clearly that the BSD community continues efforts for making BSDs more secure on various levels. This year we talked mostly about fuzzing, and in this area, it is impossible not to recognise NetBSD for great progress.
• CDN use-case: Netflix contributions to FreeBSD make it a great system for CDN, year after year innovating and increasing the performance. I hope we will see more companies using BSDs as core for their CDN infrastructure.
• ZFS: The filesystem has come a long way, despite being a project divided between different communities. Now thanks to the efforts of the developers, OpenZFS as a united community will be able to progress even faster and take advantage of projects that are using it. I believe the OpenZFS initiative is one of the most important steps taken by the community in many years.

Social Event

This year's social event took place in the Open Air Museum in Maihaugen, where we were able to see, preserved in excellent condition, parts of the Norwegian houses from the 19th century through the late 20th century. The fun part was that every house was open and you were able to go inside, some of them with people dressed up in the fashion of the same years, talking about the age. I very much enjoyed it, as it was a great opportunity to learn more about Norwegian culture and history.

The XX century city

XIX century school

Next Year!

The most important key point during closing notes is always: "where will the next EuroBSDCon take place?!" This year the guessing game was:

• Beer will be cheaper than in Norway
• [picture of Schnitzel]
• Photo of...

Vienna!

Hope to see you all next year in Vienna!

As NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration. This work is now accomplished and with MKLLVM=yes build option (by default off), the distribution will be populated with LLVM files for ASan, TSan, MSan, UBSan, libFuzzer, SafeStack and XRay.

I have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite.

NetBSD distribution changes

I have enhanced and imported my local MKSANITIZER code that makes whole distribution sanitization possible. Few real bugs were fixed and a number of patches were newly written to reflect the current NetBSD sources state. I have also merged another chunk of the fruits of the GSoC-2018 project with fuzzing the userland (by plusun@).

The following changes were committed to the sources:

• ab7de18d0283 Cherry-pick upstream compiler-rt patches for LLVM sanitizers
• 966c62a34e30 Add LLVM sanitizers in the MKLLVM=yes build
• 8367b667adb9 telnetd: Stop defining the same variables concurrently in bss and data
• fe72740f64bf fsck: Stop defining the same variable concurrently in bss and data
• 40e89e890d66 Fix build of t_ubsan/t_ubsanxx under MKSANITIZER
• b71326fd7b67 Avoid symbol clashes in tests/usr.bin/id under MKSANITIZER
• c581f2e39fa5 Avoid symbol clashes in fs/nfs/nfsservice under MKSANITIZER
• 030a4686a3c6 Avoid symbol clashes in bin/df under MKSANITIZER
• fd9679f6e8b1 Avoid symbol clashes in usr.sbin/ypserv/ypserv under MKSANITIZER
• 5df2d7939ce3 Stop defining _rpcsvcdirty in bss and data
• 5fafbe8b8f64 Add missing extern declaration of ib_mach_emips in installboot
• d134584be69a Add SANITIZER_RENAME_CLASSES in bsd.prog.mk
• 2d00d9b08eae Adapt tests/kernel/t_subr_prf for MKSANITIZER
• ce54363fe452 Ship with sanitizer/lsan_interface.h for GCC 7
• 7bd5ee95e9a0 Ship with sanitizer/lsan_interface.h for LLVM 7
• d8671fba7a78 Set NODEBUG for LLVM sanitizers
• 242cd44890a2 Add PAXCTL_FLAG rules for MKSANITIZER
• 5e80ab99d9ce Avoid symbol clashes in test/rump/modautoload/t_modautoload with sanitizers
• e7ce7ecd9c2a sysctl: Add indirection of symbols to remove clash with sanitizers
• 231aea846aba traceroute: Add indirection of symbol to remove clash with sanitizers
• 8d85053f487c sockstat: Add indirection of symbols to remove clash with sanitizers
• 81b333ab151a netstat: Add indirection of symbols to remove clash with sanitizers
• a472baefefe8 Correct the memset(3)'s third argument in i386 biosdisk.c
• 7e4e92115bc3 Add ATF c and c++ tests for TSan, MSan, libFuzzer
• 921ddc9bc97c Set NOSANITIZER in i386 ramdisk image
• 64361771c78d Enhance MKSANITIZER support
• 3b5608f80a2b Define target_not_supported_body() in TSan, MSan and libFuzzer tests
• c27f4619d513 Avoids signedness bit shift in db_get_value()
• 680c5b3cc24f Fix LLVM sanitizer build by GCC (HAVE_LLVM=no)
• 4ecfbbba2f2a Rework the LLVM compiler_rt build rules
• 748813da5547 Correct the build rules of LLVM sanitizers
• 20e223156dee Enhance the support of LLVM sanitizers
• 0bb38eb2f20d Register syms.extra in LLVM sanitizer .syms files

Almost all of the mentioned commits were backported to NetBSD-9 and will land 9.0.

As a demo, I have crafted a writing on combining RUMPKERNEL, MKSANITIZER with the honggfuzz fuzzer: Rumpkernel assisted fuzzing of the NetBSD file system kernel code in userland.

GDB

I've merged NetBSD distribution downstream GDB patches into my local GDB tree and executed the regression tests (check-gdb):

[...]
Test run by kamil on Mon Sep  2 12:36:03 2019
Native configuration is x86_64-unknown-netbsd9.99

                === gdb tests ===

Schedule of variations:
    unix

[...]
                === gdb Summary ===

# of expected passes            54591
# of unexpected failures        3267
# of expected failures          35
# of unknown successes          3
# of known failures             59
# of unresolved testcases       29
# of untested testcases         141
# of unsupported tests          399

Full log is here.

This means that there are a lot of more tests and known failures than in 2017-09-05:

$ uname -a
NetBSD chieftec 8.99.2 NetBSD 8.99.2 (GENERIC) #0: Sat Sep  2 22:55:29 CEST 2017  root@chieftec:/public/netbsd-root/sys/arch/amd64/compile/GENERIC amd64

Test run by kamil on Tue Sep  5 17:06:28 2017
Native configuration is x86_64--netbsd

                === gdb tests ===

Schedule of variations:
    unix

[...]
                === gdb Summary ===

# of expected passes            16453
# of unexpected failures        483
# of expected failures          9
# of known failures             28
# of unresolved testcases       17
# of untested testcases         41
# of unsupported tests          25

There are actually some regressions and a set of tests that fails probably due to environment differences like lack of gfortran at hand.

Full log is here

GSoC Mentoring

The Google Summer of Code programme reached the end. My mentees wrote successfully their final reports:

• Enchancing Syzkaller Support for NetBSD, Part 3
• Adapting TriforceAFL for NetBSD, Part 3

I'm also mentoring the AFL+KCOV work by Maciej Grochowski. Maciej will visit EuroBSDCon-2019 and speak about his work.

Add methods for setting and getting the thread name

I've reached out to the people from standards bodies and I'm working on defining the standard approach for setting and getting the thread name. I have received a proper ID of my proposal and I'm now supposted to submit the text in either PDF or HTML format.

This change will allow to manage the thread name with an uniform interface on all comforming platforms.

Plan for the next milestone

Keep enhancing GDB support. Keep detecting ptrace(2) bugs and addressing them.

This work was sponsored by The NetBSD Foundation.

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can:

http://netbsd.org/donations/#how-to-donate

Prepared by Siddharth Muralee(@R3x) as a part of Google Summer of Code’19

As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that supports a variety of operating systems including NetBSD.

You can take a look through the first report to see the initial changes that we made and you can look at the second report to read about the initial support we added for fuzzing the network stack.

This report details the work done during the final coding period where the target was to improve the support for fuzzing the filesystem stack.

Filesystem fuzzing is a relatively less explored area. Syzkaller itself only has filesystem fuzzing support for Linux.

Analysis of the existing Linux setup

Filesystems are more complex fuzzing target than standalone system calls. To fuzz Filesystems we do have a standard operation like mount which comes with system call vector and an additional binary image of the filesystem itself. While normal syscalls generally have a size of a few bytes, sizes of real world Filesystem images is in order of Gigabytes or larger, however for fuzzing minimal size can be used which is in order of KB-MB. Since syzkaller uses a technique called as mutational fuzzing - where it mutates random parts of the input (according to specified guidelines), having a large input size causes delay due to higher I/O time.

Syzkaller deals with large images by disassembling them to non-zero chunks of the filesystem image. Syzkaller extracts the non-zero chunks and their offsets and stores it as separate segments and just before execution it writes all the chunks into the corresponding offsets - generating back the new/modified image.

Porting it to NetBSD

As an initial step towards filesystem fuzzing we decided to port the existing Linux approach of creating random segments to NetBSD. There are a few differences between the mounting process in both the operating systems - the most significant of them being the difference in the arguments to mount(2).

Linux:

int mount(const char *source, const char *target, const char *filesystemtype, unsigned long mountflags, const void *data);

The data argument is interpreted by the different filesystems. Typically it is a string of comma-separated options understood by this filesystem. mount(8) - shows possible arguments for each of the filesystem.

possible options for xfs filesystem in linux :

    wsync, noalign, swalloc, nouuid, mtpt, grpid, nogrpid, bsdgroups, 
    sysvgroups,norecovery, inode64, inode32, ikeep, noikeep,
    largeio, nolargeio, attr2, noattr2, filestreams, quota,
    noquota, lazytime, nolazytime, usrquota, grpquota, prjquota,
    uquota, gquota, pquota, uqnoenforce, gqnoenforce, pqnoenforce,
    qnoenforce, discard, nodiscard, dax, barrier, nobarrier, logbufs,
    biosize, sunit, swidth, logbsize, allocsize, logdev, rtdev

NetBSD:

Int mount(const char *type, const char *dir, int flags, void *data, size_t data_len);

The argument data describes the file system object to be mounted, and is data_len bytes long. data is a pointer to a structure that contains the type specific arguments to mount.

For FFS (one of the most common filesystems for NetBSD) - the arguments look like :

struct ufs_args {
        char      *fspec;   /* block special file to mount */
};

Currently, we have a pseudo syscall syz_mount_image which does the job of writing the mutated chunks of the filesystem into a file based on their offsets and later configuring the loop device using vndconfig(8) and mounting the filesystem image using mount(8).

Analysis of the current approach

One way to create mountable filesystems is to convert an existing filesystem image into a syzkaller pseudo grammar representation and then add it to the corpus so that syzkaller uses it for mutation and we have a proper image.

Some of the noted issues with syzkaller approach (as noted in "Fuzzing File Systems via Two-Dimensional Input Space Exploration) :

Lack of metadata knowledge - This may lead to corruption of filesystem specific aspects such as checksums.

Lack of Context awareness - Syzkaller isn't aware of the status of the filesystem image after a few operations are performed on it.

Steps Forward

We also spent some time researching possible options to solve the existing issues and developing an approach that would give us better results.

Image mutator approach

One possible way forward is to actually use a seed image (a working filesystem image) and write a mutator which would be aware of all the metadata in the image. The mutator should be also be able to recreate metadata components such as the checksum so that the image is mountable.

An existing implementation of such a mutator is JANUS which is a filesystem mutator written for Linux with inspiration from fsck.

Grammar based approach

Syzkaller uses a pseudo-formal grammar for representing arguments to syscalls. This grammar can also be modified to actually be able to properly generate filesystem images.

Writing grammar to represent a filesystem image is quite a daunting task and we are not yet sure if it is possible but it is the approach that we have planned to take up as of now.

Proper documentation detailing the structure of a filesystem image is rather scarce which has led me to actually go through filesystem code to figure out the type, uses and limits of a certain filesystem image. This data then has to be converted to syzkaller representation to be used for fuzzing.

One advantage of writing a grammar that would be able to generate mountable images is that we would be able to get more coverage than fuzzing with a seed image, since we are also creating new images instead of just mutating the same image.

I am currently working on learning the internals of FFS and trying to write a grammar definition which can properly generate filesystem images.

Miscellaneous Work

Meanwhile, I have also been working in parallel on improving the existing state of Syzkaller.

Add kernel compiled with KUBSAN for fuzzing

So far we only used a kernel compiled with KCOV and KASAN for fuzzing with syzkaller. We also decided to add support for syzkaller building a kernel with KUBSAN and KCOV. This would help us have an another dimension in the fuzzing process.

This required some changes in the build config. We had to remove the hardcoded kernel config and add support for building a kernel with a config passed to the fuzzer. This move would also help us to easily add support for upcoming sanitizers such as KMSAN.

Improve syscall descriptions

Improving system call descriptions is a constant ongoing work - I recently added support for fuzzing syscalls such as mount, fork and posix_spawn.

We are also planning to add support for fuzzing device drivers soon.

Relevant Links

Syzkaller Dashboard for NetBSD

Syzkaller repository on Github

NetBSD docs on setting up syzkaller

GSoC'19 proof of work repository

Summary

We have managed to meet most of the goals that we had planned for the GSoC project. Overall, I have had a wonderful summer with the NetBSD foundation and I look forward to working with them to complete the project.

Last but not least, I want to thank my mentors, @kamil and @cryo for their useful suggestions and guidance. I also thank Maciej for his insight and guidance which was very fundamental during the course of the project. I would also like to thank Dmitry Vyukov, Google for helping with any issues faced with regard to Syzkaller. Finally, thanks to Google to give me a good chance to work with NetBSD community.

Prepared by Akul Pillai as part of GSoC 2019.

This is the third report summarising the work done in the third coding period for the GSoC project of Adapting TriforceAFL for NetBSD kernel syscall fuzzing.
Please also go through the first and second report.

This post also outlines the work done throughout the duration of GSoC, describes the implications of the same and future improvements to come.

Current State

As of now TriforceAFL has been made available in the form of a pkgsrc package (wip/triforceafl). This package allows you to essentially fuzz anything in QEMU’s full system emulation mode using AFL. TriforceNetBSDSyscallFuzzer is built on top of TriforceAFL, specifically to fuzz the NetBSD kernel syscalls. It has also now been made available as wip/triforcenetbsdsyscallfuzzer.
Several minor issues found in the above two packages have now been resolved, and the project restructured.

Issues found include:

• The input generators would also test the the generated inputs, causing several syscalls to be executed which messed up permissions of several other directories and lead to other unwanted consequences. Testing of syscalls has now been disabled as the working ones are specified.
• Directory specified for the BIOS, VGA BIOS and keymaps for QEMU in the runFuzz script was incorrect. This was because wip/triforceafl did not install the specified directory. The package has now been patched to install the pc-bios directory.
• The project was structured in a manner where the host was Linux and the target was NetBSD. Since that is no longer the case and the fuzzer is meant to be run on a NetBSD machine, the project was restructured so that everything is now in one directory and there is no longer a need to move files around.

The packages should now work as intended by following the instructions outlined in the README.

The fuzzer was able to detect a few bugs in the last coding period, details can be found in the last report. During this coding period, the Fuzzer was able to detect 79 unique crashes in a period of 2 weeks running on a single machine. The kernel was built with DEBUG + LOCKDEBUG + DIAGNOSTIC. Work is underway to analyse, report and fix the new bugs and to make the process faster.

With an initial analysis on the outputs on the basis of the syscall that lead to the crash, 6 of the above crashes were unique bugs, the rest were duplicates or slight variants, of which 3 have been previously reported.
Here are the backtraces of the new bugs found (The reproducers were run with kUBSan Enabled):

BUG1:

[ 110.4035826] panic: cv_enter,172: uninitialized lock (lock=0xffffe3c1b9
fc0c50, from=ffffffff81a436e9)
[ 110.4035826] cpu0: Begin traceback...
[ 110.4035826] vpanic() at netbsd:vpanic+0x1fd
[ 110.4035826] snprintf() at netbsd:snprintf
[ 110.4035826] lockdebug_locked() at netbsd:lockdebug_locked+0x45e
[ 110.4035826] cv_timedwait_sig() at netbsd:cv_timedwait_sig+0xe7
[ 110.4035826] lfs_segwait() at netbsd:lfs_segwait+0x6e
[ 110.4035826] sys___lfs_segwait50() at netbsd:sys___lfs_segwait50+0xe2
[ 110.4035826] sys___syscall() at netbsd:sys___syscall+0x121
[ 110.4035826] syscall() at netbsd:syscall+0x1a5
[ 110.4035826] --- syscall (number 198) ---
[ 110.4035826] 40261a:
[ 110.4035826] cpu0: End traceback...
[ 110.4035826] fatal breakpoint trap in supervisor mode
[ 110.4035826] trap type 1 code 0 rip 0xffffffff8021ddf5 cs 0x8 rflags 0x282 cr2
 0x73f454b70000 ilevel 0x8 rsp 0xffff8a0068390d70
[ 110.4035826] curlwp 0xffffe3c1efc556a0 pid 709.1 lowest kstack 0xffff8a006838d
2c0
Stopped in pid 709.1 (driver) at        netbsd:breakpoint+0x5:  leave
db{0}> bt
breakpoint() at netbsd:breakpoint+0x5
vpanic() at netbsd:vpanic+0x1fd
snprintf() at netbsd:snprintf
lockdebug_locked() at netbsd:lockdebug_locked+0x45e
cv_timedwait_sig() at netbsd:cv_timedwait_sig+0xe7
lfs_segwait() at netbsd:lfs_segwait+0x6e
sys___lfs_segwait50() at netbsd:sys___lfs_segwait50+0xe2
sys___syscall() at netbsd:sys___syscall+0x121
syscall() at netbsd:syscall+0x1a5
--- syscall (number 198) ---
40261a:

BUG2:

[ 161.4877660] panic: LOCKDEBUG: Mutex error: rw_vector_enter,296: spin lock hel
d
[ 161.4877660] cpu0: Begin traceback...
[ 161.4877660] vpanic() at netbsd:vpanic+0x1fd
[ 161.4877660] snprintf() at netbsd:snprintf
[ 161.4877660] lockdebug_abort1() at netbsd:lockdebug_abort1+0x115
[ 161.4877660] rw_enter() at netbsd:rw_enter+0x645
[ 161.4877660] uvm_fault_internal() at netbsd:uvm_fault_internal+0x1c5
[ 161.4877660] trap() at netbsd:trap+0xa71
[ 161.4877660] --- trap (number 6) ---
[ 161.4877660] config_devalloc() at netbsd:config_devalloc+0x644
[ 161.4877660] config_attach_pseudo() at netbsd:config_attach_pseudo+0x1c
[ 161.4877660] vndopen() at netbsd:vndopen+0x1f3
[ 161.4877660] cdev_open() at netbsd:cdev_open+0x12d
[ 161.4877660] spec_open() at netbsd:spec_open+0x2d0
[ 161.4877660] VOP_OPEN() at netbsd:VOP_OPEN+0xba
[ 161.4877660] vn_open() at netbsd:vn_open+0x434
[ 161.4877660] sys_ktrace() at netbsd:sys_ktrace+0x1ec
[ 161.4877660] sys___syscall() at netbsd:sys___syscall+0x121
[ 161.4877660] syscall() at netbsd:syscall+0x1a5
[ 161.4877660] --- syscall (number 198) ---
[ 161.4877660] 40261a:
[ 161.4877660] cpu0: End traceback...
[ 161.4877660] fatal breakpoint trap in supervisor mode
[ 161.4877660] trap type 1 code 0 rip 0xffffffff8021ddf5 cs 0x8 rflags 0x286 cr2
 0xfffffffffffff800 ilevel 0x8 rsp 0xffff9c80683cd4f0
[ 161.4877660] curlwp 0xfffffcbcda7d36a0 pid 41.1 lowest kstack 0xffff9c80683ca2
c0
db{0}> bt
breakpoint() at netbsd:breakpoint+0x5
vpanic() at netbsd:vpanic+0x1fd
snprintf() at netbsd:snprintf
lockdebug_abort1() at netbsd:lockdebug_abort1+0x115
rw_enter() at netbsd:rw_enter+0x645
uvm_fault_internal() at netbsd:uvm_fault_internal+0x1c5
trap() at netbsd:trap+0xa71
--- trap (number 6) ---
config_devalloc() at netbsd:config_devalloc+0x644
config_attach_pseudo() at netbsd:config_attach_pseudo+0x1c
vndopen() at netbsd:vndopen+0x1f3
cdev_open() at netbsd:cdev_open+0x12d
spec_open() at netbsd:spec_open+0x2d0
VOP_OPEN() at netbsd:VOP_OPEN+0xba
vn_open() at netbsd:vn_open+0x434
sys_ktrace() at netbsd:sys_ktrace+0x1ec
sys___syscall() at netbsd:sys___syscall+0x121
syscall() at netbsd:syscall+0x1a5
--- syscall (number 198) ---
40261a:

BUG3:

[ 350.9942146] UBSan: Undefined Behavior in /home/ubuntu/triforce/kernel/
src/sys/kern/kern_ktrace.c:1398:2, member access within misaligned address 0x2b0
000002a for type 'struct ktr_desc' which requires 8 byte alignment
[ 351.0025346] uvm_fault(0xffffffff85b73100, 0x2b00000000, 1) -> e
[ 351.0025346] fatal page fault in supervisor mode
[ 351.0025346] trap type 6 code 0 rip 0xffffffff81b9dbf9 cs 0x8 rflags 0x286 cr2
 0x2b00000032 ilevel 0 rsp 0xffff8780684d7fb0
[ 351.0025346] curlwp 0xffffa992128116e0 pid 0.54 lowest kstack 0xffff8780684d42
c0
kernel: page fault trap, code=0
Stopped in pid 0.54 (system) at netbsd:ktrace_thread+0x1fd:     cmpq    %rbx,8(%
r12)
db{0}> bt
ktrace_thread() at netbsd:ktrace_thread+0x1fd

Reproducing Crashes

Right now the best way to reproduce a bug detected is to use the Fuzzer’s driver program itself:

./driver -tv < crash_file

The crash_file can be found in the outputs directory and is a custom file format made for the driver.
Memory allocation and Socket Creation remain to be added to the reproducer generator(genRepro) highlighted in the previous post and will be prioritised in the future.

Implications

Considering that we have a working fuzzer now, it is a good time to analyse how effective TriforceAFL is compared to other fuzzers.

Recently Syzkaller has been really effective in finding bugs in NetBSD. As shown in the below diagrams, both TriforceAFL and Syzkaller create multiple instances of the system to be fuzzed, gather coverage data, mutate input accordingly and continue fuzzing, but there are several differences in the way they work.

TriforceAFL



Syzkaller

Key differences between the two include:

• KCOV
  Syzkaller relies on the KCOV module for coverage data in NetBSD whereas TriforceAFL gets coverage information from its modified version of QEMU.
• VMs
  Syzkaller creates multiple VM’s and manages them with syz-manager. Whereas TriforceAFL simply forks the VM for each testcase.
• Communication
  Syzkaller uses RPC and ssh to communicate with the VMs whereas TriforceAFL uses a custom hypercall.
• Hardware Acceleration
  Syzkaller can use hardware acceleration to run the VMs at native speeds, whereas TriforceAFL can only utilize QEMU’s full system emulation mode, as it relies on it for coverage data.

These differences lead to very different results. To get a perspective, here are some stats from syzkaller, which can be found on the syzbot dashboard.

Bugs Found	Upstream	Fixed
57	37	20

Comparatively in 1st Weekend of Fuzzing:

	Bugs Found
Syzkaller	18
TriforceAFL	3

• Compared to syzkaller, the number of bugs found by TriforceAFL in the first few days were significantly less, but nevertheless TriforceAFL was able to find variants of bugs found by syzkaller and 1 which was different with simpler reproducers.
• Going by the stats provided by Syzbot and AFL, Syzkaller does ~80 execs / min whereas TriforceAFL can do ~1500 execs / min on average, Although this statistic is without any of the sanitizers enabled for TriforceAFL and kASan enabled for Syzkaller.
• TriforceAFL has an advantage when it comes to the fact that it does not rely on KCOV for coverage data. This means it can get coverage data for fuzzing other interfaces easily too. This will be beneficial when we move onto Network and USB Fuzzing. Issues have been found with KCOV where in certain cases the fuzzer lost track of the kernel trace, especially in networking where after enqueuing a networking packet the fuzzer lost track as the packet was then handled by some other thread. Coverage data gathered using TriforceAFL will not be sensitive to this, considering that noise from the kernel is handled in some way to an extent.
• Efforts were made in the original design of TriforceAFL to make the traces as deterministic as possible at a basic block level. To quote from this article: Sometimes QEMU starts executing a basic block and then gets interrupted. It may then re-execute the block from the start or translate a portion of the block that wasn't yet executed and execute that. This introduced non-determinism. To reduce the non-determinism, cpu-exec.c was altered to disable QEMU's "chaining" feature, and moved AFL's tracing feature to cpu_tb_exec to trace a basic block only after it has been executed to completion. Although nothing has been specifically done to reduce noise from the kernel, such as disabling coverage during interrupts.
• TriforceAFL runs 1 fuzzing process per instance, whereas Syzkaller can run upto 32 fuzzing processes. But multiple fuzzing instance can be created with TriforceAFL as detailed in this document to take advantage of parallelization.
• Maxime Villard was also able to rework TriforceNetBSDSyscallFuzzer to now also support compat_netbsd32 kernel. This work will be integrated as soon as possible.
• On the other hand, Syzkaller has syzbot which can be thought of as a 24/7 fuzzing service. TriforceAFL does not have a such a service or a dedicated server for fuzzing.
  A service like this will surely be advantageous in the future, but it is still a bit too early to set up one. To truly efficiently utilize such a service, it would be better to improve TriforceAFL in all ways possible first.

Targets to be met before a 24/7 fuzzing service is setup, include but are not limited to:

• Automatic initial Analysis & Report Generation
  Right now, There is nothing that can automatically perform an initial analysis to detect truly unique crashes and prepare reports with backtraces, this will be required and very helpful.
• Better Reproducers
  As mentioned before, the generated reproducers do not take care of Memory Allocation and Socket Creation, etc. These need to be included, if a good C reproducer is expected.
• Parallel Fuzzing and Management
  There is no central interface that summarises the statistics from all fuzzing instances, and manages such instances in case of anomalies/errors. Something like this would be needed for a service that will be run for longer periods of time without supervision.
• Updated AFL and QEMU versions
  Updated AFL and QEMU might significantly increase executions per second and lead to better mutation of inputs and also decrease the memory requirements.
• Fuzzing more Interfaces
  Right now we are only fuzzing syscalls, Network and USB Layers are great future prospects, at least prototype versions can be added in the near future.

Future Work

Although GSoC will be officially ending, I am looking forward to continuing the development of TriforceAFL, adding features and making it more effective.
Some improvements that can be expected include:

• Fuzzing different interfaces - Network Fuzzing, USB Fuzzing
• Updating AFL and QEMU versions
• Better Reproducers
• Syzbot like service

A new repo has been created at https://github.com/NetBSD/triforce4netbsd. Collaborative work in the future will be updated here.

Conclusion

TriforceAFL has been successfully adapted for NetBSD and all of the original goals of the GSoC proposal have been met, but the work is far from complete. Work done so far shows great potential, and incremental updates will surely make TriforceAFL a great fuzzer. I am looking forward to continuing the work and making sure that this is the case.
GSoC has been an amazing learning experience! I would like to thank Maxime Villard for taking an active interest in TriforceAFL and for assisting in testing and development. I would like to thank my mentor Kamil Rytarowski for being the guiding force throughout the project and for assisting me whenever I needed help. And finally I would like to thank Google for giving me this wonderful opportunity.

This article was prepared by Surya P as a part of Google Summer of Code 2019

To begin with where we left off last time, we were able to fix the suse131 package with this commit.This commit adds the GPU-specific bits to the package. And with that we had direct rendering enabled and working.I tested it out with glxinfo and glxgears applications.

Testing

In order to make sure that applications did not break with this commit,I tried Libreoffice and to no surprise everything ran as expected without any hiccups.

Then I had to make a choice between porting steam and implementing compat_netbsd32 but since steam had lot of dependencies which needed to be resolved and since implementation of compat_netbsd32 had much more priority I started with the implementation of compat_netbsd32.

Implementing compat_netbsd32 DRM ioctls - The Setup

For the Setup I downloaded i386 sets from the official NetBSD site and extracted it in the /emul directory. I ran some arbitrary programs like cat and ls from the emulated netbsd32 directory to make sure everything ran perfectly without any problems. I then tried running the 32bit glxinfo and glxgears application and to no surprise it kept segfaulting. I ktraced the application and identified the DRM ioctl that needed to be implemented.

Implementing compat_netbsd32 DRM ioctls - The Code

There were several functions which were required for the complete working of the compat_netbsd32 DRM ioctl. We implemented each and every function and had the code compiled. We then made sure that the code compiled both as a module and as well as a non module option with which the kernel can be built.I initially tested the code with 32bit glxinfo and glxgears , and the program didn't segfault and ran as expected.

Implementing compat_netbsd32 DRM ioctls - Testing

In order to test the code I built a test application leveraging the api’s provided in libdrm. It is a very simple application which initializes the DRM connection, setup and draws a gradient on screen and exits. I initially ran it against the native amd64 architecture, but to my surprise the application didn't work as expected. After some hours of debugging I realized that there can be only one DRM master and X was already a master. After exiting the X session and running the application, everything ran perfectly for both amd64 as well as i386 architectures.

What is done

• The Drm Ioctls implementation of Netbsd has been tested and verified
• The suse131 package has patched and updated (committed)
• Compat_netbsd32 DRM ioctls has been implemented (Merged)
• Subsequently DRM ioctls for emulated 32bit linux as well
• Created a Test GUI Application for the code (yet to PR)

TODO

• Create an ATF for the code and merge it into the tree
• Read the code, look for bugs and clean it up
• Port Steam and make it available in NetBSD

Conclusion

Completing the tasks listed in the TODO is of highest priority and would be carried over even if it exceeds the GSOC time period.

Last but not the least I would like to thank my mentors @christos and @maya for helping me out and guiding me throughout the process and Google for providing me with such a wonderful opportunity to work with NetBSD community.

This report was prepared by Manikishan Ghantasala as a part of Google Summer of Code 2019

This is the third and final report of the project Add KNF (NetBSD style) clang-format configuration that I have been doing as a part of Google Summer of Code (GSoC) ‘19 with the NetBSD.

You can refer to the first and second reports here:

1. Adding NetBSD KNF to clang-format, Part 1
2. Adding NetBSD KNF to clang-format, Part 2

About the project

ClangFormat is a set of tools to format C/C++/Java/JavaScript/Objective-C/Protobuf code. It is built on top of LibFormat to support workflow in various ways including a standalone tool called clang-format, and editor integrations. It supports a few built-in CodingStyles that include: LLVM, Google, Chromium, Mozilla, Webkit. When the desired code formatting style is different from the available options, the style can be customized using a configuration file. The aim of this project is to add NetBSD KNF support to clang-format and new styles to libFormat that support NetBSD’s style of coding. This would allow us to format NetBSD code by passing `-style=NetBSD` as an argument.

How to use clang-format

While using clang-format one can choose a style from the predefined styles or create a custom style by configuring specific style options.

Use the following command if you are using one of the predefined styles

clang-format filename -style=<Name of the style>

Configuring style with clang-format

clang-format supports two ways to provide custom style options: directly specify style name in the -style= command line option or use -style=file and put style configuration in a .clang-format or _clang-format file in the project’s top directory.

Check Clang-Format Style Options to know how different Style Options works and how to use them.

When specifying configuration in the -style= option, the same configuration is applied for all input files. The format of the configuration is:

-style=’{key1: value1, key2: value2, ...}'

The .clang-format file uses YAML format. An easy way to get a valid .clang-format file containing all configuration options of a certain predefined style is:

clang-format -style=llvm -dump-config > .clang-format

After making required changes to the .clang-format file it can be used as a custom Style by:

clang-format <filename> -style=file

Changes made to clang-format

The following changes were made to the clang-format as a part of adding NetBSD KNF:

New Style options added:

BitFieldDeclarationsOnePerLine

AlignConsecutiveListElements

Modifications made to existing styles:

1. Modified SortIncludes  and IncludeCategories to support NetBSD like includes.
2. Modified SpacesBeforeTrailingComments to support block comments.

 The new NetBSD Style Configurations:

This is the final configurations for clang-format with modified changes to support NetBSD KNF.

    
        AlignTrailingComments: true
        AlwaysBreakAfterReturnType: All
        AlignConsecutiveMacros: true
        AlignConsecutiveLists: true
        BitFieldDeclarationsOnePerLine: true
        BreakBeforeBraces: Mozilla
        ColumnLimit: 80
        ContinuationIndentWidth: 4
        Cpp11BracedListStyle: false
        FixNamespaceComments: true
        IndentCaseLabels: false
        IndentWidth: 8
        IncludeBlocks: Regroup
        IncludeCategories:
         - Regex: '^<sys/param\.h>'
           Priority: 1
           SortPriority: 0
         - Regex: '^<sys/types\.h>'
           Priority: 1
           SortPriority: 1
         - Regex: '^<sys.*/'
           Priority: 1
           SortPriority: 2
         - Regex: '^<uvm/'
           Priority: 2
           SortPriority: 3
         - Regex: '^<machine/'
           Priority: 3
           SortPriority: 4
         - Regex: '^<dev/'
           Priority: 4
           SortPriority: 5
         - Regex: '^<net.*/'
           Priority: 5
           SortPriority: 6
         - Regex: '^<protocols/'
           Priority: 5
           SortPriority: 7
         - Regex: '^<(fs|miscfs|msdosfs|nfs|ufs)/'
           Priority: 6
           SortPriority: 8
         - Regex: '^<(x86|amd64|i386|xen)/'
           Priority: 7
           SortPriority: 8
         - Regex: '^<path'
           Priority: 9
           SortPriority: 11
         - Regex: '^<[^/].*\.h'
           Priority: 8
           SortPriority: 10
         - Regex: '^\".*\.h\"'
           Priority: 10
           SortPriority: 12   
        SortIncludes: true
        SpacesBeforeCpp11BracedList: true
        SpacesBeforeTrailingComments: 4
        TabWidth: 8
        UseTab: Always

    

Status of each Style Option

Styles Ready to Merge:

1.Modified SortIncludes and IncludeCategories:

     Patch: https://reviews.llvm.org/D64695

Styles needing revision

1.BitFieldDeclarationsOnePerLine:

    Patch: https://reviews.llvm.org/D63062

    Bugs: 1 

 2.SpacesBeforeTrailingComments supports Block Comments:

    Patch: https://reviews.llvm.org/D65648

    Remark:  I have to discuss more on the cases that Block comments are used but where the Spaces are not to be added before them.

WIP Style

1.AlignConsecutiveListElements:

    Commit: https://github.com/sh4nnu/clang/commit/4b4cd45a5f3d211008763f1c0235a22352faa81e

    Bugs: 1

About Styles

BitfieldDeclarationsOnePerLine:

    Patch: https://reviews.llvm.org/D63062 

    This style lines up BitField declarations on consecutive lines with correct indentation.

Input:

    
            unsigned int bas :3, hh : 4, jjj : 8;
            
            
            unsigned int baz:1,
                            fuz:5,
                zap:2;
    

Output:

            unsigned  int bas : 3, 
                        hh  : 4, 
                        jjj : 8;
            
            
            unsigned int baz:1,
                        fuz:5,
                        zap:2;

Bug: Indentation breaks in the presence of block comments in between.

Input:

        
                    unsigned  int bas : 3, /* foo */
                                  hh  : 4, /* bar */
                                  jjj : 8;
                    
        
        

Output:

        
                    unsigned  int bas : 3, /* foo */
                        hh  : 4, /* bar */
                        jjj : 8;

        
        

Modification for SortIncludes and IncludeCategories:

    Patch: https://reviews.llvm.org/D64695

    Status: Accepted, and ready to land.

Clang-format has a style option named SortIncludes which sorts the includes in alphabetical order.The IncludeCategories Style allows us to define a custom order for sorting the includes.

It supports POSIX extended regular expressions to assign Categories for includes.

The SortIncludes then sorts the #includes first according to increasing category number and then lexically within each category. When IncludeBlocks  is set to Regroup  merge multiple #includes blocks together and sort as one. Then split into groups based on category priority.

The problem arises when you want to define the order within each category, which is not supported. In this modification a new field named SortPriority is added, which is optional.

The #includes matching the regexs are sorted according to the values of SortPriority, and Regrouping after sorting is done according to the values of Priority. If SortPriority is not defined it is set to the value of Priority as a default value.

Example

    
        IncludeCategories:
         - Regex: ‘<^c/’
           Priority: 1
           SortPriority: 0
        
         - Regex: ‘^<(a|b)/’
           Priority: 1
           SortPriority: 1
        
         - Regex: ‘^<(foo)/’
           Priority: 2
      
         - Regex: ‘.*’
           Priority: 3
           

    

Input

    
            #include "exe.h"
            #include <a/dee.h>
            #include <foo/b.h>
            #include <a/bee.h>
            #include <exc.h>
            #include <b/dee.h>
            #include <c/abc.h>
            #include <foo/a.h>
    

Output

    
            #include <c/abc.h>
            #include <a/bee.h>
            #include <a/dee.h>
            #include <b/dee.h>
            
            #include <foo/a.h>
            #include <foo/b.h>
            
            #include <exc.h>
            #include "exe.h"

    

As you can observe in the above example the #includes are grouped by different priority and were sorted by different priority. Introduction of this new patch doesn’t affect the old configurations as it will work as the same old SortIncludes if sortPriority is not defined.

Refer to Report 2 for detailed examples on this.

Modification for SpacesBeforeTrailingComments

    Patch: https://reviews.llvm.org/D64695

The SpacesBeforeTrailingComments is modified to support Block Comments which was used to support only line comments. The reason for this is block comments have different usage patterns and different exceptional cases. I have tried to exclude cases where some tests doesn’t support spaces before block comments. I have been discussing in the community for getting to know which cases should be included, and which to exclude.

Cases that were excluded due to failing tests:

• If it is a Preprocessor directive,
• If it is followed by a LeftParenthesis
• And if it is after a Template closer

AlignConsecutiveListElements

    Status: Work In Progress

This is a new style that aligns elements of consecutive lists in a nested list. The Style is still in work in progress. There are few cases that needed to be covered and fix few bugs.

Input:

    
            keys[] = {
                {"all", f_all, 0 },
                { "cbreak", f_cbreak, F_OFFOK },
                {"cols", f_columns, F_NEEDARG },
                { "columns", f_columns, F_NEEDARG },
             };
    

Output:

    
            keys[] = { { "all",      f_all,        0 },
                       { "cbreak",   _cbreak,      F_OFFOK },
                       { "cols",     f_columns,    F_NEEDARG }, 
                       { "columns",  f_columns,    F_NEEDARG },
                     };

    

Work to be done:

This style option aligns list declarations that are nested inside a list, I would also like to extend this style to align individual single line list declarations that are consecutive.

The problem with this style is the case in which there can be different number of elements for each individual.

Example:

    
            keys[] =  { "all",        f_all,        0 };
            keys2[] = { "cbreak",     _cbreak,      F_OFFOK };
            keys3[] = { "cols",       f_columns,    F_NEEDARG,     7 };
            keys4[] = { "columns",    f_columns };
    

Future Work

Some Style Options that were introduced during this GSoC were made in order to meet all the cases in NetBSD KNF. So they may need some revisions with respect to other languages and coding styles that clang-format supports. I will continue working on this project even after the GSoC period on the style options that are yet to be merged and add new style options if necessary and will get the NetBSD Style merged with upstream which is the final deliverable for the project. I would like to take up the responsibility of maintaining the “NetBSD KNF” support to clang-format.

Summary

Even though officially the GSoC’19 coding period is over, I definitely look forward to keep contributing to this project. This summer has had me digging a lot into the code from CLANG and NetBSD for references for creating or modifying the Style Options. I am pretty much interested to work with NetBSD again, I like being in the community and I would like to improve my skills and learn more about Operating Systems by contributing to this organisation.

I would like to thank my mentors Michal and Christos for their constant support and patient guidance. A huge thanks to both the NetBSD and LLVM community who have been supportive and have helped me whenever I have had trouble. Finally a huge thanks to Google for providing me this opportunity.

This report was written by Naveen Narayanan as part of Google Summer of Code 2019.

This report encompasses the progress of the project during the third coding period. You can make sense of the overall progress of the project by going through the first evaluation report and second evaluation report.

WINE on amd64

Wine-4.4 (released on Mar 2019) is working fine on amd64 and i386. I have been able to use a script as a workaround for the problem of setting LD_LIBRARY_PATH. My patch for setting guard size to 0 and hence, precluding Wine from segfaulting, that got upstreamed, can be found here. I have updated the package to the latest development version of Wine which is Wine-4.13 (released on Aug 2019). I have added support to Wine pkgsrc packages to run tests using make test, and at the time of writing, they are failing. I have also noticed them fail on Linux non-pkgsrc environment and hence, will require further investigation. Initially, they were disabled owing to pkgsrc setting FORTIFY_SOURCE which is a macro that provides support for detecting buffer overflows. In pkgsrc, the wip/wine* packages honor PKGSRC_USE_FORTIFY variable passing _FORTIFY_SOURCE macro accordingly. Programs compiled with FORTIFY_SOURCE substitute wrappers for commonly used libc functions that don't do bounds checking regularly, but could in some cases. Wine unconditionally disables that via their configure script because for some platforms that triggered false positives in the past. However, in my experience, no false positive were found.

Running tests on Wine-4.13 throws errors as you can see below:

mode64# make test
/usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M adsldp.dll -p adsldp_test.exe.so sysinfo && touch sysinfo.ok
/usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations
002a:err:winediag:SECUR32_initNTLMSP ntlm_auth was not found or is outdated. Make sure that ntlm_auth >= 3.0.25 is in your path. Usually, you can find it in the winbind package of your distribution.
002a:fixme:advapi:LsaOpenPolicy ((null),0x22f150,0x00000001,0x22f120) stub
002a:fixme:security:GetWindowsAccountDomainSid (000000000022EEA0 000000000002B028 000000000022EE9C): semi-stub
002a:fixme:advapi:LsaClose (0xcafe) stub
002a:fixme:advapi:LsaOpenPolicy ((null),0x22f0e0,0x00000001,0x22f0b0) stub
002a:fixme:security:GetWindowsAccountDomainSid (000000000022EE30 000000000002B318 000000000022EE2C): semi-stub
002a:fixme:advapi:LsaClose (0xcafe) stub
002a:fixme:advapi:LsaOpenPolicy ((null),0x22f0e0,0x00000001,0x22f0b0) stub
002a:fixme:security:GetWindowsAccountDomainSid (000000000022EE30 000000000002B318 000000000022EE2C): semi-stub
002a:fixme:advapi:LsaClose (0xcafe) stub
002a:fixme:adsldp:sysinfo_get_UserName 0000000000021670,000000000022F0C8: stub
002a:fixme:advapi:LsaOpenPolicy ((null),0x22f980,0x00000001,0x22f950) stub
002a:fixme:security:GetWindowsAccountDomainSid (000000000022F6D0 000000000002B318 000000000022F6CC): semi-stub
002a:fixme:advapi:LsaClose (0xcafe) stub
002a:fixme:advapi:LsaOpenPolicy ((null),0x22f980,0x00000001,0x22f950) stub
002a:fixme:security:GetWindowsAccountDomainSid (000000000022F6D0 000000000002B318 000000000022F6CC): semi-stub
002a:fixme:advapi:LsaClose (0xcafe) stub
002a:fixme:adsldp:sysinfo_get_UserName 0000000000021670,000000000022FB48: stub
/usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so cred && touch cred.ok
/usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations
002e:fixme:cred:CredReadW unhandled type -1
002e:fixme:cred:CredReadW unhandled flags 0xdeadbeef
002e:err:cred:CredWriteW bad username L"winetest"
002e:err:cred:CredWriteW bad username (null)
002e:err:cred:CredWriteW bad username (null)
002e:fixme:cred:CredDeleteW unhandled type -1
002e:fixme:cred:CredDeleteW unhandled flags 0xdeadbeef
002e:fixme:cred:CredReadDomainCredentialsW (0x1b1d0, 0x0, 0x22fa50, 0x22f908) stub
002e:fixme:cred:CredReadDomainCredentialsW (0x1b1d0, 0x0, 0x22fa50, 0x22f908) stub
002e:fixme:cred:CredReadDomainCredentialsW (0x1b3c0, 0x0, 0x22fa50, 0x22f908) stub
cred.c:820: Tests skipped: CRED_TYPE_DOMAIN_VISIBLE_PASSWORD credentials are not supported or are disabled. Skipping
002e:fixme:cred:CredIsMarshaledCredentialW BinaryBlobCredential not checked
002e:fixme:cred:CredIsMarshaledCredentialW BinaryBlobCredential not checked
/usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt && touch crypt.ok
/usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations
/usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_lmhash && touch crypt_lmhash.ok
/usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations
/usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_md4 && touch crypt_md4.ok
/usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations
/usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_md5 && touch crypt_md5.ok
/usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations
/usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_sha && touch crypt_sha.ok
/usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations
/usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so eventlog && touch eventlog.ok
/usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations
0046:fixme:advapi:CloseEventLog (0x0) stub
0046:fixme:advapi:CloseEventLog (0x0) stub
0046:fixme:advapi:OpenEventLogW ((null),(null)) stub
0046:fixme:advapi:OpenEventLogW (L"IDontExist",(null)) stub
0046:fixme:advapi:OpenEventLogW (L"IDontExist",L"deadbeef") stub
0046:fixme:advapi:OpenEventLogW Remote server not supported
0046:fixme:advapi:OpenEventLogW ((null),L"deadbeef") stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:OpenEventLogW (L"",L"Application") stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:GetEventLogInformation (0x0, 1, 0x0, 0, 0x0) stub
0046:fixme:advapi:GetEventLogInformation (0x0, 0, 0x0, 0, 0x0) stub
0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub
0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x0, 0, 0x0) stub
0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x0, 0, 0x22fb40) stub
0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x22fb59, 0, 0x0) stub
0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x22fb59, 0, 0x22fb40) stub
0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x22fb59, 8, 0x22fb40) stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0x0,0x0) stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0x0,0x22fb40) stub
0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x0) stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fb40) stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub
0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0x0,0x22fb40) stub
0046:fixme:advapi:CloseEventLog (0x0) stub
0046:fixme:advapi:GetOldestEventLogRecord (0x0,0x0) stub
0046:fixme:advapi:GetOldestEventLogRecord (0x0,0x22fb40) stub
0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x0) stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fb40) stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub
0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub
0046:fixme:advapi:GetOldestEventLogRecord (0x0,0x22fb40) stub
0046:fixme:advapi:CloseEventLog (0x0) stub
0046:fixme:advapi:BackupEventLogW (0x0,(null)) stub
0046:fixme:advapi:BackupEventLogW (0x0,L"backup.evt") stub
0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub
0046:fixme:advapi:BackupEventLogW (0xcafe4242,(null)) stub
0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub
0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub
0046:fixme:advapi:BackupEventLogW (0x0,L"backup2.evt") stub
0046:fixme:advapi:CloseEventLog (0x0) stub
0046:fixme:advapi:OpenBackupEventLogW ((null),(null)) stub
0046:fixme:advapi:OpenBackupEventLogW ((null),L"idontexist.evt") stub
0046:fixme:advapi:OpenBackupEventLogW (L"IDontExist",(null)) stub
0046:fixme:advapi:OpenBackupEventLogW (L"IDontExist",L"idontexist.evt") stub
0046:fixme:advapi:OpenBackupEventLogW Remote server not supported
0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub
0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
eventlog.c:546: Tests skipped: We don't have a backup eventlog to work with
0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x0,0x0) stub
0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x22fa88,0x0) stub
0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x0,0x22fa8c) stub
0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x22fa88,0x22fa8c) stub
0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x0,0x00000000,0x0,0x0) stub
0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x0,0x00000000,0x22fa88,0x22fa8c) stub
0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x0,0x00000038,0x22fa88,0x22fa8c) stub
0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub
0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub
0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000000,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub
0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000001,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub
0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000002,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub
0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x0000000d,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub
0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x0000000e,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub
0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000007,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa84) stub
eventlog.c:479: Tests skipped: No records in the 'Application' log
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:ClearEventLogW (0x0,(null)) stub
0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub
0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:ClearEventLogW (0x0,L"backup.evt") stub
0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub
0046:fixme:advapi:ClearEventLogW (0x0,L"backup.evt") stub
0046:fixme:advapi:ClearEventLogW (0x0,L"backup2.evt") stub
0046:fixme:advapi:ClearEventLogW (0x0,(null)) stub
0046:fixme:advapi:CloseEventLog (0x0) stub
0046:fixme:advapi:OpenEventLogW ((null),L"Wine") stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:ReportEventA (0xcafe4242,0x0020,0x0000,0x00000000,0x0,0x0000,0x00000000,0x0,0x0): stub
0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000005,0x00000000,0x1b1e0,0x00000038,0x22fa88,0x22fa8c) stub
0046:fixme:advapi:ClearEventLogW (0xcafe4242,(null)) stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:RegisterEventSourceA ((null),"Wine"): stub
0046:fixme:advapi:RegisterEventSourceW (L"",L"Wine"): stub
0046:fixme:advapi:ReportEventA (0xcafe4242,0x0004,0x0001,0x00000001,0x0,0x0001,0x00000000,0x7f7ff72beb00,0x0): stub
0046:fixme:advapi:ReportEventW (0xcafe4242,0x0004,0x0001,0x00000001,0x0,0x0001,0x00000000,0x1b1e0,0x0): stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub
0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub
0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc") stub
0046:fixme:advapi:ReportEventA (0xcafe4242,0x0002,0x0001,0x00000002,0x0,0x0000,0x00000000,0x0,0x0): stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc1"): stub
0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc1"): stub
0046:fixme:advapi:ReportEventA (0xcafe4242,0x0010,0x0001,0x00000003,0x0,0x0002,0x00000000,0x7f7ff72beaf0,0x0): stub
0046:fixme:advapi:ReportEventW (0xcafe4242,0x0010,0x0001,0x00000003,0x0,0x0002,0x00000000,0x1b1e0,0x0): stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub
0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub
0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc20") stub
0046:fixme:advapi:ReportEventA (0xcafe4242,0x0001,0x0001,0x00000004,0x0,0x0000,0x00000000,0x0,0x0): stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc300"): stub
0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc300"): stub
0046:fixme:advapi:ReportEventA (0xcafe4242,0x0002,0x0001,0x00000005,0x0,0x0001,0x00000000,0x7f7ff72beb00,0x0): stub
0046:fixme:advapi:ReportEventW (0xcafe4242,0x0002,0x0001,0x00000005,0x0,0x0001,0x00000000,0x1b1e0,0x0): stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub
0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub
0046:fixme:advapi:OpenEventLogW ((null),L"Wine") stub
0046:fixme:advapi:ReportEventA (0xcafe4242,0x0000,0x0002,0x00000006,0x1b3d0,0x0002,0x00000000,0x7f7ff72beaf0,0x0): stub
0046:fixme:advapi:ReportEventW (0xcafe4242,0x0000,0x0002,0x00000006,0x1b3d0,0x0002,0x00000000,0x1b1e0,0x0): stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc"): stub
0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc"): stub
0046:fixme:advapi:ReportEventA (0xcafe4242,0x0010,0x0002,0x00000007,0x1b3d0,0x0000,0x00000000,0x0,0x0): stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub
0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub
0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc1") stub
0046:fixme:advapi:ReportEventA (0xcafe4242,0x0008,0x0002,0x00000008,0x1b3d0,0x0002,0x00000000,0x7f7ff72beaf0,0x0): stub
0046:fixme:advapi:ReportEventW (0xcafe4242,0x0008,0x0002,0x00000008,0x1b3d0,0x0002,0x00000000,0x1b1e0,0x0): stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc20"): stub
0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc20"): stub
0046:fixme:advapi:ReportEventA (0xcafe4242,0x0002,0x0002,0x00000009,0x1b3d0,0x0000,0x00000000,0x0,0x0): stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub
0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub
0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc300") stub
0046:fixme:advapi:ReportEventA (0xcafe4242,0x0001,0x0002,0x0000000a,0x1b3d0,0x0001,0x00000000,0x7f7ff72beb00,0x0): stub
0046:fixme:advapi:ReportEventW (0xcafe4242,0x0001,0x0002,0x0000000a,0x1b3d0,0x0001,0x00000000,0x1b1e0,0x0): stub
0046:err:eventlog:ReportEventW L"First string"
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
0046:fixme:advapi:OpenEventLogW ((null),L"Wine") stub
0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub
0046:fixme:advapi:CloseEventLog (0xcafe4242) stub
eventlog.c:889: Tests skipped: No events were written to the eventlog
0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub
0046:fixme:advapi:StartTraceA (0x22fb40, "this name is too long", 0x1b1e0) stub
0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x0) stub
0046:fixme:advapi:StartTraceA (0x0, "wine", 0x1b1e0) stub
0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub
0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub
0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub
0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub
0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub
0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub
0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub
0046:fixme:advapi:ControlTraceA (cafe4242, "wine", 0x1b1e0, 1) stub
/usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so lsa && touch lsa.ok
/usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations
004a:fixme:advapi:LsaOpenPolicy ((null),0x22f960,0x000f0fff,0x22f930) stub
004a:fixme:security:GetWindowsAccountDomainSid (000000000022F690 0000000000019078 000000000022F68C): semi-stub
004a:fixme:advapi:LsaEnumerateAccountRights (0xcafe,0x22fa20,0x22f990,0x22f92c) stub
004a:fixme:advapi:LsaClose (0xcafe) stub
004a:fixme:advapi:LsaOpenPolicy ((null),0x22fb10,0x000f0fff,0x22fae8) stub
004a:fixme:advapi:LsaClose (0xcafe) stub
004a:fixme:advapi:LsaOpenPolicy ((null),0x22fb40,0x00000800,0x22fb00) stub
004a:fixme:advapi:LsaClose (0xcafe) stub
004a:fixme:advapi:LsaOpenPolicy ((null),0x22fb40,0x00000800,0x22fb08) stub
/usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so registry && touch registry.ok
/usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations
/usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations
004e:fixme:reg:RegQueryInfoKeyA security argument not supported.
004e:fixme:reg:RegQueryInfoKeyW security argument not supported.
004e:fixme:reg:RegQueryInfoKeyA security argument not supported.
004e:fixme:reg:RegQueryInfoKeyW security argument not supported.
004e:fixme:reg:RegQueryInfoKeyA security argument not supported.
004e:fixme:reg:RegQueryInfoKeyW security argument not supported.
registry.c:2850: Tests skipped: HKCR key merging not supported
registry.c:3146: Tests skipped: HKCR key merging not supported
004e:fixme:winspool:PerfOpen (null): stub
004e:fixme:winspool:PerfCollect L"Global", 0x22ead8, 0x22eabc, 0x22eac0: stub
004e:fixme:winspool:PerfClose stub
004e:fixme:winspool:PerfOpen (null): stub
004e:fixme:winspool:PerfCollect L"invalid counter name", 0x22ead8, 0x22eabc, 0x22eac0: stub
004e:fixme:winspool:PerfClose stub
registry.c:4032: Test failed: [ 9] expected 1168, got 2
registry.c:4032: Test failed: [10] expected 1814, got 2
*** Error code 2

Stop.
make[1]: stopped in /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/wine64/dlls/advapi32/tests
*** Error code 1

Stop.
make: stopped in /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/wine64

Running programs on Wine

You can find obligatory screenshots of Wine-4.4/4.13 on amd64/i386 below:

How to run Wine on NetBSD/amd64

• Compile kernel with USER_LDT enabled.
• Install kernel.
• Clone wip repo.
• cd /usr/pkgsrc/wip/wine64; make install (for 4.4)
or
• cd /usr/pkgsrc/wip/wine64-dev; make install (for 4.13)
• wine notepad

Future Plans

Wine requires the kernel option USER_LDT to be able to run 32-bit applications on amd64 - facilitated by WoW64. Presently, this feature isn't enabled by default on NetBSD and hence, the kernel has to be compiled with USER_LDT enabled. I will work on getting the tests to pass and finding a better approach to deal with LD_LIBRARY_PATH issue. In addition to that, I shall work on incorporating a NetBSD VM to Wine Test Bot infrastructure so we can preclude Wine from getting out of shape on NetBSD in the future (work in progress).

Summary

Presently, Wine-4.4 and Wine-4.13 are working fine on amd64/i386. I have been able to meet all the goals as per my original plan. I would like to attribute the success to the support provided by my mentors @leot, @maya and @christos. I would also like to thank my mentor @maxv for solving the conflict between SVS and USER_LDT kernel options. I intend to maintain the packages wine64/wine64-dev/wine32 for the foreseeable future. As stated above, I shall try to set up a NetBSD VM as Wine Test Bot for purpose of CI. Once again, thanks to Google for enabling me to do what I love.

Source Code

Commit: Wine

Commit: pkgsrc WIP

Package: Wine64

Package: Wine64-dev

Package: Wine32

Once upon a time a developer wrote a temperature sensor driver for one system and decided to port it to another, similar, system. For days and then weeks, the developer waited and waited for the other, similar, system to arrive.

One day, joy of joys, the other, similar, system arrived. A National Day of Celebration commenced, and the porting effort began. Over hours, perhaps even as many as five hours of time, the sensors were finally able to tell you whether they were hot or not.

This other, similar, system suddenly was purposeless and was pondering what else life could provide when the Remote Server task popped up and said "Hey, first media file is free", and sadly, this other, similar, system tried its first media file, and then purchased many, many more media files, and suddenly this other, similar, system was suddenly hooked.

Unfortunately, this other, similar, system had a problem talking on the network without falling asleep on the job, so the developer says "let's try a USB network instead!", and initially this seemed like a good idea. Many bits were transferred over USB, but soon whispers of a lurking monster, known to developers, experience or otherwise, as KASSERT, were heard and soon found common.

The developer attempted other USB network as the other, similar, system was destined to be flown many thousands of miles away soon, but the only other option was similarly plagued by the KASSERT monster. The developer reached into his Bag Of Holding and pulled out a magical weapon known capable of slaying the KASSERT monster. The mighty blade of MPSAFE was free again!

After much planning and many failed attacks, the developer was able to slay the KASSERT monster and all the bits wanting to be transferred were able to be.

For a day and for a night there were celebrations. Much food and ale was consumed until finally everyone was asleep, and the music and lights were finally turned off. In the morning a great clean up was needed and while the developer was cleaning off the shiny, happy and working USB network the original USB network was accidentally reconnected. Oh no, the KASSERT monster has returned! Lock your doors and hide your children.

The developer quickly pulled out MPSAFE again, and once again the KASSERT monster was slain, though the face of this monster was different to the previous monster. The developer then searched and searched for surely they were more KASSERT monsters to be found. Indeed, many many others were found, though they retreated to safety after two more of their number were slain by the mighty MPSAFE.

The developer called upon his friends Shared and Code and with them forged a new weapon against the KASSERT monster, using the mighty MPSAFE in ways unheard of before. After much research and planning, and with the help of some friends, the USBNET was born. All the angels and all the birds of the world were said to sing all at once at this moment, for the USBNET would bring happiness to both the Code Deletionist and the Code Sharers, bound to war against each other from time immemorial.

With this new USBNET the developer was able to blaze a trail across the landscape, searching out each KASSERT monster lurking in every USB network corner. All told, fourteen faces of KASSERT monster were found and the developer and his friends have slain seven of these faces, with the remaining seven under attack, life looks grim for them.

The other, similar, system is safe now. Turns out that MPSAFE also has cleared up the sleeping problem using the cousins, NET and FDT in a tight, dual-blade combination.

Let the world rejoice, for soon the KASSERT monster will be no more!

--mrg @ 2019-08-11

tl;dr:

i fixed many bugs across several USB ethernet adapters and got sick of fixing the same bug across multiple drivers so made common code for them to use instead. the original 4 drivers fixed were axen(4), axe(4), cdce(4), and ure(4). the current list of fixed drivers, at time of writing, includes smsc(4), udav(4) and urndis(4). all drivers except umb(4) are ported but either not tested or not yet working with the usbnet(9) framework.

update 2019-09-02:

all 13 known working drivers converted and will be present in netbsd 9.