--- wikisrc/kyua.mdwn 2012/07/13 20:18:38 1.2 +++ wikisrc/kyua.mdwn 2013/02/01 22:20:02 1.8 @@ -1,5 +1,5 @@ [[!meta title="Kyua: An introduction for NetBSD users"]] -[[!toc ]] +[[!toc levels=2]] The [Automated Testing Framework](ATF), or ATF for short, is a software package composed of two parts: the *ATF libraries* and the *ATF tools*. @@ -32,6 +32,9 @@ This page provides instructions on how t test suite so that you can experiment with the tool, familiarize yourself with it and provide feedback early on. +**If you would like more details on how the import of Kyua into NetBSD will + happen, please see [[Kyua: The way into NetBSD|/kyua/import]].** + # What's in the name? You should really think of Kyua as ATF 2.x. Then, why isn't it just ATF @@ -50,6 +53,21 @@ pronounciation of the Q.A. acronym. Ori pronounce Kyua as Q.A., but in reality this never happened. Today, just read the name as your instinct would: "Kyu-ah". +# Why is Kyua a third-party project? + +Kyua's main consumer is NetBSD. One could argue that Kyua should be +developed within NetBSD and maintained in the NetBSD source tree. However, +there is nothing in the Kyua project that inherently depends on NetBSD, and +maintaining it as a third-party package is a way to keep the developers +honest regarding portability. + +Ideally, other projects (such as FreeBSD) would make use of Kyua too for +their testing needs, and if that happened we would be able to share tests +with them pretty easily. Forcing a portable codebase in the upstream +repository helps in this. (Be aware that different individuals from +FreeBSD and Minix have shown interest in adopting Kyua for their respective +systems!) + # Main differences (aka "what to expect") As of version 0.5, Kyua has (or is supposed to have) feature parity with @@ -93,6 +111,13 @@ results into a single report. At the mo also lower the barrier of entry to writing test programs for NetBSD, as using the ATF libraries will become optional. +These interfaces are implemented as independent binaries, called testers, +that are fully scriptable. The testers are provided in the `kyua-testers` +package. The idea behind having these as independent programs is to +restrict the OS-specific code to a small subset of Kyua written in C, and +thus to allow the higher-level layers to be written in other languages +(possibly Lua). + ## Lua configuration files Kyua has two kind of configuration files: the Kyuafiles, which are the @@ -107,6 +132,19 @@ that the parser for these files is well- Lua will allow the implementation of more-intelligent test (and maybe even build) scripts. +## Direct HTML output + +All of the NetBSD continous build and testing systems provide status +reports through the releng web interface. In the case of ATF, this has +traditionally been tricky because ATF cannot generate HTML contents +directly; instead, `atf-report` generates XML output which later must be +postprocessed with `xsltproc` to create the HTML pages. + +Kyua has the ability to generate HTML reports straight from the tool, +without having to go through any XML toolchain. This means that NetBSD, +out of the box, can generate such reports and publish them with the builtin +httpd(8) server. + ## Heavier code base If you take a look at the Kyua distribution file, you may notice that it is @@ -141,6 +179,12 @@ components exist in pkgsrc, and are: * pkgsrc/devel/atf: The ATF tools, namely atf-run and atf-report. These are deprecated and this package should eventually disappear. +* pkgsrc/devel/kyua-testers: The Kyua testers, which provide the + `kyua-atf-tester` and `kyua-plain-tester` helper binaries. These + binaries implement the logic to execute test cases in an isolated manner + and to expose the test programs using a common and abstract command-line + interface. + * pkgsrc/devel/kyua-cli: The Kyua command-line interface, which provides a superset of the functionality of atf-run and atf-report. @@ -160,7 +204,7 @@ The easiest (but also the least "future- suite with Kyua is to use the backwards compatibility ATF tools provided by the kyua-atf-compat module. First of all, install the package: - $ cd /usr/pkgsrc/deve/kyua-atf-compat + $ cd /usr/pkgsrc/devel/kyua-atf-compat $ make install && make clean And then, running the test suite is as easy as: @@ -198,16 +242,16 @@ at these steps. To get started, install the Kyua packages: - $ cd /usr/pkgsrc/deve/kyua-cli + $ cd /usr/pkgsrc/devel/kyua-cli $ make install && make clean - $ cd /usr/pkgsrc/deve/kyua-atf-compat + $ cd /usr/pkgsrc/devel/kyua-atf-compat $ make install && make clean Once this is done, configure Kyua in the same way ATF is configured "out of the box" in NetBSD. Create the /usr/pkg/etc/kyua/kyua.conf file with these contents: - syntax('kyuafile', 1) + syntax('config', 1) unprivileged_user = '_tests' The next step is to populate /usr/tests with Kyuafiles, as Kyua is unable