Diff for /wikisrc/kyua.mdwn between versions 1.2 and 1.10

version 1.2, 2012/07/13 20:18:38 version 1.10, 2016/03/17 02:32:16
Line 1 Line 1
 [[!meta title="Kyua: An introduction for NetBSD users"]]  [[!meta title="Kyua: An introduction for NetBSD users"]]
 [[!toc ]]  
   **Contents**
   
   [[!toc levels=2]]
   
 The [Automated Testing Framework](ATF), or ATF for short, is a software  The [Automated Testing Framework](ATF), or ATF for short, is a software
 package composed of two parts: the *ATF libraries* and the *ATF tools*.  package composed of two parts: the *ATF libraries* and the *ATF tools*.
Line 32  This page provides instructions on how t Line 35  This page provides instructions on how t
 test suite so that you can experiment with the tool, familiarize yourself  test suite so that you can experiment with the tool, familiarize yourself
 with it and provide feedback early on.  with it and provide feedback early on.
   
   **If you would like more details on how the import of Kyua into NetBSD will
     happen, please see [[Kyua: The way into NetBSD|/kyua/import]].**
   
 # What's in the name?  # What's in the name?
   
 You should really think of Kyua as ATF 2.x.  Then, why isn't it just ATF  You should really think of Kyua as ATF 2.x.  Then, why isn't it just ATF
Line 50  pronounciation of the Q.A. acronym.  Ori Line 56  pronounciation of the Q.A. acronym.  Ori
 pronounce Kyua as Q.A., but in reality this never happened.  Today, just  pronounce Kyua as Q.A., but in reality this never happened.  Today, just
 read the name as your instinct would: "Kyu-ah".  read the name as your instinct would: "Kyu-ah".
   
   # Why is Kyua a third-party project?
   
   Kyua's main consumer is NetBSD.  One could argue that Kyua should be
   developed within NetBSD and maintained in the NetBSD source tree.  However,
   there is nothing in the Kyua project that inherently depends on NetBSD, and
   maintaining it as a third-party package is a way to keep the developers
   honest regarding portability.
   
   Ideally, other projects (such as FreeBSD) would make use of Kyua too for
   their testing needs, and if that happened we would be able to share tests
   with them pretty easily.  Forcing a portable codebase in the upstream
   repository helps in this.  (Be aware that different individuals from
   FreeBSD and Minix have shown interest in adopting Kyua for their respective
   systems!)
   
 # Main differences (aka "what to expect")  # Main differences (aka "what to expect")
   
 As of version 0.5, Kyua has (or is supposed to have) feature parity with  As of version 0.5, Kyua has (or is supposed to have) feature parity with
Line 93  results into a single report.  At the mo Line 114  results into a single report.  At the mo
   also lower the barrier of entry to writing test programs for NetBSD, as    also lower the barrier of entry to writing test programs for NetBSD, as
   using the ATF libraries will become optional.    using the ATF libraries will become optional.
   
   These interfaces are implemented as independent binaries, called testers,
   that are fully scriptable.  The testers are provided in the `kyua-testers`
   package.  The idea behind having these as independent programs is to
   restrict the OS-specific code to a small subset of Kyua written in C, and
   thus to allow the higher-level layers to be written in other languages
   (possibly Lua).
   
 ## Lua configuration files  ## Lua configuration files
   
 Kyua has two kind of configuration files: the Kyuafiles, which are the  Kyua has two kind of configuration files: the Kyuafiles, which are the
Line 107  that the parser for these files is well- Line 135  that the parser for these files is well-
 Lua will allow the implementation of more-intelligent test (and maybe even  Lua will allow the implementation of more-intelligent test (and maybe even
 build) scripts.  build) scripts.
   
   ## Direct HTML output
   
   All of the NetBSD continous build and testing systems provide status
   reports through the releng web interface.  In the case of ATF, this has
   traditionally been tricky because ATF cannot generate HTML contents
   directly; instead, `atf-report` generates XML output which later must be
   postprocessed with `xsltproc` to create the HTML pages.
   
   Kyua has the ability to generate HTML reports straight from the tool,
   without having to go through any XML toolchain.  This means that NetBSD,
   out of the box, can generate such reports and publish them with the builtin
   httpd(8) server.
   
 ## Heavier code base  ## Heavier code base
   
 If you take a look at the Kyua distribution file, you may notice that it is  If you take a look at the Kyua distribution file, you may notice that it is
Line 135  Kyua, as a project, is made up of a vari Line 176  Kyua, as a project, is made up of a vari
 ATF, because the ATF libraries are *not* being rewritten).  All of these  ATF, because the ATF libraries are *not* being rewritten).  All of these
 components exist in pkgsrc, and are:  components exist in pkgsrc, and are:
   
 * pkgsrc/devel/atf-libs: The C, C++ and POSIX shell libraries provided by  
   ATF.  These are *NOT* meant to be replaced by Kyua.  
   
 * pkgsrc/devel/atf: The ATF tools, namely atf-run and atf-report.  These  * pkgsrc/devel/atf: The ATF tools, namely atf-run and atf-report.  These
   are deprecated and this package should eventually disappear.    are deprecated and this package should eventually disappear.
   
 * pkgsrc/devel/kyua-cli: The Kyua command-line interface, which provides a  * pkgsrc/devel/kyua: The Kyua command-line interface, which provides a
   superset of the functionality of atf-run and atf-report.    superset of the functionality of atf-run and atf-report.
   
 * pkgsrc/devel/kyua-atf-compat: Drop-in replacements for atf-run and  
   atf-report that use kyua-cli in the backend.  
   
 # Running the NetBSD test suite  # Running the NetBSD test suite
   
 There are two ways to run the NetBSD test suite with Kyua.  The easy (or  There are two ways to run the NetBSD test suite with Kyua.  The easy (or
Line 160  The easiest (but also the least "future- Line 195  The easiest (but also the least "future-
 suite with Kyua is to use the backwards compatibility ATF tools provided by  suite with Kyua is to use the backwards compatibility ATF tools provided by
 the kyua-atf-compat module.  First of all, install the package:  the kyua-atf-compat module.  First of all, install the package:
   
     $ cd /usr/pkgsrc/deve/kyua-atf-compat      $ cd /usr/pkgsrc/devel/kyua-atf-compat
     $ make install && make clean      $ make install && make clean
   
 And then, running the test suite is as easy as:  And then, running the test suite is as easy as:
Line 196  Using the native command-line interface  Line 231  Using the native command-line interface 
 the existing NetBSD test suite is not prepared for Kyua.  Let's take a look  the existing NetBSD test suite is not prepared for Kyua.  Let's take a look
 at these steps.  at these steps.
   
 To get started, install the Kyua packages:  To get started, install the Kyua package:
   
     $ cd /usr/pkgsrc/deve/kyua-cli      $ cd /usr/pkgsrc/devel/kyua
     $ make install && make clean      $ make install clean
     $ cd /usr/pkgsrc/deve/kyua-atf-compat  
     $ make install && make clean  
   
 Once this is done, configure Kyua in the same way ATF is configured "out of  Once this is done, configure Kyua in the same way ATF is configured "out of
 the box" in NetBSD.  Create the /usr/pkg/etc/kyua/kyua.conf file with these  the box" in NetBSD.  Create a /usr/pkg/etc/kyua/kyua.conf file containing:
 contents:  
   
     syntax('kyuafile', 1)      syntax('config', 1)
     unprivileged_user = '_tests'      unprivileged_user = '_tests'
   
 The next step is to populate /usr/tests with Kyuafiles, as Kyua is unable  The next step is to populate /usr/tests with Kyuafiles, as Kyua is unable

Removed from v.1.2  
changed lines
  Added in v.1.10


CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb