[BACK]Return to kyua.mdwn CVS log [TXT] [DIR] Up to [NetBSD Developer Wiki] / wikisrc

Diff for /wikisrc/kyua.mdwn between versions 1.1 and 1.8

version 1.1, 2012/07/13 01:35:00 version 1.8, 2013/02/01 22:20:02
Line 1 Line 1
 [[!meta title="Kyua: An introduction for NetBSD users"]]  [[!meta title="Kyua: An introduction for NetBSD users"]]
 [[!toc ]]  [[!toc levels=2]]
   
 The [Automated Testing Framework](ATF), or ATF for short, is a software  The [Automated Testing Framework](ATF), or ATF for short, is a software
 package composed of two parts: the *ATF libraries* and the *ATF tools*.  package composed of two parts: the *ATF libraries* and the *ATF tools*.
Line 32  This page provides instructions on how t Line 32  This page provides instructions on how t
 test suite so that you can experiment with the tool, familiarize yourself  test suite so that you can experiment with the tool, familiarize yourself
 with it and provide feedback early on.  with it and provide feedback early on.
   
   **If you would like more details on how the import of Kyua into NetBSD will
     happen, please see [[Kyua: The way into NetBSD|/kyua/import]].**
   
   # What's in the name?
   
   You should really think of Kyua as ATF 2.x.  Then, why isn't it just ATF
   2.x?
   
   To be honest, I never liked the ATF name: it was picked for me as part of
   the Google Summer of Code 2007 program and I did not think about changing
   it at that time.  A year later, I learned that the ATF acronym is severely
   overloaded, which makes it hard to find the project on popular search
   engines, and has unpopular connotations in specific countries.
   
   So, as part of the rewrite, I decided to choose a new name: a name that is
   not an acronym and thus can be easily pronounced, and a name that is quite
   unique in search results.  The name is Kyua, which is just a play on the
   pronounciation of the Q.A. acronym.  Originally, my intention was to
   pronounce Kyua as Q.A., but in reality this never happened.  Today, just
   read the name as your instinct would: "Kyu-ah".
   
   # Why is Kyua a third-party project?
   
   Kyua's main consumer is NetBSD.  One could argue that Kyua should be
   developed within NetBSD and maintained in the NetBSD source tree.  However,
   there is nothing in the Kyua project that inherently depends on NetBSD, and
   maintaining it as a third-party package is a way to keep the developers
   honest regarding portability.
   
   Ideally, other projects (such as FreeBSD) would make use of Kyua too for
   their testing needs, and if that happened we would be able to share tests
   with them pretty easily.  Forcing a portable codebase in the upstream
   repository helps in this.  (Be aware that different individuals from
   FreeBSD and Minix have shown interest in adopting Kyua for their respective
   systems!)
   
 # Main differences (aka "what to expect")  # Main differences (aka "what to expect")
   
 As of version 0.5, Kyua has (or is supposed to have) feature parity with  As of version 0.5, Kyua has (or is supposed to have) feature parity with
Line 75  results into a single report.  At the mo Line 111  results into a single report.  At the mo
   also lower the barrier of entry to writing test programs for NetBSD, as    also lower the barrier of entry to writing test programs for NetBSD, as
   using the ATF libraries will become optional.    using the ATF libraries will become optional.
   
   These interfaces are implemented as independent binaries, called testers,
   that are fully scriptable.  The testers are provided in the `kyua-testers`
   package.  The idea behind having these as independent programs is to
   restrict the OS-specific code to a small subset of Kyua written in C, and
   thus to allow the higher-level layers to be written in other languages
   (possibly Lua).
   
 ## Lua configuration files  ## Lua configuration files
   
 Kyua has two kind of configuration files: the Kyuafiles, which are the  Kyua has two kind of configuration files: the Kyuafiles, which are the
Line 89  that the parser for these files is well- Line 132  that the parser for these files is well-
 Lua will allow the implementation of more-intelligent test (and maybe even  Lua will allow the implementation of more-intelligent test (and maybe even
 build) scripts.  build) scripts.
   
   ## Direct HTML output
   
   All of the NetBSD continous build and testing systems provide status
   reports through the releng web interface.  In the case of ATF, this has
   traditionally been tricky because ATF cannot generate HTML contents
   directly; instead, `atf-report` generates XML output which later must be
   postprocessed with `xsltproc` to create the HTML pages.
   
   Kyua has the ability to generate HTML reports straight from the tool,
   without having to go through any XML toolchain.  This means that NetBSD,
   out of the box, can generate such reports and publish them with the builtin
   httpd(8) server.
   
 ## Heavier code base  ## Heavier code base
   
 If you take a look at the Kyua distribution file, you may notice that it is  If you take a look at the Kyua distribution file, you may notice that it is
Line 123  components exist in pkgsrc, and are: Line 179  components exist in pkgsrc, and are:
 * pkgsrc/devel/atf: The ATF tools, namely atf-run and atf-report.  These  * pkgsrc/devel/atf: The ATF tools, namely atf-run and atf-report.  These
   are deprecated and this package should eventually disappear.    are deprecated and this package should eventually disappear.
   
   * pkgsrc/devel/kyua-testers: The Kyua testers, which provide the
     `kyua-atf-tester` and `kyua-plain-tester` helper binaries.  These
     binaries implement the logic to execute test cases in an isolated manner
     and to expose the test programs using a common and abstract command-line
     interface.
   
 * pkgsrc/devel/kyua-cli: The Kyua command-line interface, which provides a  * pkgsrc/devel/kyua-cli: The Kyua command-line interface, which provides a
   superset of the functionality of atf-run and atf-report.    superset of the functionality of atf-run and atf-report.
   
Line 142  The easiest (but also the least "future- Line 204  The easiest (but also the least "future-
 suite with Kyua is to use the backwards compatibility ATF tools provided by  suite with Kyua is to use the backwards compatibility ATF tools provided by
 the kyua-atf-compat module.  First of all, install the package:  the kyua-atf-compat module.  First of all, install the package:
   
     $ cd /usr/pkgsrc/deve/kyua-atf-compat      $ cd /usr/pkgsrc/devel/kyua-atf-compat
     $ make install && make clean      $ make install && make clean
   
 And then, running the test suite is as easy as:  And then, running the test suite is as easy as:
Line 180  at these steps. Line 242  at these steps.
   
 To get started, install the Kyua packages:  To get started, install the Kyua packages:
   
     $ cd /usr/pkgsrc/deve/kyua-cli      $ cd /usr/pkgsrc/devel/kyua-cli
     $ make install && make clean      $ make install && make clean
     $ cd /usr/pkgsrc/deve/kyua-atf-compat      $ cd /usr/pkgsrc/devel/kyua-atf-compat
     $ make install && make clean      $ make install && make clean
   
 Once this is done, configure Kyua in the same way ATF is configured "out of  Once this is done, configure Kyua in the same way ATF is configured "out of
 the box" in NetBSD.  Create the /usr/pkg/etc/kyua/kyua.conf file with these  the box" in NetBSD.  Create the /usr/pkg/etc/kyua/kyua.conf file with these
 contents:  contents:
   
     syntax('kyuafile', 1)      syntax('config', 1)
     unprivileged_user = '_tests'      unprivileged_user = '_tests'
   
 The next step is to populate /usr/tests with Kyuafiles, as Kyua is unable  The next step is to populate /usr/tests with Kyuafiles, as Kyua is unable
Removed from v.1.1  
changed lines
  Added in v.1.8

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb