File:  [NetBSD Developer Wiki] / wikisrc / projects / project / bulktracker.mdwn
Revision 1.3: download - view: text, annotated - select for diffs
Mon Feb 16 07:52:31 2015 UTC (6 years, 11 months ago) by dholland
Branches: MAIN
CVS tags: HEAD
Note that this partly exists now.

[[!template id=project

title="Bulk build tracker application"




Currently, bulk build results are sent to the pkgsrc-bulk mailing
list. To figure out if a package is or has been building successfully,
or when it broke, one must wade through the list archives and search
the report e-mails by hand. Furthermore, to figure out what commits if
any were correlated with a breakage, one must wade through the
pkgsrc-changes archive and cross-reference manually.

The project is to produce a web/database application that can be run
from the pkgsrc releng website on that tracks bulk build
successes and failures and provides search and crossreferencing

The application should subscribe to the pkgsrc-bulk and pkgsrc-changes
mailing lists and ingest the data it finds into a SQL database. It
should track commits to each package (and possibly infrastructure
changes that might affect all packages) on both HEAD and the current
stable branch, and also all successful and failed build reports on a
per-platform (OS and/or machine type) basis.

The web part of the application should be able to retrieve summaries
of currently broken packages, in general or for a specific platform
and/or specific branch. It should also be able to generate a more
detailed report about a single package, containing for example which
platforms it has been built on recently and whether it succeeded or
not; also, if it is broken, how long it has been broken, and the
history of package commits and version bumps vs. build results. There
will likely be other searches/reports wanted as well.

The application should also have an interface for people who do
partial or individual-package check builds; that is, it should be able
to generate a list of packages that have not been built since they
were last committed, on a given platform or possibly on a per-user
basis, and accept results from attempting to build these or subsets of
these packages. It is not entirely clear what this interface should be
(and e.g. whether it should be command-line-based, web-based, or what,
and whether it should be limited to developers) and it's reasonable to
expect that some refinements or rearrangements to it will be needed
after the initial deployment.

The application should also be able to record cross-references to the
bug database. To begin with at least it's reasonable for this to be
handled manually.

This project should be a routine web/database application; there is
nothing particularly unusual about it from that standpoint. The part
that becomes somewhat less trivial is making all the data flows work:
for example, it is probably necessary to coordinate an improvement in
the way bulk build results are tagged by platform. It is also
necessary to avoid importing the reports that appear occasionally on
pkgsrc-bulk from misconfigured pbulk installs.

Note also that "OS" and "machine type" are not the only variables that
can affect build outcome. There are also multiple compilers on some
platforms, for which the results should be tracked separately, plus
other factors such as non-default installation paths. Part of the
planning phase for this project should be to identify all the
variables of this type that should be tracked.

Also remember that what constitutes a "package" is somewhat slippery
as well. The pkgsrc directory for a package is not a unique key;
multiversion packages, such as Python and Ruby extensions, generate
multiple results from a single package directory. There are also a few
packages where for whatever reason the package name does not match the
pkgsrc directory. The best key seems to be the pkgsrc directory paired
with the package-name-without-version.

Some code already exists for this, written in Python using Postgres.
Writing new code (rather than working on this existing code) is
probably not a good plan.

CVSweb for NetBSD wikisrc <> software: FreeBSD-CVSweb