[Scons-dev] Buildbot, Java and D...

[Dirk Bächle]

On 13.09.2014 17:44, William Blevins wrote:
> Dirke,
>
'tis a week t'early ferrr that. ;)

> The top two no_result calls need to work when we add the test for 
> "real", so we might as well fix them.  I think you answered your own 
> question, looks like the skip option needs to be enabled, so the test 
> is skipped rather than fails; however, the real question here is, why 
> are we running tests on a buildbot without Java installed?  Is 
> ignoring a whole language toolchain reasonable?
>

I would say so, yes. The testsuite can be run by any user who wants to 
check whether his patch introduces a regression. For this, he wants a 
clean run of all tests (passed or skipped) before, and after applying 
his changes. And we don't want to force him having to install Java, 
SWIG, Qt, D,..., and whatnot.
The tests have to cope with uninstalled applications/tools, and this 
gets tested as well...by not installing anything on every Buildbot. As 
long as we have *some* machines with Java in the mix, we should be doing 
fine.

Which leads to the more interesting question: How do we guarantee full 
test coverage, i.e. ensure that each test is properly run at least once 
on a Buildslave?

Dirk