[HTCondor-users] Black box testing

[Tiago Macarios <tiagomacarios@xxxxxxxxx>]

Hi,

Apologies in advance since this is not a Condor question directly, but I guess this is one of the best places to ask this question.

tl;dr: Does anyone know any good black box testing framework for numerical applications? Preferably with condor support.

We develop a simulation software which relies on 20+ other packages. Because we cannot unit test third party code we have to do black box testing to guarantee that ours and partners changes are not changing the final code behavior. Our current procedure works great and we normally detect problems on third party libraries (even the Intel compiler). Problem is that lately we have been pushing the boundaries of the current workflow, so before re-inventing the wheel we are looking for other alternatives.

Basically what the current code does is:

Loop through a set of inputs: #inputs are files and settings

    - compute output

    - compare candidate output with known set of good results

We were hoping to use condor on the mix to speed things up. Current system used to use condor, but it is currently disabled and we miss it.

I had a look at Bamboo, Jenkins, Hudson… but it feels like I am missing something and none support condor afaik.

If someone has any experience/idea related to this please point me the way. Can be a separate direct email...

Thank you,

Mac