Running the Unit Tests¶
We use CTest for building and running our unit tests. This wraps the underlying cxxtest or other (e.g. pyunit) test code.
CMake/CTest: Command Line¶
The unit tests are currently excluded from the ‘all’ target. To build all of the tests, including the dependent framework code run:
cmake --build . --target AllTests
To build only one package of tests (and its dependencies):
cmake --build . --target KernelTest
To run all the tests:
To build and run all the tests in one shot:
cmake --build . --target check
To run a specific test or set of tests (will run all those that match the search string):
ctest -R KernelTest_TimerTest
So to run all tests in a suite (using a search string):
ctest -j8 -R KernelTest
To exclude things from your tests (matches the string as with the -R option) - useful for those building the performance tests:
ctest -j8 -E Performance
Useful CTest Options¶
ctest -h gives the full list of command line arguments. Some useful
ones to note are:
--output-on-failure: displays the log and any stderr output that is otherwise hidden by CTest.
--schedule-random: run the tests in a random order. Useful to weed out accidental dependencies
--repeat-until-fail: require each test to run times without failing in order to pass. Useful to try and find random failures
Running Unit Tests Directly¶
Starting in your build folder:
Running an entire test suite (e.g. KernelTest):
Running a specific test class.
Running a specific test from a CxxTest test class (not possible via CTest).
./bin/KernelTest MyTestClassName MySingleTestName
Running a specific test from a Python
unittesttest class (not possible via CTest).
PYTHONPATH=/path/to/build/bin python3 /path/to/src/Framework/PythonInterface/test/python/plugins/algorithms/MeanTest.py MeanTest.test_mean
Running Unit Tests With Visual Studio and ctest¶
Open the Mantid solution in Visual Studio. To run a subset of tests (for example
In the Solution Explorer, right click the project for the tests (in this case
UnitTests/AlgorithmsTest) and select Properties.
In the Debugging tab of Properties change the Command Arguments box to the name of the test, for example “AddNoteTest”.
Right click the directory again and select Debug->Start new instance.
Once the build has finished, open a file browser and navigate to the mantid build directory, run the command-prompt.bat file to open a command prompt and run
ctest -C Debug -V -R AddNoteTest
For this example, there should be several lines of output ending with the time taken and the line
100% tests passed, 0 tests failed out of 1
-R AddNoteTestoption runs all the tests, but note that any tests which were not built according to the above instructions will fail. Adding the
-Vincreases the output verbosity.
Running Unit Tests With Visual Studio¶
The unit tests can be run from within Visual Studio, following steps 1-3 above, with the addition in step 2 of;
Add the name of the test to the Target Name field in the General tab of Properties. Then add a breakpoint somewhere in the test header file.
Debugging unit tests¶
See the instructions here