Tools Overview

Creating classes: class_maker.py

To make it faster to create a new class. This is a small python script located in /buildconfig/. It generates the .cpp, .h and test files for a class along with some code stubs. It can also flesh out more methods for new Algorithms, using the “–alg” option.

usage: class_maker.py [-h] [--force] [--test] [--alg] SUBPROJECT CLASSNAME
Utility to create Mantid class files: header, source and test.
positional arguments:
 SUBPROJECT  The subproject under Framework/; e.g. Kernel
 CLASSNAME   Name of the class to create
optional arguments:
 -h, --help  show this help message and exit
 --force     Force overwriting existing files. Use with caution!
 --test      Create only the test file.
 --alg       Create an Algorithm stub. This adds some methods common to
             algorithms.

Moving/Renaming classes: move_class.py

This python script is located in in /buidconfig/. It will move a class from one subproject to another and/or rename the class. Namespaces and cmakelists are adjusted. For details, run:

buildconfig/move_class.py --help

Deleting a class: delete_class.py

This python script is located in in /buildconfig/. It will delete a class from one subproject. CMakeList.txt is adjusted. For details, run:

buildconfig/delete_class.py --help

Profiling an algorithm

On Linux the build can be configured to generated algorithm profiling information. See AlgorithmProfiler for more details.

Leak checking etc

Linux

Memcheck

  • Keeps track of allocs/deallocs and reports anything missing at exit.
  • Slow but thorough
  • Useful options to run with

valgrind --tool=memcheck --leak-check=full --show-reachable=yes --num-callers=20 --track-fds=yes --track-origins=yes --freelist-vol=500000000 ``\ \ `` [args...]

Windows

Visual Leak Detector

  1. Setup the additional paths as defined in the readme file
  2. Adjust the configuration file, “C:Program FilesVisual Leak Detectorvld.ini” to output to both File and debugger by changing the ReportTo to

ReportTo = both

  1. Add #include <vld.h> to the system.h file in Kernel
  2. Compile everything in debug
  3. Running unit tests should now create a file memory_leak_report.txt in the test directory.
  4. IMPORTANT remove the #include <vld.ini> before checking in.

Thread checking

Helgrind or drd

  • Identifies race conditions & dead-locks
  • Slow but accurate
  • A pain to get working with OpenMP. GCC must be recompiled to use a different call to create OMP threads or helgrind/drd cannot “see” the thread calls. Use this script to recompile the same version off gcc that is onyour system. The script will need editing to change the appropriate variables.

Profiling

Linux

Callgrind/KCachegrind

gperftools

  • Takes snapshot of run and prints percentage of calls in functions

See here for a list of other tools: http://www.pixelbeat.org/programming/profiling/

Windows

Very Sleepy (Windows):

  • Start/stop recording of program using a button
  • Not as detailed or flexible as callgrind

IWYU

include what you use (iwyu) is a clang-based tool for determining what include statements are needed in C/C++ files. Below are instructions for getting it to run with mantid on linux which is a filled in version of this bug.

  1. Install the software. The version available from system installs should be fine (e.g. yum or apt-get).
  2. Get a copy of iwyu_tool.py which is in the project’s repository, but may not be installed if you got it from your operating system locations (e.g. yum).
  3. Run cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE. This will generate an extra file, compile_commands.json, in your build area which has instructions on compiling every file in mantid.
  4. Run iwyu_tool.py -p `pwd 2> iwyu.log` to generate the report of changes redirecting into the file iwyu.log. This will take a long time since it is going through the whole repository. If you want it for a single file, then supply that as an additional argument with full path. Only one file can be supplied at a time.
  5. Run fix_includes < iwyu.log and compile the results. Depending on how you installed iwyu, the program may be called fix_includes.py. If it doesn’t compile, the most likely suspect is that iwyu included a private header. See iwyu instructions for users for ways to handle this. Generally, they suggest deleting the offending lines.
  6. Check that your build path didn’t make it into source files. Since compile_commands.json has full paths, iwyu will put full paths in the include statements. This will not produce an error on your system, but it will on the build servers. The easiest way to check is to use the silver searcher to check for your username in your source tree.
  7. Enjoy your success.

Note: iwyu outputs to stderr and always returns a failure status code since it generates no output. The output stream also affects iwyu_tool.py

Convert Wiki Docs to Sphinx

wiki2rst reads in mediawiki formatted webpages and converts them to .rst files, for use in Sphinx. The code attempts to take all images and internal links and re-create the documentation structure in the Sphinx format.

Use

  • Having added mantid to your Python path wiki2rst is run by:
python wiki2rst.py -o <output_file.rst> <url_extension>
  • The <url_extension> is the part of the url after the main address (https://www.mantidproject.org/).

  • There are several additional options:
    • --index_url change the name of the main url address
    • --images-dir set a relative location for the images directory
    • --ref-link give a reference link
    • --ref-link-prefix give a link prefix
    • --add_handle add a handle for linking to the page
    • --page_handle the page handle to use [default page name]
    • --add_heading add a heading to the page (uses the page name)

Clang-tidy

Clang-tidy is a set of tools which allows a developer to detect and fix code which does not follow current best practices, such as unused parameters or not using range-based for loops. Primarily this is used for modernising C++ code.

The full list of clang-tidy checks can be seen here.

Installing

Mantid does not come packaged with clang-tidy; each developer must download it themselves. Windows users can utilise clang-tidy support for Visual Studio. For other operating systems, Mantid provides clang-tidy functionality through cmake.

  • Ubuntu: Run sudo apt-get install clang-tidy in the command line.
  • Windows: Download the Visual Studio extension. Windows can operate clang-tidy from Visual Studio alone and so do not need to touch cmake.

For non-Ubuntu systems, download the latest clang-tidy pre-compiled binary. Windows users should add to path when prompted.

Visual Studio

Once you have installed the clang-tidy extension, Visual Studio will have additional options under Tools -> Options -> Clang Power Tools. Here you can select which checks to run via a checkbox or by supplying a custom check string. For example:

-*,modernize-*

will disable basic default checks (-*) and run all conversions to modern C++ (modernize-*), such as converting to range-based for loops or using the auto keyword. Other settings should not require alteration for clang-tidy to work.

To run clang-tidy, right click on a target and highlight Clang Power Tools. You will have a number of options. Tidy will highlight code which fails one of the checks, whereas Tidy fix will automatically change your code.

Note: clang-tidy does not work on ALL BUILD so it is necessary to select a subtarget.

Cmake

In the cmake gui, find the CLANG_TIDY_EXE parameter. If you are a non-Linux developer, you may have to manually point to your clang-tidy install. Configure, and check the cmake log for the message clang-tidy found. If the clang-tidy not found warning was posted instead then it has not worked.

Once you have clang-tidy, there are several relevant parameters you will want to change:

  • ENABLE_CLANG_TIDY will turn on clang-tidy support.
  • CLANG_TIDY_CHECKS is a semi-colon separated list of checks for clang-tidy to carry out. This defaults to all modernize- checks.
  • APPLY_CLANG_TIDY_FIX will automatically change the code whenever a check has returned a result. The behaviour of ENABLE_CLANG_TIDY without this checked is to highlight issues only.

Configure the build to check that your selected options are reflected in CMAKE_CXX_CLANG_TIDY, and then generate. When you next build, clang-tidy will perform the selected checks on the code included in the target.

Note: There is a known issue that clang-tidy is only being applied to certain directories within Framework and Mantidplot.

Options

Some clang-tidy checks have optional arguments. For example, modernize-loop-convert, which changes loops to range-based, assigns a riskiness to each loop and can accept a MinConfidence argument to determine which risk levels to address.

Adding the optional arguments is clunky and will rarely be required, so it has not been directly added to Mantid’s cmake setup. To add optional arguments, add the following onto the end of CLANG_TIDY_CHECKS:

;-config={CheckOptions: [ {key: check-to-choose-option.option-name, value: option-value} ]}

For example, to convert all loops classified as risky or above, we would append:

;-config={CheckOptions: [ {key: modernize-loop-convert.MinConfidence, value: risky} ]}

CMake-format

CMake-format is a tool which is used to format individual CMakeLists.txt to make them easier to read. The package can be installed using pip install cmake_format or sudo pip install cmake_format.

To use cmake-format on a specific CMakeLists.txt file in the command line run

python -m cmake_format -c /path/to/mantid/.cmake-format.json -i /path/to/CMakeLists.txt

This will format the file using the config file .cmake-format.json which can be found in the root of the mantid directory.

There is an official Visual Studio extension, details of which can be found here.

To format on all the CMakeLists run the following from within the mantid source folder:

import os

dir = os.getcwd()
for path, subdirs, files in os.walk(dir):
        for file in files:
                if file.endswith("CMakeLists.txt"):
                        cmakefile = os.path.join(path, file)
                        print("Formatting " + cmakefile)
                        os.system('python -m cmake_format -c ' + os.path.join(dir, '.cmake-format.json') +' -i ' + cmakefile)