Analyze NeoMutt

Wikipedia has a nice list of static analyzers for C source code. Those can be used to find bugs without compiling, executing and debugging NeoMutt.

If you have any questions, please send them to the developers’ mailing list: neomutt-devel@neomutt.org

Tool Description
clang-format Source code formatter
coccinelle Source code manipulator
compiler Enable warnings in gcc/clang
coverage Code coverage testing
coverity Source code anaylser
cppcheck Source code anaylser
cproto Function prototype generator
ctags Source tags generator
iwyu Header file checker
scan-build Source code anaylser
travis Continuous Integration
valgrind Run time memory checker

Clang-Format - Source code formatter

clang-format is a source code formatter – it changes source files according to a config file.

In NeoMutt we use it to:

  • Place {}s in the right place
  • Adjust the whitespace: indent and around operators
  • Sort the #includes (see the weighting strategy)
  • Align a pointer * to the variable, not the type
  • … and much more

Unlike many similar tools, it really understands the code it’s transforming. It uses clang to create an AST for the code.

NeoMutt’s config file ships with the code.

The config file looks like this:

Language: Cpp

TabWidth:          8
UseTab:            Never
IndentWidth:       2
ColumnLimit:       80
BreakBeforeBraces: Allman

Clang has documentation for all of the options.

Running it is as simple as:

clang-format -i source.c
  • As part of the release process, clang-format is run on all the ‘c’ source.
  • The header files are tidied by hand to preserve the whitespace layout.

Coccinelle - Source code manipulation

Coccinelle is a tool for manipulating C source code. Because it really understands C, you can make complex changes, e.g. If x is an integer, replace if (!x) with if (x == 0)

This means that you could define some code style rules and automatically check that they’re being used.

Coccinelle uses a transformation language that it applies to C source. The output is a diff that can be applied.

/* Source file to be checked */
int main()
{
  char *x;

  if (x == NULL)
    something();
}
// Find pointers that are checked against NULL
@@
type T;
identifier I;
statement S1;
@@

T *I;

(
- if (I == NULL)
+ if (!I)
S1
)
# Generate the diff
spatch --sp-file null-check.cocci source.c
--- source.c
+++ source.c
@@ -4,6 +4,6 @@ int main()
 {
   char *x;
 
-  if (x == NULL)
+  if (!x)
     something();
 }

Here are more examples that have been used on NeoMutt:

Compiler - Enable gcc/clang warnings

One of the simplest ways to check your code is to turn on lots of compiler warnings. You may not agree with some of them, but they highlight potential problems that can be easily avoided.

Extra Flags

It’s simple to add extra compiler/linker flags: set EXTRA_CFLAGS and/or EXTRA_LDFLAGS
They will be appended to NeoMutt’s settings, e.g.

./configure OPTIONS
# Enable debugging
make EXTRA_CFLAGS="-ggdb3 -O0 -DDEBUG"

ccache

The Compiler Cache ccache is a must for any developer.
Every time you compile a file, it keeps a hash of the preprocessed file and the object file that it created. If you build that file again (and it hasn’t changed), then the cached version will be used.

Developer Build Scripts

NeoMutt has published two build scripts for developers.

Their style makes it very easy to enable/disable or add/remove compilation options.

Coverage - Code coverage testing

When testing a program, it’s often useful to know which parts of the code have actually been used. Coverage testing collects statistics about a running program.

First the program needs to be compiled and linked with some extra options. This will generate a .gcno coverage files for each object.

CFLAGS  += -fprofile-arcs -ftest-coverage
LDFLAGS += -fprofile-arcs -ftest-coverage

When the program is run, every function will write data to a .gcda file. lcov can convert the saved data into an html table.

lcov -t "result" -o lcov.info -c -d config
genhtml -o lcov lcov.info

Coveralls performs the same function, but it has a much prettier website.

# Install the coveralls helper programm
pip install --user cpp-coveralls

export COVERALLS_REPO_TOKEN=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

# Update the stats, excluding (-e) certain files/dirs
coveralls -e mutt -e test -e dump -e main.c -e config/dump.c

Currently, the only part of NeoMutt that has a coverage report is the new config code:

Coverity - Source code anaylser

Coverity Scan is a free static-analysis cloud-based service for the open source community. As the free service is limited to a few scans per day, it isn’t run automatically.

  • Resource leaks
  • Potential NULL dereferences
  • Logically dead code
  • and much more

To run the check, create a branch of the NeoMutt repo and use the coverity .travis.yml file. The rest of the process is automated.

The latest statistics on NeoMutt can be found here:

For security, the details of the defects are not made public. If you wish to see the details, you must create an account (or login with GitHub) and click “Add me to project”.

CppCheck - Source code analyser

Since NeoMutt’s code base is relatively big, it is easier to execute CppCheck from the command line and let it save its analysis to an output file.

CppCheck is available on all major platforms, so it can probably be installed through the package manager of your system. In case you are using Windows (but not Cygwin) it is possible to download the installer from the website. We can use cppcheck-gui afterwards for filtering the warnings we want to analyze.

You can use following command for analyzing the source code

cppcheck --enable=all --language=c --std=c99 -i autosetup --platform=unspecified -D_POSIX_PATH_MAX=4096 -DPATH_MAX=2048 -I/usr/include -I/usr/include/mit-krb5 -I. --output-file=cppcheck.xml --xml --xml-version=2 <path to neomutt source code directory>

A complete description of all parameters can be found through man cppcheck or cppcheck --help. Some considerations:

  • --enable-all --output-file=<filename> --xml --xml-version=2 collect as many warning as possible and save them to a file, in order to view them later with cppcheck-gui
  • --platform=unspecified do not restrict the analysis to a 32 or 64 bit platform
  • --language=c --std=c99 NeoMutt follows this standard
  • -D_POSIX_PATH_MAX=4096 -DPATH_MAX=2048 can be left out, but CppCheck may warn that it does not know the values for those macros. Since most of the time the actual values are not relevant, you might just set them to two numeric values.
  • -i autosetup for excluding the autosetup directory
  • -I<path> where to find include files

If cppcheck takes too much time for analyzing the source code, it is always possible to analyze only specific files, tell it to test less macro combinations, or to not enable all warnings.

Cproto - Function prototype generator

Given a source file, cproto, will generate prototypes for all the functions. This is slightly useful on its own, but it can be used as the basis for:

  • analysing functions and parameters
  • generating doxygen comment blocks
cproto -D USE_SIDEBAR=1 -I .  -s source.c

Ctags - Source tags generator

ctags generates tags files that your editor can use to lookup symbols in the source: all the functions, structs, global variables, etc.

It can be run:

# Recursively
ctags -R .

# On specific files
ctags source1.c source2.c

Unfortunately, that will include some source that isn’t useful. Using find can help exclude some files and directories:

find . -name '*.[ch]' ! -path './autosetup/*' ! -path './test/*' ! -path './doc/*' ! -path './pgp*.c' | cut -b3- | xargs ctags

ctags can also extract certain types of information. This can be useful for analysis of the code.

ctags -R -x --c-kinds=f   . > functions.txt
ctags -R -x --c-kinds=gsu . > structs.txt
ctags -R -x --c-kinds=v   . > variables.txt

Include-What-You-Use - Header file checker

iwyu is a tool for deciding which #includes are needed, or not. Including the right files, and no more, can speed up builds. It also helps to show the dependencies of a file.

iwyu uses clang to create an AST for the code.

Example

A sample call and its output.

iwyu -I. -Xiwyu --no_comments -Xiwyu --pch_in_code -Xiwyu --mapping_file=neomutt.imp thread.c
thread.c should remove these lines:
- #include "globals.h"

The full include-list for thread.c:
#include "config.h"
#include "thread.h"
#include <limits.h>
#include <stdlib.h>
#include <time.h>
#include "body.h"
#include "header.h"
#include "mutt/mutt.h"
#include "sort.h"

Common options used are:

Option Description Notes
-I. Add an include dir . Passed to clang
-DSYMBOL Define a SYMBOL Passed to clang
-Xiwyu --no_comments Don’t add notes to the output IWYU option
-Xiwyu --pch_in_code The file has an important header first IWYU option
-Xiwyu --mapping_file=FILE Use this lookup file IWYU option

Mapping File

It’s hard for iwyu to work out which header files the developers think are important. For example, in NeoMutt, you can replace lots of library headers:

#include "mutt/message.h"
#include "mutt/address.h"
#include "mutt/rfc2047.h"
#include "mutt/mapping.h"
#include "mutt/hash.h"

with one helper header

#include "mutt/mutt.h"

The mapping file also provides fixes for some header files in glibc that cause confusion. It looks like this:

# Include one library header rather than the individual headers.
{ include: [ '@"mutt/.*"', private, '"mutt/mutt.h"', public ] },
{ include: [ '@"conn/.*"', private, '"conn/conn.h"', public ] },

Scan-Build - Source code anaylser

scan-build is a static analyzer and part of the clang family of tools. It can detect:

  • Resource leaks
  • Potential NULL dereferences
  • Logically dead code
  • and much more

It will create an HTML report for all the issues that it finds. It’s very simple to run:

# Configure NeoMutt as normal
./configure [OPTIONS]
# Run make under scan-build
scan-build make

Travis - Continuous Integration

Travis provides a continuous integration service which is connected to NeoMutt’s git repos on GitHub.

Each repo has a .travis.yml file which tells Travis what to do. After every commit to GitHub and for every Pull-Request, Travis performs the instructions in that file.

Here are some of they ways that NeoMutt uses this service.

Building of the source

To test the source code, Travis performs a set of builds with different configure options. .travis.yml installs all the dependencies that are needed, but the build is controlled by a separate script.

The script has per-branch rules to determine how many builds to do and which options to use. read more…

To test the website, Travis checks all of the HTML for errors.

.travis.yml installs Jekyll and html-proofer.

The testing is done by running the Rakefile.

Deployment of the translations

After a translator makes an update, Travis checks the results and updates the Translation Leaderboard

The translate branch has a custom .travis.yml and some scripts to generate a webpage.

On success, the website is automatically updated.
For more details, read Deployment using Travis.

Travis Tool

Travis also supply a command line tool for managing their service.

gem install travis

NeoMutt used this to encrypt ssh keys for use in automatic deployment.

Valgrind - Run time memory checker

Valgrind is a set of debugging tools.

Note: Running a program under will consume more memory and run much more slowly than normal.

Memchecker

The default tool is a memory-checker, e.g.

valgrind -v --leak-check=full --track-origins=yes neomutt

The results can be saved to a file by adding --log-file v.txt

It reports:

  • Leaked memory (and where it was allocated)
  • Out of bounds memory accesses
  • Conditionals that relied on uninitialised memory
LEAK SUMMARY:
   definitely lost: 1,368 bytes in 22 blocks
   indirectly lost: 179 bytes in 5 blocks
     possibly lost: 1,352 bytes in 18 blocks
   still reachable: 672,679 bytes in 640 blocks

Memchecker Suppressions

Memchecker reports every problem it finds, including any shared libraries. These false-positives can be ignored using a suppression file. To generate one, run the program with an extra argument:

valgrind -v --leak-check=full --track-origins=yes --gen-suppressions=all neomutt

Valgrind’s output will include suppression blocks like this:

{
   <insert_a_suppression_name_here>
   Memcheck:Leak
   match-leak-kinds: possible
   fun:calloc
   fun:g_malloc0
   fun:type_node_any_new_W
   fun:type_node_fundamental_new_W
   fun:g_type_register_fundamental
   fun:_g_object_type_init
   fun:gobject_init
   fun:gobject_init_ctor
   fun:call_init.part.0
   fun:call_init
   fun:_dl_init
   obj:/usr/lib64/ld-2.27.so
}

Cut out the ones you’d like to ignore, then put them in a file, sup.txt, then re-run your program:

valgrind -v --leak-check=full --track-origins=yes --suppressions=sup.txt neomutt

Callgrind

Callgrind is a profiler. It records which functions are called, how often and how long each one took, e.g.

valgrind --tool=callgrind neomutt

It creates a log file callgrind.out.[PID] which can be viewed in kcachegrind.

kcachegrind

Debugging with Valgrind

Running NeoMutt under a debugger can be a good way to find bugs, but sometimes it’s hard to see the exact point when something goes wrong. Using Valgrind and gdb in combination, it’s possible to trap the instruction that causes a buffer overrun, or reads an uninitialised piece of memory.

Note: Memory leaks can’t be detected as they happen.

Window 1:

valgrind --vgdb=yes --vgdb-error=0 neomutt [ARGS]

Window 2:

gdb neomutt
# Inside gdb
target remote | vgdb
continue

Valgrind adds extra commands to gdb to examine the memory. See Valgrind’s documentation for more details.

Search by Algolia