Mutation Testing Leaving the Stone Age

71 views · Published 8 March 2018 · 37:49 · Indexed 20 November 2026

Channel: FOSDEM · 2018 · Science & Technology

Watch on YouTube

by Alex Denisov

At: FOSDEM 2017

For decades developers try to build a bug free software. Numerous techniquesand metrics were developed. However, there is still a need in an approach thatcan both assert program domain and provide some reliable metrics. And suchapproach exists. It is called Mutation Testing. Here is a brief overview:

There is a function and a test for that function. Any semantic change in thefunction (so called 'mutant') should break the test ('kill mutant'). If that'snot the case, then mutant is considered as survived. No mutants should surviveever. If they are, then either the test is bad or the function is not correct.

Research in this area exists since 70's. However, this approach is not widelyadopted. The computational cost is one of the blockers. Another one, is theway such systems are implemented: most of them do mutations at the AST level,hence they are tightly coupled to the particular languages.

We want to present you Mull: our implementation of Mutation Testing system. Weuse JIT and runtime compilation to speedup the system and make it accessiblefor everyday usage. Besides that we go down to the IR level, making the systemapplicable for any LLVM-based language such as C, C++, Rust, Objective-C, andmany others.

  * https://github.com/mull-project/mull

### Abstract

For decades developers try to build a bug free software. Numerous techniquesand metrics were developed. However, there is still a need in an approach thatcan both assert program domain and provide some reliable metrics. And suchapproach exists. It is called Mutation Testing. Here is a brief overview:

There is a function and a test for that function. Any semantic change in thefunction (so called 'mutant') should break the test ('kill mutant'). If that'snot the case, then mutant is considered as survived. No mutants should surviveever. If they are, then either the test is bad or the function is not correct.

Research in this area exists since 70's. However, this approach is not widelyadopted. The computational cost is one of the blockers. Another one, is theway such systems are implemented: most of them do mutations at the AST level,hence they are tightly coupled to the particular languages.

We want to present you Mull*: our implementation of Mutation Testing system.We use JIT and runtime compilation to speedup the system and make itaccessible for everyday usage. Besides that we go down to the IR level, makingthe system applicable for any LLVM-based language such as C, C++, Rust,Objective-C, and many others.

  * https://github.com/mull-project/mull

### Brief Outline

  * General introduction into Mutation Testing  * Overview of our Mutation Testing system  * High-level overview of the system design  * Results and findings from LLVM's test suite  * Future work  * Q&A


Room: K.3.201
Scheduled start: 2017-02-05 13:30:00

More from this channel