What is Mutation test
Mutation testing allows you to test the efficiency of your unit tests (xUnit and NUnit), it allows you to examine the performance and the quality of the written tests. When you run the Mutation test, you will see percentage score reflecting your test coverage, and different inner testing processing like Killed and survived, the higher score your test get, the better test you are writing, ideally the score should be 100%, but in bigger projects anything above 80% is good.
Let’s say you have a Calculator class;
public class Calculator
{
public int Add(int x, int y)
{
return x + y;
}
public int Substract(int x, int y)
{
return x - y;
}
public int Multiply(int x, int y)
{
return x * y;
}
public int Divide(int x, int y)
{
return x / y;
}
}And your Calculator unit tests are;
public class CalculatorTests {
private readonly Calculator _sut = new();
[Theory]
[InlineData(1,2,3)]
public void AddTest(int x, int y, int expected)
{
Assert.Equal(_sut.Add(x,y), expected);
}
[Theory]
[InlineData(2, 2, 0)]
public void SubstractTest(int x, int y, int expected)
{
Assert.Equal(_sut.Substract(x, y), expected);
}
[Theory]
[InlineData(1, 1, 1)]
public void MultiplyTest(int x, int y, int expected)
{
Assert.Equal(_sut.Multiply(x, y), expected);
}
[Theory]
[InlineData(1, 1, 1)]
public void DivideTest(int x, int y, int expected)
{
Assert.Equal(_sut.Divide(x, y), expected);
}
}Now lets run the mutation test to check our testing score;
But first of all we need to install the Stryker tool;
You can install it from Nuget, or from the CLI (Tools → Command Line → PowerShell);
cd ../Location of your CalculatorTests project
dotnet tool install dotnet-stryker
After installation finished, you just type in the CLI
dotnet-stryker
And it will execute a full scan of your project test (CalculatorTests)
As you see in the CLI, the score is 62.50%
Killed:5
Survived:3
and an html webpage containing all the statistics is created
C:\Users\JoeChalhoub\source\repos\ConsoleApp2\CalculatorTests\StrykerOutput\2022-05-31.12-12-39\reports\mutation-report.html
If you open the report;
In this report you will see a kind of chart, telling you the score (62,50%), how many mutant tests executed (8 mutant tests), how many killed (5), and how many survived (3).
Note that the score is the percentage of Killed over total mutants
If you press the Program link;
Killed vs survived
In the first Add function we see it is green,, which means it is killed but didn’t survived after that, what we mean by killed, is the compiler replaced return x + y by return 0, and then it goes to execute the corresponding test, which is in this case the AddTest. In the AddTest, it applies the InlineData (1,2,3), the expected number is 3, while the actual number is 0 (returned by the actual function Add), so this test fails, and after the test was killed by the compiler, it didn’t pass and for that it didn’t survived.
In the Substract function, we see it is killed but survived. If you look at the corresponding test (SubstractTest), you see the Inline Data is InlineData(2, 2, 0) , so, if the compiler kills the test (return 0; instead of return x – y), SubstractTest will return the same result after it is killed, which is 0 (2 – 2 == 0), and that way the test did not survive after it has been killed, so for that reason survived will increment 1 and killed will increment 1 as well.
Now to fix your test score, all you need is to add more Inline data, and in other complicated cases you need to ameliorate your unit testing. In the SubstractTest scenario, you can fix your score by adding a new inline data, for example InlineData(3, 2, 1) , and that way it will not survive after it is killed.
Rpc Technology
Please subscribe to our YouTube channel;