Install Gremlins Mutation Testing Tool on Ubuntu 24.04

The Gremlins is an open-source mutation testing tool for Go programming language that improves the quality of software tests by making small changes (mutations), to the code and checking if the tests can catch these changes. This tutorial explains how to install Gremlins mutation testing tool on Ubuntu 24.04.

Prepare environment

Make sure you have Go installed on your system before getting started. You can find installation instructions in a separate post.

Install Gremlins

Get the latest version of Gremlins release and assign it to variable:

GREMLINS_VERSION=$(curl -s "https://api.github.com/repos/go-gremlins/gremlins/releases/latest" | grep -Po '"tag_name": "v\K[0-9.]+')

Download tar.gz file from releases page:

wget -qO gremlins.tar.gz https://github.com/go-gremlins/gremlins/releases/latest/download/gremlins_${GREMLINS_VERSION}_linux_amd64.tar.gz

Extract executable to /usr/local/bin directory:

sudo tar xf gremlins.tar.gz -C /usr/local/bin gremlins

We can run this command to check the Gremlins version:

gremlins --version

Delete unneeded file:

rm -rf gremlins.tar.gz

Testing Gremlins

First, create a new directory for your project and navigate into it:

mkdir math && cd math

Initialize a Go module:

go mod init example.com/math

Create a Go source file:

nano calculator.go

Add the following code to define a function that checks if a number is even:

package math

func IsEven(num int) bool {
  return num%2 == 0
}

Now, create a test file:

nano calculator_test.go

Add the following code to write a test for the IsEven function:

package math

import (
  "testing"

  "github.com/stretchr/testify/assert"
)

func TestIsEven(t *testing.T) {
  t.Run("Number is even", func(t *testing.T) {
    assert.Equal(t, false, IsEven(3))
  })
}

Download the necessary module dependencies:

go mod tidy

Finally, run Gremlins to perform mutation testing:

gremlins unleash

This command introduces mutations into your code and checks whether your test can detect them. Output example:

Starting...
Gathering coverage... done in 582.902596ms
       LIVED ARITHMETIC_BASE at calculator.go:4:13
      KILLED CONDITIONALS_NEGATION at calculator.go:4:16

Mutation testing completed in 547 milliseconds 67 microseconds
Killed: 1, Lived: 1, Not covered: 0
Timed out: 0, Not viable: 0, Skipped: 0
Test efficacy: 50.00%
Mutator coverage: 100.00%

Uninstall Gremlins

To uninstall Gremlins, remove the related file:

sudo rm -rf /usr/local/bin/gremlins