FSeam: A Mocking Framework That Requires No Change In Code (Part 2)

Published July 5, 2019

This post is the second part of guest writer Quentin Balland‘s series on FSeam, his testing framework that allows to test legacy code without heavy refactoring. Today Quentin walks us through how FSeam works. Quentin is a French C++ developer eager to learn and share his knowledge on his free time. You can find Quentin online on @FreeYourSoul68 and on his blog.

The purpose of this article isn’t to explain everything about FSeam into details as it could be too long and getting boring to read if we go into specifics features that everyone wouldn’t obviously use. We are going to see what are the limitations of GMock and how to use FSeam to try to solve those issues via a quick tutorial.

In this article and in the FSeam API, I call the action that change the behaviour of a function a dupe. It will prevent confusion between a mock, which is an instance of a class that has its behaviour altered, and the actual alterations done on this mock.

In order to understand this tutorial, it would be required to have the following:

A Basic understanding of how do unit test and testing framework works, here we will use Catch2.
Basic CMake knowledge.

If after reading this tutorial, you want to know more about what you can do, and how far you can go with FSeam, go to the GitHub repository. It contains a pretty complete documentation which explains everything in more depth with examples.

How does FSeam work?

I said in the previous articles that you could basically use FSeam in order to mock your classes without even touching your production code. It looks like a silver bullet. But I never explained how it does this.

FSeam is actually going to generate a mocked implementation of your production code by parsing your header files, and compile it instead of yours for your test.

To do so FSeam is split into three distinct parts:

A Code Generator: Developed in Python, it will parse a C++ header file and generate the implementation of the methods and functions that it encounters (of course if the implementation is done in the header file, FSeam shouldn’t re-implement them). This generator is based on an open source C++ header parser formerly developed by Jashua Cloutier, it is now maintained by robotpy on this git repository.
A CMake Helper: This is actually a very important part of FSeam, as everything happens at compile time. If we asked users to link the generated files themselves when they need to, FSeam would be nearly impossible to use correctly. Which is why we provide CMake functions in order to handle the generation of the source files and link those generated files in the test binary. We will see later how using CMake almost doesn’t make the CMake code grow, and is pretty easy to use.
A C++ Header only library: This library has to be used in order to manipulate the generated mock (check how many times a method/function has been called, with what arguments, or to dupe a method/function)

Changing the behaviour of a class at compile time has been named link seam in Michael Feathers book: Working effectively with legacy code, hence the name FSeam.

The classic way to do

Before entering into “how FSeam works”, I would like to show how we could test this code with the standard methods, explain the pro/cons of such method, to finally have an understanding of how FSeam do things differently.

#include <iostream>
#include "External.hh" // Contains DatabaseAccessor

class MainClass {
public:
    std::vector<std::string> getKnowledgeablePeopleFromCountry(const std::string &country) {
      auto knowns = dbAccess.retrieveKnowledgeablePeople(country);
      if (std::find(knowns.begin(), knowns.end(), "John Snow") != knowns.end()) {
        std::cerr << "You know nothing John Snow\n";
        throw 42;
      }
      return knowns;
    }

private:
    DatabaseAccessor dbAccess;
};

The above code contains the class we are going to test with GMock.

This is a quite classical external dependency issue we encounter in our everyday code when it comes down to unit testing. The above class contains a DatabaseAccessor object which is our own abstraction of the database connector (could be MySQL, Redis, PostgreSQL, whatever, it is not interesting in this case, let’s just assume that the DatabaseAccessor need a connection to a DB).

If we want to test the function getKnowledgeablePeopleFromCountry, we need a database connection… Well it is inconvenient and there is two way to get around this problem by using a mocking framework (GMock for instance):

By inheritance: we can take advantage of the dynamic polymorphism mechanism. To do so we should modify the code above in order to contain a pointer on an Abstract type or an interface representing a DatabaseAccessor. We also need a way to provide our mocked implementation (or production implementation) to the class, the most classical way to do so is to set the value via the constructor.

class MainClassGoogle {
public:
  MainClassGoogle(std::unique_ptr<IDbAccessorPolicy> dbAccess) : dbAccess(dbAccess) {}
  std::vector<std::string> getKnowledgeablePeopleFromCountry(const std::string &country) {   
    // ...
  }
  
private:
    std::unique_ptr<IDbAccessorPolicy> dbAccess;
};

By templatization: or we could templatize external dependency away, by doing so, it will be required to add a getter on the dbAccess instance in order to be able to manipulate it (you could also inject it via the constructor as in the inheritance method)

template<typename DbAccessorPolicy>
class MainClassGoogle {
public:
  std::vector<std::string> getKnowledgeablePeopleFromCountry(const std::string &country) {   
    // ...
  }
  
  DBAccessorPolicy &getDbAccessHandler() const { return _dbAccess; };

private:
    DbAccessorPolicy dbAccess;
};

Those techniques work fine but have the issue that you need to have your code comply with some requirements in order to use them (Interface usage, template usage). Which mean you need to refactor some of your code in order to use those techniques. The previous article already explained what were the other downsides of each of those techniques so we won’t go back to that into further details.

Now let’s see how FSeam works and solve the above explained issues. Note that the code examples of this tutorial are available on GitHub.

#1 Installation

It is needed to install FSeam first, you just have to follow this link to know how to do so. Some dependencies for the installation; catch2, python (and python package ply), C++17.

#2 FSeam Test Case

In this tutorial, we will have two different classes to test, one that contains a dependency on a Database connection represented by an object (GameOfThronesDatabase) and another one that will have dependency on free functions and static functions.

I will expose the Catch2 test cases and explain more or less line per line what FSeam does. I won’t explain too many features in order to keep it simple, these examples will be enough for you to start and use FSeam in most cases. For more complex needs, the framework can still help you, but I redirect you to the GitHub documentation that explains everything in further details.

#pragma once

#include <algorithm>

#include <vector>

#include <ClassesToMock.hh>

class ClassToTest {    
public:
    bool isWinnerOfGameOfThrones(const std::string &name) {
        GameOfThronesDatabase databaseConnectionHandler; // A new database connection is created out of the blue
        return databaseConnectionHandler.isPlayerWinnerOfGameOfThrones(name);
    }

    bool isGoodGameOfThronesSeason(int season) {
        std::vector<int> goodSeason = _dbCouchbase.getGoodSeasons(season);
        std::vector<int> underwhelmingSeason = _dbCouchbase.getNotSoGoodSeasons(season);
        if (goodSeason.empty()) {
            goodSeason = _dbSql.getGoodSeasons(season);
        }
        if (underwhelmingSeason.empty()) {
            underwhelmingSeason = _dbSql.getNotSoGoodSeasons(season);
        }
        if (std::find_first_of(goodSeason.begin(), goodSeason.end(), underwhelmingSeason.begin(),
                underwhelmingSeason.end()) != goodSeason.end()) {
            std::cerr << "Database inconsistency\n";
            throw std::string("Database inconsistency");
        }
        if (std::find(goodSeason.begin(), goodSeason.end(), season) != goodSeason.end())
            return true;
        if (std::find(underwhelmingSeason.begin(), underwhelmingSeason.end(), season) != underwhelmingSeason.end())
            return false;
        std::cerr << "The season is not referenced\n";
        throw std::string("The season is not referenced");
    }
    GameOfThronesDatabase &getDbSql() { return _dbSql; }
    GameOfThronesDatabase &getDbCouchbase() { return _dbCouchbase; }

private:
    GameOfThronesDatabase _dbSql;
    GameOfThronesDatabase _dbCouchbase;
};

In order to see how to mock classes with FSeam, we are going to test the class above.

This one is composed of two simple functions:

isWinnerOfGameOfThrones: that is just checking in the database if the given name is the winner of the Games of Thrones. This first function is interesting because it has the flaw of a lot of legacy code, it creates an instance on a dependency on the fly (databaseConnectionHandler instantiated in the body of the function) and it would be needed to extract this dependency in order to be able to mock it. We will see that it is not needed with FSeam.

isGoodGameOfThronesSeason: that is going to use two different instances of database connection (_dbSql and _dbCouchbase). One representing the cache (couchbase) and the other one representing the persistent database (sql).
Fetching the data from one or the other and verifying if the given season is good or not that good.

#2.1 Test a class: `isWinnerOfGameOfThrones`

Usually, mocking frameworks require direct access to the instance of the object that you need to mock. But the advantage of having our mock linked at compile time make us able to alter the behaviour of any object instantiated at any time easily; the below example shows you how:

#include <catch.hpp>
#include <vector>
#include <FSeamMockData.hpp>
#include "ClassToTest.hh"

using namespace FSeam;

TEST_CASE("Test Mocking default behavior on class") {

    // Create the TestingClass
    ClassToTest testingClass;

    // Get the default mock for the class ClassToTest
    auto fseamMock = FSeam::getDefault<::GameOfThronesDatabase>();

    SECTION("Check number of time called") {
        fseamMock->dupeReturn<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(true);
        REQUIRE(testingClass.isWinnerOfGameOfThrones("The Broken"));
        fseamMock->dupeReturn<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(false);
        REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Not knowledgeable guy"));
        REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Dragon girl"));        
        REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Wolf girl"));
        // Because this was quite unnexpected, we had 4 tries before finding the correct answer
        REQUIRE(fseamMock->verify(FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones::NAME, 4));
    }

    SECTION("Check argument on call") {
        // We check that at least 1 call has been done to isPlayerWinnerOfGameOfThrones with "The Broken" as parameter
        fseamMock->expectArg<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(Eq(std::string("The Broken")));
        // We check that exactly 3 call has been done to isPlayerWinnerOfGameOfThrones with "Not knowledgeable guy" as parameter
        fseamMock->expectArg<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(Eq(std::string("Not knowledgeable guy")), VerifyCompare{3});
        // We check that isPlayerWinnerOfGameOfThrones with "LittleFinger" as parameter has never been called
        fseamMock->expectArg<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(Eq(std::string("LittleFinger")), NeverCalled{});
        fseamMock->dupeReturn<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(false);
        REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Not knowledgeable guy"));
        REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Not knowledgeable guy"));        
        REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Not knowledgeable guy"));
        fseamMock->dupeReturn<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(true);
        REQUIRE(testingClass.isWinnerOfGameOfThrones("The Broken"));
        REQUIRE(fseamMock->verify(FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones::NAME)); // verify expectations
   
    }
    FSeam::MockVerifier::cleanUp();
}

Let’s go through this code step by step.

Get the FSeam MockHandler:

    // Create the TestingClass
    ClassToTest testingClass;
    // Get the default mock for the class ClassToTest
    auto fseamMock = FSeam::getDefault<::GameOfThronesDatabase>();

First, we instantiate the ClassToTest we want to unit test, the second line is more important. FSeam works with MockHandler (an object of type FSeam::MockClassVerifier), this object contains the behaviour you want your mock to have. It also stores how the mock has been used (how many times each method has been called, with what argument and so on). Those pieces of information can then be used to do your assertion.

There are multiple ways to get those Handlers, FSeam::getDefault<TypeToMock> return the default mock handler used for the given mocked type.

Actually, the only thing that can differ when mocking a static function/a free function or class with FSeam, is the way to get the MockHandler to manipulate.

Dupe return value and Assert number of calls:

SECTION("Check number of time called") {
    fseamMock->dupeReturn<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(true);
    REQUIRE(testingClass.isWinnerOfGameOfThrones("The Broken"));
    fseamMock->dupeReturn<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(false);
    REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Not knowledgeable guy"));
    REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Dragon girl"));        
    REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Wolf girl"));
    // Because this was quite unexpected, we had 4 tries before finding the correct answer
    REQUIRE(fseamMock->verify(FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones::NAME, 4));
}

Here is how FSeam is going to be used in most cases. Under the namespace FSeam::ClassMockedName, a “blank type” is generated for each method of the type mocked (here GameOfThronesDatabase).

Without entering into implementation details, you can use dupeReturn in order to decide what is going to be the returned value of your method. This method is present at the MockHandler level and takes only one parameter and has to be of the type that is returned by the function (a compilation error “undefined reference” is generated otherwise).

// Dupe the return value of the method isPlayerWinnerOfGameOfThrones to always return true
fseamMock->dupeReturn<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(true);

Then we call the function of our test class, it returns true (as expected). The string we send has actually no impact on the answer as isWinnerOfGameOfThrones will always return true.

Then for the fun of it, we change the return value (because we can) to false, and call the isWinnerOfGameOfThrones method of the ClassToTest some more.

REQUIRE(fseamMock->verify(FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones::NAME, 4));

Another important function of the FSeam MockHandler, verify, this function is used in order to check that the given method has been called a certain number of times (4 times in this case).

If you don’t specify any number, the verify method is going to check if the method has been called at least once (you can find additional details here

Expect your methods to be called with specific arguments:

SECTION("Check argument on call") {
    // We check that at least 1 call has been done to isPlayerWinnerOfGameOfThrones with "The Broken" as parameter
    fseamMock->expectArg<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(Eq(std::string("The Broken")));
    // We check that exactly 3 call has been done to isPlayerWinnerOfGameOfThrones with "Not knowledgeable guy" as parameter
    fseamMock->expectArg<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(Eq(std::string("Not knowledgeable guy")), VerifyCompare{3});
    // We check that isPlayerWinnerOfGameOfThrones with "LittleFinger" as parameter has never been called
    fseamMock->expectArg<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(Eq(std::string("LittleFinger")), NeverCalled{});
    fseamMock->dupeReturn<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(false);
    REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Not knowledgeable guy"));
    REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Not knowledgeable guy"));        
    REQUIRE_FALSE(testingClass.isWinnerOfGameOfThrones("Not knowledgeable guy"));
    fseamMock->dupeReturn<FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones>(true);
    REQUIRE(testingClass.isWinnerOfGameOfThrones("The Broken"));
    REQUIRE(fseamMock->verify(FSeam::GameOfThronesDatabase::isPlayerWinnerOfGameOfThrones::NAME)); // verify expectations
}

We can add expectations on the arguments that are going to be sent to our mocked method by using expectArg<FunctionName>. Here we basically set expectations as follow: We want isPlayerWinnerOfGameOfThrones to be called at least once with “The Broken” as input, we also want it to be called exactly 3 times with “Know nothing guy” but never with “LittleFinger”.

We then launch our method (following our expectations) and we call the verify method on the mock handler. The verify function is going to validate our expectations.

It is important to use the verify method when using expectArg as it is at this moment that the expectations are checked

There are two important things to note when using expectArg:

It is required that the expectations are set before starting the test.
In order to validate those expectations, verify method has to be called.

Do not forget to cleanup:

FSeam is working with a singleton object that lives for the entirety of the test. You need to cleanup FSeam mock at the end of each test in order to ensure that there is no pollution for the next test case started. To do so, you need to write this line:

FSeam::MockVerifier::cleanUp();

#2.2 Test multiple instances of the same class type:

`isGoodGameOfThronesSeason`

Now let’s try to test the isGoodGameOfThronesSeason method of our ClassToTest. Previously, we defined a default mocked behaviour for the class, GameOfThronesDatabase, but it wouldn’t work here as we would like to have different behaviour from different instances of the mock on the same methods (like we would normally do with GMock for example).

In this case, we decided to have a getter method returning a handler on the connector objects (but we could have just injected the dependency via the constructor).

As previously said, the only thing that will actually change now is the way to retrieve the MockHandler. When the mock handler is retrieved, the way to dupe, add expectations and verify are exactly the same.

#include <catch.hpp>
#include <vector>
#include <FSeamMockData.hpp>
#include "ClassToTest.hh"

using namespace FSeam;

TEST_CASE("Class Mock Specific Instance") {
    // Create the TestingClass
    ClassToTest testingClass;
    // Get the mock instances
    auto fseamMock_SqlDatabase = FSeam::get(&testingClass.getDbSql());
    auto fseamMock_CoucheBase = FSeam::get(&testingClass.getDbCouchbase());

    SECTION("Test CoucheBase without duping SQL") {
        // Couchbase return values
        fseamMock_CoucheBase->dupeReturn<FSeam::GameOfThronesDatabase::getGoodSeasons>(std::vector<int>({1,2,3,4,5,6}));
        fseamMock_CoucheBase->dupeReturn<FSeam::GameOfThronesDatabase::getNotSoGoodSeasons>(std::vector<int>({7,8}));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(1));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(2));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(3));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(4));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(5));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(6));
        REQUIRE_FALSE(testingClass.isGoodGameOfThronesSeason(7));
        REQUIRE_FALSE(testingClass.isGoodGameOfThronesSeason(8));
        // Check fseamMock_CoucheBase has been called 8 times (for both methods)
        REQUIRE(fseamMock_CoucheBase->verify(FSeam::GameOfThronesDatabase::getGoodSeasons::NAME, 8));
        REQUIRE(fseamMock_CoucheBase->verify(FSeam::GameOfThronesDatabase::getNotSoGoodSeasons::NAME, 8));
        // Because data are present in the cache, SQL connector is never called
        REQUIRE(fseamMock_SqlDatabase->verify(FSeam::GameOfThronesDatabase::getGoodSeasons::NAME, FSeam::NeverCalled{}));
        REQUIRE(fseamMock_SqlDatabase->verify(FSeam::GameOfThronesDatabase::getNotSoGoodSeasons::NAME, FSeam::NeverCalled{}));
    }

    SECTION("Test Couchbase (goodSeason) and SQL (badSeason)") {
        // Couchbase will return goodSeasons, SQL will return bad seasons
        fseamMock_CoucheBase->dupeReturn<FSeam::GameOfThronesDatabase::getGoodSeasons>(std::vector<int>({1,2,3,4,5,6}));
        fseamMock_SqlDatabase->dupeReturn<FSeam::GameOfThronesDatabase::getNotSoGoodSeasons>(std::vector<int>({7,8}));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(1));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(2));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(3));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(4));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(5));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(6));
        REQUIRE_FALSE(testingClass.isGoodGameOfThronesSeason(7));
        REQUIRE_FALSE(testingClass.isGoodGameOfThronesSeason(8));
        REQUIRE(fseamMock_CoucheBase->verify(FSeam::GameOfThronesDatabase::getGoodSeasons::NAME, 8));
        REQUIRE(fseamMock_CoucheBase->verify(FSeam::GameOfThronesDatabase::getNotSoGoodSeasons::NAME, 8));
        REQUIRE(fseamMock_SqlDatabase->verify(FSeam::GameOfThronesDatabase::getNotSoGoodSeasons::NAME, 8));
        // Because data are present in the cache, getGoodSeasons is never called for SQL
        REQUIRE(fseamMock_SqlDatabase->verify(FSeam::GameOfThronesDatabase::getGoodSeasons::NAME, FSeam::NeverCalled{}));
    }

    SECTION("Test Couchbase (badSeason) and SQL (goodSeason)") {
        // Couchbase will return goodSeasons, SQL will return bad seasons
        fseamMock_CoucheBase->dupeReturn<FSeam::GameOfThronesDatabase::getNotSoGoodSeasons>(std::vector<int>({7,8}));
        fseamMock_SqlDatabase->dupeReturn<FSeam::GameOfThronesDatabase::getGoodSeasons>(std::vector<int>({1,2,3,4,5,6}));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(1));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(2));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(3));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(4));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(5));
        REQUIRE(testingClass.isGoodGameOfThronesSeason(6));
        REQUIRE_FALSE(testingClass.isGoodGameOfThronesSeason(7));
        REQUIRE_FALSE(testingClass.isGoodGameOfThronesSeason(8));
        REQUIRE(fseamMock_CoucheBase->verify(FSeam::GameOfThronesDatabase::getGoodSeasons::NAME, 8));
        REQUIRE(fseamMock_CoucheBase->verify(FSeam::GameOfThronesDatabase::getNotSoGoodSeasons::NAME, 8));
        REQUIRE(fseamMock_SqlDatabase->verify(FSeam::GameOfThronesDatabase::getGoodSeasons::NAME, 8));
        // Because data are present in the cache, getNotSoGoodSeasons is never called for SQL
        REQUIRE(fseamMock_SqlDatabase->verify(FSeam::GameOfThronesDatabase::getNotSoGoodSeasons::NAME, FSeam::NeverCalled{}));
    }

    SECTION("Test database incoherency") {
        fseamMock_CoucheBase->dupeReturn<FSeam::GameOfThronesDatabase::getGoodSeasons>(std::vector<int>({1,2,3,4,5,6}));
        fseamMock_CoucheBase->dupeReturn<FSeam::GameOfThronesDatabase::getNotSoGoodSeasons>(std::vector<int>({1,2,3,4,5,6}));
        REQUIRE_THROWS_WITH(testingClass.isGoodGameOfThronesSeason(1), Catch::Contains("Database incoherency"));
        // Because data are present in the cache, SQL connector is never called
        REQUIRE(fseamMock_SqlDatabase->verify(FSeam::GameOfThronesDatabase::getGoodSeasons::NAME, FSeam::NeverCalled{}));
        REQUIRE(fseamMock_SqlDatabase->verify(FSeam::GameOfThronesDatabase::getNotSoGoodSeasons::NAME, FSeam::NeverCalled{}));
    }

    SECTION("Test undefined season") {
        fseamMock_CoucheBase->dupeReturn<FSeam::GameOfThronesDatabase::getGoodSeasons>(std::vector<int>({1,2,3,4,5,6}));
        fseamMock_CoucheBase->dupeReturn<FSeam::GameOfThronesDatabase::getNotSoGoodSeasons>(std::vector<int>({8}));
        REQUIRE_THROWS_WITH(testingClass.isGoodGameOfThronesSeason
(7), Catch::Contains("The season is not referenced"));
        // Because data are present in the cache, SQL connector is never called
        REQUIRE(fseamMock_SqlDatabase->verify(FSeam::GameOfThronesDatabase::getGoodSeasons::NAME, FSeam::NeverCalled{}));
        REQUIRE(fseamMock_SqlDatabase->verify(FSeam::GameOfThronesDatabase::getNotSoGoodSeasons::NAME, FSeam::NeverCalled{}));
    }
    FSeam::MockVerifier::cleanUp();
}

Looks scary? Well not at all, actually, you already know everything about how this code works now! The only real difference is the way we retrieve the MockHandler, and it is summed up in those 3 lines.

// Create the TestingClass
ClassToTest testingClass(dbSql, dbCouchbase);
// Get the mock instances
auto fseamMock_SqlDatabase = FSeam::get(&testingClass.getDbSql());
auto fseamMock_CoucheBase = FSeam::get(&testingClass.getDbCouchbase());

As before, we create the instance of the ClassToTest, but this time we retrieve the specific instance MockHandler by using FSeam::get(PointerOnMock). And the rest is exactly the same, dupeReturn, expectArg and verify works the same way than before on fseamMock_SqlDatabase and fseamMock_CoucheBase.

The test case is then quite straightforward, we mock separately each instances using dupeReturn in order to enter in the piece of code we want.

And again (at the end of each test) do not forget to call FSeam::MockVerifier::cleanUp() to clean up the FSeam context.

#2.3 Test a free function or a static method

/**
* ClassToTest.hh
*/

#include <string>

class ClassToTest {
public:
    bool isOneOfBestGameInTheWorld(const std::string &user, const std::string &game);
    bool isFavoriteNumber(int number);
};

/**
* ClassToTest.cpp
*/

#include <algorithm>
#include <FreeFunction.hh>
#include <StaticFunction.hh>
#include <iostream>
#include "include/ClassToTest.hh"

bool ClassToTest::isOneOfBestGameInTheWorld(const std::string &user, const std::string &game) {
    std::vector<std::string> games = DatabaseAccessor::getAllGames();
    if (games.empty())
        throw std::string("We live in a sad and cruel world without game :'(");
    std::vector<std::string> favoriteGames = DatabaseAccessor::getFavoriteGameForUser(user, game);
    if (favoriteGames.empty())
        throw std::string("Really?...");
    return std::find(favoriteGames.begin(), favoriteGames.end(), game) != favoriteGames.end();
}

bool ClassToTest::isFavoriteNumber(int number) {
    int random = generateRandomNumber();
    return number == random;
}

For this example, let’s change our ClassToTest (but I keep Games Of Thrones theme :p). This one is split into two different files (in order to show a different way to compile in the last part of the tutorial).

In the above ClassToTest, we need to mock a free function called generateRandomNumber() and two static methods of the class DatabaseAccessor, getFavoriteGameForUser(string user, string game) and getAllGames().

Testing the Free Function:

#include <catch.hpp>

#include <FSeamMockData.hpp>

#include "../include/ClassToTest.hh"

TEST_CASE("FreeFunction") {
    ClassToTest testingClass;
    auto mockFreeFunctions = FSeam::getFreeFunc();
    mockFreeFunctions->dupeReturn<FSeam::FreeFunction::generateRandomNumber>(42);
    SECTION("Check return value") {
        REQUIRE(testingClass.isFavoriteNumber(42));
        REQUIRE_FALSE(testingClass.isFavoriteNumber(-1));
        REQUIRE_FALSE(testingClass.isFavoriteNumber(1337));
        REQUIRE_FALSE(testingClass.isFavoriteNumber(16));
        SECTION("Check number of time called") {
            REQUIRE_FALSE(mockFreeFunctions->verify(FSeam::FreeFunction::generateRandomNumber::NAME, FSeam::NeverCalled{}, false));
            REQUIRE(mockFreeFunctions->verify(FSeam::FreeFunction::generateRandomNumber::NAME, 4));
        } // End section : Check number of time called
    } // End section : Check return value
    FSeam::MockVerifier::cleanUp();
} // End TestCase : FreeFunction

Here we go, the third and last way to retrieve a MockHandler (last because we use the same way for a static method), FSeam::getFreeFunc(). And the rest is the same.

The method name will be found in the namespace FSeam::FreeFunction.

Testing the static method:

#include <catch.hpp>
#include <FSeamMockData.hpp>
#include "../include/ClassToTest.hh"

TEST_CASE("StaticFunction") {

    ClassToTest testingClass;
    auto mockStaticFunctions = FSeam::getFreeFunc(); // static functions are considered as free functions

    SECTION("Check call") {
        mockStaticFunctions->dupeReturn<FSeam::FreeFunction::getAllGames>(std::vector<std::string>({"FyS", "SC2"}));
        mockStaticFunctions->dupeReturn<FSeam::FreeFunction::getFavoriteGameForUser>(std::vector<std::string>({"SC2"}));
        REQUIRE(testingClass.isOneOfBestGameInTheWorld("UserName", "SC2"));
        REQUIRE_FALSE(testingClass.isOneOfBestGameInTheWorld("UserName", "FyS"));
        REQUIRE_FALSE(testingClass.isOneOfBestGameInTheWorld("UserName", "Warcraft3"));
        SECTION("Check number of time called") {
            REQUIRE(mockStaticFunctions->verify(FSeam::FreeFunction::getAllGames::NAME, 3));
            REQUIRE(mockStaticFunctions->verify(FSeam::FreeFunction::getFavoriteGameForUser::NAME, 3));
        } // End section : Check number of time called
    } // End section : Check call
    FSeam::MockVerifier::cleanUp();
} // End TestCase : StaticFunction

As said, getting the static method MockHandler is exactly the same as for the free function, I know it may look counter-intuitive, but the name of the functions are also in FSeam::FreeFunction. The reason is that static methods act exactly like free functions so it was more convenient from an implementation point of view.

#3 Let’s compile it!

Our tests are using Catch2, it has the advantage to be fully integrated with FSeam (FSeam does an automatically register of the Catch2 test via the CMake function catch_discover_tests). Nothing is preventing you from using any other testing framework, but you will need to do the CTest registration by yourself.

## ============ CLIENT FACING ====================
## Function to call in order to generate a test executable from the generated FSeam mock and the provided test source
##
 ## Using CMake Parse Argument (explicitly named in the function call)
 ## Mandatory
 ## arg DESTINATION_TARGET : target name of the test executable generated via this method
 ## arg TST_SRC  : files containing the actual test to compile (Catch2, GTest test files for example)
## arg TO_MOCK  : files to mock for this specific given test
##
## either
## arg TARGET_AS_SOURCE : target of the library that contains the code to test
## arg FILES_AS_SOURCE  or source file containing the code to test
## arg FOLDER_INCLUDES  with includes folder
## The above either would be translated to : TARGET_AS_SOURCE || (FILES_AS_SOURCE && FOLDER_INCLUDES)
##
## optional
## arg MAIN_FILE  : file containing the main (if any), this file will be removed from the compilation of the test
##
function(addFSeamTests)

The above function declaration is directly taken from the GitHub repository. It is the only function needed to use FSeam with CMake which is using cmake_parse_arguments.

# find the FSeam dependency, check if installed and include CMake functions
 find_package(FSeam)
# required as we are going to launch the test via CTest
 enable_testing()

First of all, you need to write is those two lines in your CMake files in order to include the functions you will need (and check that FSeam is correctly installed).

And here is the file system with the files we are going to compile below (everything coming from the GitHub)

#3.1 Compile with a defined set of files

##
## In This example we directly specify where is the include folders and what are the source to
compile
## In case of a big project, using this method make you able to only compile what you want to
test
##
addFSeamTests(
DESTINATION_TARGET testFSeam_3 # FSeam testing binary target created for this testcase
FILES_AS_SOURCE
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/ClassToTest.cpp
FOLDER_INCLUDES ${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/include
TST_SRC
${CMAKE_CURRENT_SOURCE_DIR}/test.cpp
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/test/MockFreeFunctionTestCase.cpp
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/test/MockStaticFunctionTestCase.cpp
TO_MOCK
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/include/FreeFunction.hh
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/include/StaticFunction.hh)

The above snippet shows you how to compile your tests by providing a comprehensive list of the file you need to compile. Here is a detailed explanation of the arguments:

FILES_AS_SOURCE is the list of files that contain the code you want to test.
FOLDER_INCLUDE is the list of folders containing your includes (in order to have your source files finding their includes)
TST_SRC is the list of files that contain the testing code (basically the files containing GTest / Catch2 / AnyTestingFramework tests)
TO_MOCK is the list of header files that contain the classes / functions signatures you want to mock with FSeam.

#3.2 Compile with a binary target

## Create an executable containing the code we want to test
add_executable(classMock_Example_target
${CMAKE_CURRENT_SOURCE_DIR}/ClassMock_Example/main.cpp
${CMAKE_CURRENT_SOURCE_DIR}/ClassMock_Example/ClassesToMock.hh
${CMAKE_CURRENT_SOURCE_DIR}/ClassMock_Example/ClassesToMock.cpp
${CMAKE_CURRENT_SOURCE_DIR}/ClassMock_Example/ClassToTest.hh)
include_directories(classMock_Example_target ${CMAKE_CURRENT_SOURCE_DIR}/ClassMock_Example)
##
## In This example we give the target that contains the code we want to test, if the target contains a main (if it is an executable)
## it is required to provide the name path of the file providing this main (in order to remove it from the compilation) via the Argument MAIN_FILE
##
## This method is particularly useful as fast to implement (no need to explicitly specify the files to compile)
## it is a time-consuming method that compile-file that are not obviously needed for the test (usage of ccache is recommended to save time)
##
addFSeamTests(
DESTINATION_TARGET testFSeam_1 # FSeam testing binary target created for this testcase
TARGET_AS_SOURCE classMock_Example_target
MAIN_FILE ${CMAKE_CURRENT_SOURCE_DIR}/ClassMock_Example/main.cpp
TST_SRC
${CMAKE_CURRENT_SOURCE_DIR}/test.cpp
${CMAKE_CURRENT_SOURCE_DIR}/ClassMock_Example/test/MockSpecificInstanceTestCase.cpp
${CMAKE_CURRENT_SOURCE_DIR}/ClassMock_Example/test/MockUsingDefaultTestCase.cpp
TO_MOCK
${CMAKE_CURRENT_SOURCE_DIR}/ClassMock_Example/ClassesToMock.hh)

The above snippet shows that it is possible to compile your test with a binary. To do so you need to give the target of your binary into the TARGET_AS_SOURCE argument instead of FILES_AS_SOURCE. It is also required to give the parameter MAIN_FILE which provide the file that contains the main. This file is going to be ignored during the compilation of the test (as the main is generated or provided by your test files).
The advantage of this method is the fact that you don’t need to specify all the files needed for the test.
The drawback is that unnecessary files are going to be compiled.

#3.3 Compile with a library target

## Create an executable containing the code we want to test
add_library(static-Free_FunctionMock_Example_target
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/include/StaticFunction.hh
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/StaticFunction.cpp
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/include/FreeFunction.hh
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/FreeFunction.cpp
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/ClassToTest.cpp
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/include/ClassToTest.hh)
include_directories(static-Free_FunctionMock_Example_target ${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/include/)
##
## In This example is similar to the previous one but use a library target
## (no need to specify a main to remove from the compilation)
##
addFSeamTests(
DESTINATION_TARGET testFSeam_2 # FSeam testing binary target created for this testcase
TARGET_AS_SOURCE static-Free_FunctionMock_Example_target
TST_SRC
${CMAKE_CURRENT_SOURCE_DIR}/test.cpp
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/test/MockFreeFunctionTestCase.cpp
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/test/MockStaticFunctionTestCase.cpp
TO_MOCK
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/include/FreeFunction.hh
${CMAKE_CURRENT_SOURCE_DIR}/Static-Free_FunctionMock_Example/include/StaticFunction.hh)

The above snippet shows how to compile your test with a library.

To do so you need to give the target of your library into the TARGET_AS_SOURCE argument. It is the same as compiling with the binary target (previous snippet) but you don’t need to give a MAIN_FILE to not take into account in the compilation of the test.

As when compiling with a binary target, the advantage of this method is the fact that you don’t need to specify all the files needed for the test. The disadvantage is that unnecessary files are going to be compiled.

As you can see the impact on using the CMake functions provided by FSeam isn’t null but at least it is not too significant. Instead of compiling the tests files via add_exectutable, you just use the addFSeamTests function (which takes some additional arguments). I think it is a correct trade-off for the possibility to test legacy code easily. Please leave a comment if to express your opinion about this trade-off.

In big project, it is recommended to take the extra time to specify each file you need to compile for each test in order to not have really long compilation time for little tests. Using another target (binary or library) to compile your test is useful for little projects.

FSeam can do more

This tutorial cover pretty much 90% of the use cases you would need for your test. But it can do more (customComparator, custom logging, etc…) check it out directly in the documentation.

The project is open source, any feedback on how to improve it is welcome.

About Jonathan Boccara

FSeam: A Mocking Framework That Requires No Change In Code (Part 2)

How does FSeam work?

The classic way to do

#1 Installation

#2 FSeam Test Case

#2.1 Test a class: `isWinnerOfGameOfThrones`

#2.2 Test multiple instances of the same class type:

`isGoodGameOfThronesSeason`

#2.3 Test a free function or a static method

#3 Let’s compile it!

#3.1 Compile with a defined set of files

#3.2 Compile with a binary target

#3.3 Compile with a library target

FSeam can do more

You will also like

About Jonathan Boccara

FSeam: A Mocking Framework That Requires No Change In Code (Part 2)

How does FSeam work?

The classic way to do

#1 Installation

#2 FSeam Test Case

#2.1 Test a class: isWinnerOfGameOfThrones

#2.2 Test multiple instances of the same class type:

isGoodGameOfThronesSeason

#2.3 Test a free function or a static method

#3 Let’s compile it!

#3.1 Compile with a defined set of files

#3.2 Compile with a binary target

#3.3 Compile with a library target

FSeam can do more

You will also like

#2.1 Test a class: `isWinnerOfGameOfThrones`

`isGoodGameOfThronesSeason`