How to Return a Smart Pointer AND Use Covariance
Today we’re going to take a big step back on the specific problem of the clone interface we’ve dealt with on the last post. For this Raoul Borges is taking over on this topic to show you a solution to the general problem of smart pointers and covariance in C++.
Raoul is a C++ developer since 1999, a physics enthusiast and rpg storyteller/player. You can find him online on his twitter account and on his blog.
The original problem Jonathan proposed a solution for was how to clone a concrete class when inheriting from multiple interfaces, all declaring the clone method, and all returning a smart pointer (in order to manage life cycle and produce exception safe code).
That solution is simple and targeted to that situation. But here I want to expand on this and tackle the more general problem: in C++, it seems that we can have covariant return, or smart pointer return, but not both. Or can we?
Note: In this discussion, we will avoid type erasure as it generates a lot of boiler plate, which goes against our objective here. We will assume instead a fully generic OO solution. Also, this is not a discovery: Partial implementations of the techniques shown below can easily be found on the Internet. We are standing on the shoulders of the giants out there, and just compiling it all together in one post.
The problem: Covariant return type vs. smart pointers
C++ has support for covariant return type. That is, you can have the following code:
struct Base {};
struct Derived : Base {};
struct Parent
{
virtual Base * foo();
} ;
struct Child : Parent
{
virtual Derived * foo() override ;
} ;
Here, we expect the foo method from Child to return Base * for a successful overriding (and compilation!). With the covariant return type, we can actually replace Base * by any of its derived types. For example, Derived *.
This works for pointers, and for references… But the moment you try to use smart pointers:
#include <memory>
struct Base {};
struct Derived : Base {};
struct Parent
{
virtual std::unique_ptr<Base> foo();
} ;
struct Child : Parent
{
virtual std::unique_ptr<Derived> foo() override ;
} ;
… the compiler generates an error.
Use cases
Since the problem is general, let’s take a wide panel of use cases with increasing complexity:
- Simple hierarchy:
- Multiple inheritance:
- Deep hierarchy:
- Diamond inheritance:
By handling all those cases in a natural way, the solution should be usable for most production problems.
Preamble: Separation of concerns + private virtual function
Instead of having one clone member function handling everything, we will separate it into two member functions. In the following piece of code:
class some_class
{
public:
std::unique_ptr<some_class> clone() const
{
return std::unique_ptr<some_class>(this->clone_impl());
}
private:
virtual some_class * clone_impl() const
{
return new some_class(*this) ;
}
};
The first function, clone_impl(), does the actual work of cloning using the copy-constructor. It offers a strong guarantee (as long as the copy-constructor offers it), and transfers the ownership of the pointer to the newly created object. While this is usually unsafe, we assume that in this case no one can call this function except the clone() function, which is enforced by the private access of clone_impl().
The second function, clone(), retrieves the pointer, and gives its ownership to a unique_ptr. This function cannot fail by itself, so it offers the same strong guarantee as clone_impl().
Simple Hierarchy: Covariance + Name hiding
Using the technique above, we can now produce a simple OO hierarchy:
class cloneable
{
public:
virtual ~cloneable() {}
std::unique_ptr<cloneable> clone() const
{
return std::unique_ptr<cloneable>(this->clone_impl());
}
private:
virtual cloneable * clone_impl() const = 0;
};
///////////////////////////////////////////////////////////////////////////////
class concrete : public cloneable
{
public:
std::unique_ptr<concrete> clone() const
{
return std::unique_ptr<concrete>(this->clone_impl());
}
private:
virtual concrete * clone_impl() const override
{
return new concrete(*this);
}
};
int main()
{
std::unique_ptr<concrete> c = std::make_unique<concrete>();
std::unique_ptr<concrete> cc = c->clone();
cloneable * p = c.get();
std::unique_ptr<clonable> pp = p->clone();
}
Do you see what we did, here?
By separating the concerns, we were able to use covariance at each level of the hierarchy to produce a clone_impl member function returning the exact type of pointer we wanted.
And using a little (usually) annoying feature in C++, name hiding (i.e. when declaring a name in a derived class, this name hides all the symbols with the same name in the base class), we hide (not override) the clone() member function to return a smart pointer of the exact type we wanted.
When cloning from a concrete, we obtain a unique_ptr<concrete>, and when cloning from a cloneable, we obtain a unique_ptr<cloneable>.
One could get uneasy at the idea of having a clone_impl member function using a RAII-unsafe transfer of ownership, but the problem is mitigated as the member function is private, and is called only by clone. This limits the risk as the user of the class can’t call it by mistake.
This solves the problem but adds some amount of boilerplate code.
Simple Hierarchy, v2: Enter the CRTP
The CRTP is a C++ idiom that enables the injection of the derived class name into its templated base. You can learn all about it in the series on CRTP on Fluent C++.
We will use it to declare methods with the correct derived prototypes in the CRTP base class, methods that will then be injected through inheritance into the derived class itself:
template <typename Derived, typename Base>
class clone_inherit<Derived, Base> : public Base
{
public:
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
private:
virtual clone_inherit * clone_impl() const override
{
return new Derived(*this);
}
};
clone_inherit is a CRTP that knows its derived class, but also all its direct base class. It implements the covariant clone_impl() and hiding clone() member functions as usual, but they use casts to move through the hierarchy of types.
This enables us to change the concrete class defined above into:
class concrete
: public clone_inherit<concrete, cloneable>
{
};
int main()
{
std::unique_ptr<concrete> c = std::make_unique<concrete>();
std::unique_ptr<concrete> cc = b->clone();
cloneable * p = c.get();
std::unique_ptr<clonable> pp = p->clone();
}
As you can see, the concrete class is now free of clutter.
This effectively adds a polymorphic and covariant clone() to a hierarchy of class.
This CRTP is the foundation of our general solution: Every next step will build upon it.
Multiple Inheritance: Variadic templates to the rescue
One complication of OO hierarchies is multiple inheritance.
In our case, how can we extend our solution to support the case where the concrete class inherits from two bases classes that both provide the same clone feature?
The solution first needs the two base classes, foo and bar, to offer the clone/clone_impl member functions:
class foo
{
public:
virtual ~foo() = default;
std::unique_ptr<foo> clone() const
{
return std::unique_ptr<foo>(this->clone_impl());
}
private:
virtual foo * clone_impl() const = 0;
};
///////////////////////////////////////////////////////////////////////////////
class bar
{
public:
virtual ~bar() = default;
std::unique_ptr<bar> clone() const
{
return std::unique_ptr<bar>(this->clone_impl());
}
private:
virtual bar * clone_impl() const = 0;
};
There’s a bit of boilerplate, here, but we’ll address it later. For now, we must solve the inheritance issue, and C++11 provides us with an easy solution: Variadic templates.
We only need to modify the clone_inherit CRTP to support it:
template <typename Derived, typename ... Bases>
class clone_inherit : public Bases...
{
public:
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
private:
virtual clone_inherit * clone_impl() const override
{
return new Derived(static_cast<const Derived & >(*this));
}
};
We can now write our concrete class using it:
class concrete
: public clone_inherit<concrete, foo, bar>
{
};
Last, but not least, we can use our classes with both covariance and smart pointers:
int main()
{
std::unique_ptr<concrete> c = std::make_unique<concrete>();
std::unique_ptr<concrete> cc = c->clone();
foo * f = c.get();
std::unique_ptr<foo> ff = f->clone();
bar * b = c.get();
std::unique_ptr<bar> bb = b->clone();
}
Multiple Inheritance v2: Specialization to the rescue
Now, let’s address the clutter: Both foo and bar offer the same “cloneable” feature. And in our case, both should be virtually destructible.
The solution is to specialize clone_inherit to handle the case when no base class is desired, provide the virtual destructors, and have foo and bar inherit from it:
template <typename Derived, typename ... Bases>
class clone_inherit : public Bases...
{
public:
virtual ~clone_inherit() = default;
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
private:
virtual clone_inherit * clone_impl() const override
{
return new Derived(static_cast<const Derived & >(*this));
}
};
///////////////////////////////////////////////////////////////////////////////
template <typename Derived>
class clone_inherit<Derived>
{
public:
virtual ~clone_inherit() = default;
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
private:
virtual clone_inherit * clone_impl() const = 0;
};
This way, we can now write:
class foo
: public clone_inherit<foo>
{
};
///////////////////////////////////////////////////////////////////////////////
class bar
: public clone_inherit<bar>
{
};
///////////////////////////////////////////////////////////////////////////////
class concrete
: public clone_inherit<concrete, foo, bar>
{
};
Last, but not least, we can use our classes with both covariance and smart pointers:
int main()
{
std::unique_ptr<concrete> c = std::make_unique<concrete>();
std::unique_ptr<concrete> cc = c->clone();
foo * f = c.get();
std::unique_ptr<foo> ff = f->clone();
bar * b = c.get();
std::unique_ptr<bar> bb = b->clone();
}
Deep Hierarchy: Abstracting
Another complication of OO Hierarchies is that they can go deeper than two levels:
The thing is, as Scott Meyers advised us, non-leaf classes are not supposed to be instantiable by themselves (More Effective C++, item 33).
In our case, the clone_impl method in the non-leaf class must then be pure virtual.
Our solution must thus support the choice of declaring clone_impl pure virtual, or implemented.
First, we add a dedicated type who will be used to “mark” a type:
template <typename T>
class abstract_method
{
};
Then, we partially specialize the clone_inherit class again to use that type, which means (because of the previous specialization), 4 different clone_inherit implementations:
// general: inheritance + clone_impl implemented
template <typename Derived, typename ... Bases>
class clone_inherit : public Bases...
{
public:
virtual ~clone_inherit() = default;
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
private:
virtual clone_inherit * clone_impl() const override
{
return new Derived(static_cast<const Derived & >(*this));
}
};
///////////////////////////////////////////////////////////////////////////////
// specialization: inheritance + clone_impl NOT implemented
template <typename Derived, typename ... Bases>
class clone_inherit<abstract_method<Derived>, Bases...> : public Bases...
{
public:
virtual ~clone_inherit() = default;
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
private:
virtual clone_inherit * clone_impl() const = 0;
};
///////////////////////////////////////////////////////////////////////////////
// specialization: NO inheritance + clone_impl implemented
template <typename Derived>
class clone_inherit<Derived>
{
public:
virtual ~clone_inherit() = default;
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
private:
virtual clone_inherit * clone_impl() const override
{
return new Derived(static_cast<const Derived & >(*this));
}
};
///////////////////////////////////////////////////////////////////////////////
// specialization: NO inheritance + clone_impl NOT implemented
template <typename Derived>
class clone_inherit<abstract_method<Derived>>
{
public:
virtual ~clone_inherit() = default;
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
private:
virtual clone_inherit * clone_impl() const = 0;
};
It starts to be is a lot of code, but this will enable the user to actually use the feature with no boilerplate at all, as demonstrated by the following code:
class cloneable
: public clone_inherit<abstract_method<cloneable>>
{
};
///////////////////////////////////////////////////////////////////////////////
class abstracted
: public clone_inherit<abstract_method<abstracted>, cloneable>
{
};
///////////////////////////////////////////////////////////////////////////////
class concrete
: public clone_inherit<concrete, abstracted>
{
};
int main()
{
std::unique_ptr<concrete> c = std::make_unique<concrete>();
std::unique_ptr<concrete> cc = c->clone();
abstracted * a = c.get();
std::unique_ptr<abstracted> aa = a->clone();
cloneable * p = c.get();
std::unique_ptr<clonable> pp = p->clone();
}
Again, we succeeded in not cluttering too much the user code, and make this pattern scalable.
Diamond Inheritance: Virtual-ing
Yet another complication of OO Hierarchies is that we can have a diamond inheritance:
In C++, this means we have a choice to do: Is the base class inherited virtually, or not?
This choice must thus be provided by clone_inherit. The thing is, declaring a virtual inheritance is much more tricky because of the template parameter pack… Or is it?
Let’s write a class that will do the indirection:
template <typename T>
class virtual_inherit_from : virtual public T
{
using T::T;
};
This class actually applies the virtual inheritance to its base class T, which is exactly what we wanted. Now, all we need is to use this class to explicit our virtual inheritance need:
class foo
: public clone_inherit<abstract_method<foo>, virtual_inherit_from<cloneable>>
{
};
class bar
: public clone_inherit<abstract_method<bar>, virtual_inherit_from<cloneable>>
{
};
///////////////////////////////////////////////////////////////////////////////
class concrete
: public clone_inherit<concrete, foo, bar>
{
};
int main()
{
std::unique_ptr<concrete> c = std::make_unique<concrete>();
std::unique_ptr<concrete> cc = c->clone();
foo * f = c.get();
std::unique_ptr<foo> ff = c->clone();
bar * b = c.get();
std::unique_ptr<bar> bb = c->clone();
cloneable * p = c.get();
std::unique_ptr<cloneable> pp = p->clone();
}
Again, we succeeded in not cluttering too much the user code, and make this pattern scalable.
… Et voilà!
The whole package
The whole clone-ing code is:
///////////////////////////////////////////////////////////////////////////////
template <typename T>
class abstract_method
{
};
///////////////////////////////////////////////////////////////////////////////
template <typename T>
class virtual_inherit_from : virtual public T
{
using T::T;
};
///////////////////////////////////////////////////////////////////////////////
template <typename Derived, typename ... Bases>
class clone_inherit : public Bases...
{
public:
virtual ~clone_inherit() = default;
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
protected:
// desirable, but impossible in C++17
// see: http://cplusplus.github.io/EWG/ewg-active.html#102
// using typename... Bases::Bases;
private:
virtual clone_inherit * clone_impl() const override
{
return new Derived(static_cast<const Derived & >(*this));
}
};
///////////////////////////////////////////////////////////////////////////////
template <typename Derived, typename ... Bases>
class clone_inherit<abstract_method<Derived>, Bases...> : public Bases...
{
public:
virtual ~clone_inherit() = default;
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
protected:
// desirable, but impossible in C++17
// see: http://cplusplus.github.io/EWG/ewg-active.html#102
// using typename... Bases::Bases;
private:
virtual clone_inherit * clone_impl() const = 0;
};
///////////////////////////////////////////////////////////////////////////////
template <typename Derived>
class clone_inherit<Derived>
{
public:
virtual ~clone_inherit() = default;
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
private:
virtual clone_inherit * clone_impl() const override
{
return new Derived(static_cast<const Derived & >(*this));
}
};
///////////////////////////////////////////////////////////////////////////////
template <typename Derived>
class clone_inherit<abstract_method<Derived>>
{
public:
virtual ~clone_inherit() = default;
std::unique_ptr<Derived> clone() const
{
return std::unique_ptr<Derived>(static_cast<Derived *>(this->clone_impl()));
}
private:
virtual clone_inherit * clone_impl() const = 0;
};
///////////////////////////////////////////////////////////////////////////////
… and the user code is:
///////////////////////////////////////////////////////////////////////////////
class cloneable
: public clone_inherit<abstract_method<cloneable>>
{
};
///////////////////////////////////////////////////////////////////////////////
class foo
: public clone_inherit<abstract_method<foo>, virtual_inherit_from<cloneable>>
{
};
///////////////////////////////////////////////////////////////////////////////
class bar
: public clone_inherit<abstract_method<bar>, virtual_inherit_from<cloneable>>
{
};
///////////////////////////////////////////////////////////////////////////////
class concrete
: public clone_inherit<concrete, foo, bar>
{
};
///////////////////////////////////////////////////////////////////////////////
… which is not bad, all in all.
Would we use it in production code? While this set of techniques is interesting, it doesn’t compile on Visual Studio 2017 (virtual inheritance, diamond and covariance don’t mix well in Visual Studio), which is in our case, a showstopper.
But it compiles at least with GCC 5.4.0+, as well as Clang 3.8.0+.
This set of techniques shows how, by using a clever but all-in-all simple combo of two orthogonal C++ paradigms, object oriented and generic (templates), we can factor out code to produce results with a concision that would have been difficult or impossible to get in other C-like languages.
It also shows a list of techniques (simulated covariance, inheritance indirection providing features) that can be applied elsewhere, each relying on C++ features assembled like lego pieces to produce the desired result.
Which is pretty cool IMHO.
🙂
Don't want to miss out ? Follow:
Share this post!
