Exposing a callback interface from C++ to Python and passing it back to C++

I develop a wrapper of our C++ library for Python users and I have a memory leak when I try to expose a callback interface.

My library code looks like this:

// Simulate our librarynamespace library {    // Callback pure interface    class Callback{    public:        virtual ~Callback() {            // Just for test!            std::cout << "~Callback() destructed!" << std::endl;        }         virtual void onProgress(int progress) = 0;    };    // Some library class who gets and stores callback instance    class Library {        std::shared_ptr<Callback> callback;    public:        Library(std::shared_ptr<Callback> _callback)        : callback { _callback } {            std::cout << "Library() constructed!" << std::endl;        }        ~Library() {            // Ligrary is free!            std::cout << "~Library() destructed!" << std::endl;        }        void func() {            // Use callback            if (callback != nullptr) {                callback->onProgress(66);            }        }    };    // Show module is unloaded.    struct Lifecycle {        ~Lifecycle() {            std::cout << "Lifecycle has terminated." << std::endl;        }    };    Lifecycle lc;}

Lookup to object LifeCycle. It shows message when our module is unloaded.

Next I have a C++ module with boost::python

#include <boost/python.hpp>namespace py = boost::python;namespace {    // Python wrapper for callback    struct CallbackWrapper        : library::Callback        , py::wrapper<library::Callback>        {            virtual void onProgress(int progress) override {                this->get_override("onProgress")(progress);            }        };    // Special testing callback    struct CallbackWrapperTest        : library::Callback        , py::wrapper<library::Callback>        {            virtual void onProgress(int progress) override {                std::cout << "TEST: onProgress(" << progress << ")" << std::endl;            }        };    // Creation finctuon    std::shared_ptr<library::Library> createLibrary(std::shared_ptr<library::Callback> callback) {        return std::make_shared<library::Library>(callback);    }}BOOST_PYTHON_MODULE(callback) {    py::class_<library::Library, boost::noncopyable>("Library",        py::no_init        )        .def("__init__", py::make_constructor(&createLibrary))        .def("func", &library::Library::func)        ;    py::class_<CallbackWrapper, boost::noncopyable>("Callback");    py::class_<CallbackWrapperTest, boost::noncopyable>("CallbackTest");}

Class CallbackWrapperTest will show the problem late.

My Python code is:

import syssys.path.append(r"./build")import callbackclass PyCallback(callback.Callback):'''Python callback implementation'''    def onProgress(self, progress):        print("PY onProgress", progress)cb = PyCallback()# Replacing by this works fine but useless# cb = callback.CallbackTest()lib = callback.Library(cb)lib.func()# explicit library deletion works fine  # del libprint("==========================")

Note the script doesn't have function "main". It is important.

I have an output:

Library() constructed!PY onProgress 66==========================Lifecycle has terminated.

There are not messages from destructors! I expect to see

~Library() destructed!~Callback() destructed!

I realised some strange thinks:

1. If I wrap script code to a function (e.g. main()) dectructors are called.
2. If I add explicit deletion del lib desctuctors are called.
3. If I remove Python inheritance and I replace Python class to pure C++CallbackTest desctuctors are called.

But only if I do Python derived class I have a memory leak. I suppose the problem in reference counting, but I don't understand where. Can anybody help me?

The test project is placed on GitHub