Yes, it could be garbage collection, but it could also be some synchronisation with the C++ code, or something completely different (hard to say without code).
Anyway, you should have a look at SIG for development of Python/C++ integration to find issues and how to speed up things.