I have been running overnight memory tests on an embedded Linux system. Using vmstat I have observed that the free memory steadily decreases over time. According to some
The kernel will reclaim cached memory pages when it needs them, i.e. when the system would otherwise run out of memory. whether memory pages from the processes's heap (free store) are ever returned to the OS is at the discretion of the process's memory manager, in this case the new/delete implementation in the C++ library. This is a completely voluntary operation with which the kernel has nothing to do.
From the fact that drop_caches did the trick, you can infer that it was the kernel cache, not the process's heap, that was filling up memory. Use the free command to find out how much memory is actually available for application use, esp. the -/+ buffers/cache line it reports.