Conceptually you need to store at least the binaries (and the headers if you do C/C++)
That's usually the only way with third party libraries where you don't have the source.
If you have the source you can opt to store the source and build the third party libraries in your build process.