I have a psychological tic which makes me reluctant to use large libraries (like GLib or Boost) in lower-level languages like C and C++. In my mind, I think:
The term I like for frameworks, library sets, and some types of development tools, is platform technologies. Platform technologies have costs beyond impact on code size and performance.
If your project is itself intended to be used as a library or framework, you may end up pushing your platform technology choices on developers that use your library.
If you distribute your project in source form, you may end up pushing platform technology choices on your end users.
If you do not statically link all your chosen frameworks and libraries, you may end up burdening your end users with library versioning issues.
Compile time effects developer productivity. Incremental linking, precompiled headers, proper header dependency management, etc., can help manage compile times, but do not eliminate the compiler performance problems associated with the massive amounts of inline code some platform technologies introduce.
For projects that are distributed as source, compile time affects the end users of the project.
Many platform technologies have their own development environment requirements. These requirements can accumulate making it difficult and time consuming for new developers on a project to be able to replicate the environment needed to allow compiling and debugging.
Using some platform technologies in effect creates a new programming language for the project. This makes it harder for new developers to contribute.
All projects have platform technology dependencies, but for many projects there are real benefits to keeping these dependencies to a minimum.