To answer the original question: no, the definitions of [member] function templates don't have to go into the header. However, the compiler needs to see the definition to instantiate the template. For templates instantiated with many different types you want the templates to be implicitly instantiated when they are used. This is e.g. the case for class templates like std::vector<...> and for function templates like std::copy(...). In this case it is almost certainly impractical to separate the template definition from its declaration although I personally put the definitions into a separate file included at the bottom of the header file.
For templates instantiated with only a few types like the stream classes or std::basic_string<...> it is often better to define the function templates in a separate header-like file which is only included in implementation files which explicitly instantiates them. This way the effort of instantiating the template is only spent once rather than in every translation unit using it. Especially for the stream classes this makes a huge difference (primarily for compile and link time but on some systems also for executable size). ... and I'm pretty sure hardly anybody has gone to the trouble of using the stream classes with different character types than char and wchar_t (hint: it is non-trivial to arrange for the various facets to be implemented and present in the std::locale). The technique of explicitly instantiating templates also works well if there is only a limited set of types with which the template is intended to work.