We used to have quite a number of separate assemblies in our shared code. Over time, we found that we almost invariably referenced these in groups. This made more work for the developers, and we had to hunt to find what assembly a class or interface was in. We ended up combining some of these assemblies based on usage patterns. Life got easier.
There are a lot of considerations here - are you writing a library for developers, are you deploying the DLLs to offsite customers, are you using remoting (thanks, Maximilian Mayerl) or writing WCF services, etc. There is no one right answer - it depends.
In general I agree with Jeff Sternal - don't break up the assemblies unless it offers a proven benefit.
