Avoiding new as much as possible, means many benefits, such as:
First and foremost, you also avoid delete statements.Though smart pointers can help you here. So this is not that strong point.
You avoid Rule of Three (in C++03), or Rule of Five (in C++11). If you use new when designing a class, that is, when your class manages raw memory internally, you probably have to consider this rule.
It's easy to implement exception-safe code when you don't use new. Otherwise, you've to face hell lot of problems, making your code exception-safe.
Using new unnecessarily means you're inviting problems. I've seen when an inexperienced programmer uses new, he has often better alternatives, such as usage standard containers, and algorithms. Use of standard containers avoids most problems which comes with explicit use of new.