This question stems from watching Rasmus Lerdorf\'s talk from Drupalcon. This question and his talk have nothing specifically to do with Drupal, by the way... it was just gi
Like anything in software development, it depends. Rasmus's objection to the front-controller style frameworks is the performance hit you take from having to load so much code up-front on each request. This is 100% true. Even if you're using a smart-resource loading module/object/etc of some kind, using a framework is a performance trade-off. You take the performance hit, but in return you get back
Encouraged seperation of "business logic" (whatever that is) and Template/Layout Logic
Instant and (more importantly) unified access to the objects you'll use for database queries, service called, your data model, etc.
To a guy like Rasmus, this isn't worth the performance hit. He's a C/C++ programmer. For him, if you want to separate out business logic in a highly performant way, you write a C/C++ Extension to PHP.
So if you have an environment and team where you can easily write C/C++ extensions to PHP and your time-to-market vs. performance ratio is acceptable, then yes, throw away your front-controller framework.
If that's not your environment, consider the productivity increases a front-controller framework can bring to your (likely) simple CRUD Application.