(I don\'t want to hear about how crazy I am to want that! :)
Focus-follows-mouse is also known as point-to-focus, pointer focus, and (in some implementations) sloppy
Focus-follows-mouse is not a particularly suitable input method for OS X because its menu bar was designed to be at the top of the screen. When you move the mouse out of your application window to get to the menus, if it crosses any other application's windows on the way, the menu changes.
So yes, in reply to dreeves comment, it works perfectly fine for Terminal (or for any other single application on the desktop), because the only other windows it's going to affect are Terminal windows, so the menu never changes as you switch windows. And it works fine for X11 because X11 apps generally have their menu bars embedded in the window, so you don't have to leave the window to access them.
Of course you can work around the menu-changing issue by introducing an artificial delay before the focus changes and/or the menu switches, but it's never going to work as well as it does on other desktops.