Can browsers enforce any sort of limit on the amount of data that can be stored in JavaScript objects? If so, is there any way to detect that limit?
It appears that
Since a web app can't have access to any system-related information (like the available amount of memory), and since you would prefer not having to ask users to manually set their performance settings, you must rely on a solution that allows you to get such information about the user's system (available memory) without asking them. Seems impossible ? Well, almost...
But I suggest you do the following : make a Java applet that will automatically get the available memory size (e.g. using Runtime.exec(...) with an appropriate command), provided your applet is signed, and return that information to the server or directly to the web page (with JSObject, see http://docs.oracle.com/javafx/2/api/netscape/javascript/JSObject.html).
However, that would assume your users can all run a Java applet within their browsers, which is not always the case. Therefore, you could ask them to install a small piece of software on their machines that will measure how much memory your app should use without crashing the browser, and will send that information to your server. Of course, you would have to re-write that little program for every OS and architecture (Windows, Mac, Linux, iPhone, Android...), but it's simpler that having to re-write the whole application in order to gain some performance. It's a sort of in-between solution.
I don't think there is an easy solution. There will be some drawbacks, whatever you choose to do. Remember that web applications don't have the reputation of being fast, so if performance is critical, you should consider writing a traditional desktop application.