Catching stack overflow

Question

What\'s the best way to catch stack overflow in C?

More specifically:

A C program contains an interpreter for a scripting language.

Scripts are not t

Answer 1

AFAIK, all mechanisms for detecting stack overflow will incur some runtime cost. You could let the CPU detect seg-faults, but that's already too late; you've probably already scribbled all over something important.

You say that you want your interpreter to call precompiled library code as much as possible. That's fine, but to maintain the notion of a sandbox, your interpreter engine should always be responsible for e.g. stack transitions and memory allocation (from the interpreted language's point of view); your library routines should probably be implemented as callbacks. The reason being that you need to be handling this sort of thing at a single point, for reasons that you've already pointed out (latent bugs).

Things like Java deal with this by generating machine code, so it's simply a case of generating code to check this at every stack transition.