How to OpenGL on Linux Platform Without X-Window System, can I send OpenGL Graphics Directly to Framebuffer Device?
There Is Project Named DirectFB (Direct FrameBuff
Apparently is possible to have SDL running without X.
Basically, your kernel should have framebuffer support and you could use SDL on top of DirectFB.
These threads are ok:
http://lists.libsdl.org/pipermail/sdl-libsdl.org/2006-October/058305.html
http://forums.libsdl.org/viewtopic.php?t=4079