Iterating through a Lua table from C++?

Question

I'm trying to load tables from Lua to C++ but I'm having trouble getting it right. I'm getting through the first iteration just fine but then at the second call to lua_next it crashes. Any ideas?

Lua file:

level = { 1, 2, 3, }

C++ file - First I did this:

lua_getglobal( L, "level" );
for( lua_pushnil( L ); lua_next( L, 1 ); lua_pop( L, -2 ) )
{
    if( lua_isnumber( L, -1 ) ) {
        int i = (int)lua_tonumber( L, -1 );
        //use number
    }
}
lua_pop( L, 1 );

Then I tried from the reference manual:

lua_getglobal( L, "level" );
int t = 1;
lua_pushnil( L );
while( lua_next( L, t ) ) {
    printf( "%s - %s", 
        lua_typename( L, lua_type( L, -2 ) ),
        lua_typename( L, lua_type( L, -1 ) ) );
    lua_pop( L, 1 );
}
lua_pop( L, 1 );

And finally this:

lua_getglobal( L, "level" );
lua_pushnil( L );

lua_next( L, 1 );
if( lua_isnumber( L, -1 ) ) {
    int i = (int)lua_tonumber( L, -1 );
    //use number fine
}
lua_pop( L, 1 );

lua_next( L, 1 ); //crashes

etc...

Naturally L is a lua_State* and I'm initializing it and parsing the file okay.

Edit: In response to Jesse Beder answer I tried this code, with a logger, but I still can't get it to work.

Log::Get().Write( "engine", "stack size: %i", lua_gettop( L ) );

lua_getglobal(L, "level");
if( lua_istable( L, -1 ) )
    Log::Get().Write( "engine", "-1 is a table" );

lua_pushnil(L);
if( lua_isnil( L, -1 ) )
    Log::Get().Write( "engine", "-1 is now nil" );
if( lua_istable( L, -2 ) )
    Log::Get().Write( "engine", "-2 is now table" );

int pred = lua_next( L, -2 );
Log::Get().Write( "engine", "pred: %i", pred );
while( pred ) {
    Log::Get().Write( "engine", "loop stuff" );
    if( lua_isnumber( L, -1 ) ) {
        int i = (int)lua_tonumber( L, -1 );
        //use number
        Log::Get().Write( "engine", "num: %i", i );
    }
    Log::Get().Write( "engine", "stack size: %i", lua_gettop( L ) );
    if( lua_istable( L, -3 ) )
        Log::Get().Write( "engine", "-3 is now table" );

    lua_pop( L, 1 );
    Log::Get().Write( "engine", "stack size: %i", lua_gettop( L ) );
    if( lua_istable( L, -2 ) )
        Log::Get().Write( "engine", "-2 is now table" );

    pred = lua_next( L, -2 );
    Log::Get().Write( "engine", "pred: %i", pred );
}
lua_pop( L, 1 );

Which gave this output:

stack size: 0
-1 is a table
-1 is now nil
-2 is now table
pred: 1
loop stuff
num: 1
stack size: 3
-3 is now table
stack size: 2
-2 is now table

Everything you said, Jesse, seems to hold true. But it still fails to go to the next iteration.

Edit2: I tried to copy the exact code into a new project, skipping all the surrounding classes and stuff I didn't bother to include here and there it works. But here it doesn't, and it will just survive one call the lua_next.

Edit3: I've narrowed it down a bit further now. I'm using hge as my 2D engine. I put all the previous code in the function test:

test(); //works
if( hge->System_Initiate() )
{       
    test(); //fails
    hge->System_Start();
}

As far as I understand hge doesn't do anything with lua. Here's the source code for a small test I made. The source for hge 1.81 is here.

Edit4: The question size is getting out of control but it can't be helped. This is the smallest code I've been able to reduce it to.

extern "C"
{
    #include <lua/lua.h>
    #include <lua/lualib.h>
    #include <lua/lauxlib.h>
}
#include <hge\hge.h>

bool frame_func()
{   
    return true;
}

bool render_func()
{
    return false;
}

void test()
{
    lua_State *L = lua_open();
    luaL_openlibs( L );

    if( luaL_dofile( L, "levels.lua" ) ) {
        lua_pop( L, -1 );
        return;
    }
    lua_getglobal(L, "level");
    lua_pushnil(L);

    while( lua_next( L, -2 ) ) {
        if( lua_isnumber( L, -1 ) ) {
            int i = (int)lua_tonumber( L, -1 );
            //use number
        }
        lua_pop( L, 1 );
    }
    lua_pop( L, 1 );

    lua_close( L );
}

int main()
{
    HGE *hge = hgeCreate( HGE_VERSION );

    hge->System_SetState( HGE_FRAMEFUNC, frame_func );
    hge->System_SetState( HGE_RENDERFUNC, render_func );
    hge->System_SetState( HGE_WINDOWED, true );
    hge->System_SetState( HGE_SCREENWIDTH, 800 );
    hge->System_SetState( HGE_SCREENHEIGHT, 600 );
    hge->System_SetState( HGE_SCREENBPP, 32 );

    //test(); //works

    if( hge->System_Initiate() )
    {       
        test(); //fails
        hge->System_Start();
    }

    hge->Release();

    return 0;
}

Answer 1

When you call lua_next, the second argument should be the index of the table. Since you're just pushing the table onto the stack with

lua_getglobal(L, "level");

after that call your stack will look like

-1: table "level"

(not +1, since the stack is read going down). Then you call

lua_pushnil(L);

so your stack will be

-1: key (nil)
-2: table "level"

Your table is at -2, so when you call lua_next, you should use the index -2. Finally, after each iteration, your stack should look like:

-1: value
-2: key
-3: table "level"

So you want to read the value (at -1) and then pop it (so just pop once), and then call lua_next to get the next key. So something like this should work:

lua_getglobal(L, "level");
lua_pushnil(L);

while(lua_next(L, -2)) {  // <== here is your mistake
    if(lua_isnumber(L, -1)) {
        int i = (int)lua_tonumber(L, -1);
        //use number
    }
    lua_pop(L, 1);
}
lua_pop(L, 1);

Edit based on your second edit

Since it works when you remove external stuff, but doesn't when you add it back in, my best guess is that you're corrupting the stack somehow (either the C++ stack or the lua stack). Look really carefully at your pointers, especially when you manipulate the lua state.

Answer 2

Reading LUA manual lua_next you find that problems may arise on using lua_tostring on a key directly, that is you have to check the value of the key and then decide to use lua_tostring or lua_tonumber. So you could try this code:

   std::string key
   while(lua_next(L, -2) != 0){ // in your case index may not be -2, check

    // uses 'key' (at index -2) and 'value' (at index -1)

    if (lua_type(L, -2)==LUA_TSTRING){ // check if key is a string
         // you may use key.assign(lua_tostring(L,-2));
    }
    else if (lua_type(L, -2)==LUA_TNUMBER){ //or if it is a number
         // this is likely to be your case since you table level = { 1, 2, 3, }
         // don't declare field ID's
         // try:
         // sprintf(buf,"%g",lua_tonumber(L,-2));
         // key.assign(buf);
    }
    else{
        // do some other stuff
    }
    key.clear();
    lua_pop(L,1)
   }

Hope it helps.

Answer 3

Why are you doing the extra lua_pop(L, 1) at the end of the version from the reference manual? I can see how that could be a problem, since you're popping beyond the stack depth.