How to determine the execution time of a Lua script in Redis?

Question

I have a small Lua script to run in Redis, and I'm interested in getting the execution time.

Due to the nature of Redis and it's Lua implementation, I cannot use the TIME function at the start/return points of the script, and include this information in the return for processing (See http://redis.io/commands/eval - Scripts as pure functions). This results in an error: (error) ERR Error running script (call to f_a49ed2fea72f1f529843d6024d1515e76e69bcbd): Write commands not allowed after non deterministic commands

I have searched around for a function/call I could make which will return the execution time of the last run script, but have not found anything yet.

I am using PHP and the Predis Library. While I can check the execution time from PHP side, I wish to remove the transmission overhead and find out how long the Lua script will block access to the database. I have successfully had times returned if I do not need to alter any of the data stored in Redis, and these have been about 1/10th the time PHP reports.

How can I determine the execution time of the Lua script in Redis, and not via PHP?

Answer 1

You can activate Redis slow log feature, by changing the slowlog-log-slower-than parameter to 0. It will record execution time of ALL commands (including Lua scripts, and whatever the execution time).

The slow log is kept in an in-memory queue you have to dump on a regular basis to collect data. Depending on the volume of traffic, you may have to increase slowlog-max-len to be sure to catch the execution times you are interested in.

You can use the slowlog get command to dump the slow log. Up to you to filter out the results you don't need. AFAIK, there is no possibility to filter at data collection time (to keep only Lua statistics).

Answer 2

You can use os.clock() inside script if you enable os module when building Redis.

Change #if 0 to #if 1 on line 484 in https://github.com/antirez/redis/blob/unstable/src/scripting.c:

void luaLoadLibraries(lua_State *lua) {
    luaLoadLib(lua, "", luaopen_base);
    luaLoadLib(lua, LUA_TABLIBNAME, luaopen_table);
    luaLoadLib(lua, LUA_STRLIBNAME, luaopen_string);
    luaLoadLib(lua, LUA_MATHLIBNAME, luaopen_math);
    luaLoadLib(lua, LUA_DBLIBNAME, luaopen_debug); 
    luaLoadLib(lua, "cjson", luaopen_cjson);
    luaLoadLib(lua, "struct", luaopen_struct);
    luaLoadLib(lua, "cmsgpack", luaopen_cmsgpack);

#if 0 /* Stuff that we don't load currently, for sandboxing concerns. */
    luaLoadLib(lua, LUA_LOADLIBNAME, luaopen_package);
    luaLoadLib(lua, LUA_OSLIBNAME, luaopen_os);
#endif
}

Answer 3

EDIT: rld is obsolete.

You can use rld (https://github.com/RedisLabs/redis-lua-debugger) and have it print to the log - the log has timing information (although rld will cause your script to run slower, naturally).

Answer 4

As of Redis v3.2, you can choose to replicate commands instead of scripts with a call to the new redis.replicate_commands() API. This will allow you, among other things, to call TIME in scripts that perform write operations without triggering the non-determinism error.