Hello
As stated in my previous email, I went ahead to do some load testing
comparing ngx.shared.DICT with lua-resty-memcache. When I am loading
ngx.shared.DICT everything is perfectly fine and the response times
and query/second are very impressive. When I do the same on memcache,
I see a lot of errors in my error.log like:
[error] 9324#0: *9086 lua socket read timed out
[error] 9324#0: *9086 [lua] common.lua:128: getURLsFromMemcache():
failed to retrieve key URLs-8081 from memcache: failed to receive 1st
line: timeout
[error] 9324#0: *9086 attempt to send data on a closed socket:
u:0000000041BF86A8, c:0000000000000000, ft:2 eof:0
[error] 9324#0: *9086 [lua] common.lua:146: setURLsToMemcache():
Unable to set variable URLs-8081,closed
[error] 9324#0: *9086 attempt to send data on a closed socket:
u:0000000041BF86A8, c:0000000000000000, ft:2 eof:0
[error] 9324#0: *9086 [lua] common.lua:240: getSettingsFromMemcache():
failed to retrieve key SETTINGS-8081_103 from memcache: failed to send
command: closed
[error] 9324#0: *9086 attempt to send data on a closed socket:
u:0000000041BF86A8, c:0000000000000000, ft:2 eof:0
[error] 9324#0: *9086 [lua] common.lua:259: setSettingsToMemcache():
Unable to set variable SETTINGS-8081_103,closed
[error] 9324#0: *9086 [lua] common.lua:454: closeMemcache(): failed to
set keepalive of Memcache: closed
I am enclosing some code pieces from my lua code (relevant parts I think):
-- MAIN.LUA
local common = require "common"
local rows = common.getURLs(a) -- Can be nil or valid list
...
-- A lot of logic around here (calles common.getSettingsFromMemcache()
which is very similar to getURLsFromMemcache())
...
common.closeMemcache()
--COMMON.LUA
local memcached = require "resty.memcached"
local memc -- To hold instance of Memcache
function getMemcache()
if memc == nil then
-- Connect Memcache Server
memc = memcached:new()
memc:set_timeout(500)
local ok, err = memc:connect("unix:/tmp/memcached.sock")
if not ok then
ngx.log(ngx.ERR, "failed to connect memcache: ", err)
return nil
end
end
return memc
end
function getURLs(a)
local res = getURLsFromMemcache(a) -- Res can be NIL or List
-- If res is NIL, i.e. no result is found in the memcache
if not res then
res = getURLsFromMySQL(a) -- Get the list of URLs from MySQL
-- Set it into Memcache for next time :)
if res ~= nil then
setURLsToMemcache(a, res)
else
ngx.log(ngx.ERR, "Result from MySQL (URLs) was found nil
and hence couldnt be saved in memcache")
end
end
return res -- NIL or valid List with URLs
end
-- Function to get list of URLs from Memcache for a given account ID
-- Input: account ID
-- Output: NIL or Lua list with URLs
function getURLsFromMemcache(a)
local memc = getMemcache()
if memc then
local res, flags, err = memc:get("URLs-" .. a)
if not res then
ngx.log(ngx.ERR, "failed to retrieve key URLs-"..a.." from
memcache: ", err)
return nil
end
return cjson.decode(res)
end
return nil
end
-- Function to set URLs to Memcache for an account
-- Input: account ID, list of URLs
-- Output: nothing
function setURLsToMemcache(a, res)
local memc = getMemcache()
if memc then
local ok, err = memc:set("URLs-" .. a, cjson.encode(res))
if not ok then
ngx.log(ngx.ERR, "Unable to set variable URLs-"..a..",", err)
end
return ok
end
return nil
end
--Set Memcache keepalives
function closeMemcache()
--local memc = getMemcache() No point creating a new memcache if
its not there already
if memc ~= nil then
local ok, err = memc:set_keepalive(0, 1000)
if not ok then
ngx.log(ngx.ERR, "failed to set keepalive of Memcache: ", err)
end
end
end
Questions:
1. Am I doing anything wrong here. I am maintaining an instance of
memcache in common.lua and accessing it via getMemcache() function. My
understanding is that the socket ends memc doesnt become nil (ofcourse
as it doesnt know). Should I not check for if memc == nil in my
getMemcache() function and always go ahead and initiate a connect
function (which will do the rest, considering if keepalive is active).
Should I do memcached:new() everytime getMemcache() is called or how
should I go about it?
2. Is it normal to use such code in Lua as I come from PHP background
and still learning Lua
Thanks
Sparsh Gupta