access_by_lua在X-Accel-Redirect过程中无法require

xm1994 · 2015-07-10T19:02:51+00:00

Hello! 2015-07-10 17:28 GMT+08:00 Jun Wang: > nginx域名解析，在nginx配置文件使用r...

access_by_lua在X-Accel-Redirect过程中无法require

xm1994

我们的服务会通过设置X-Accel-Redirect在X-Accel-Redirect过程中重选upstream。原来的逻辑是通过X-Accel-Redirect结合X-Sharding-Group去选择upstream。但是在X-Accel-Redirect的过程中lua无法require一个公用的文件，导致出错。

错误日志：

2015/07/10 09:06:14 [alert] 34#0: *8 [lua] sharding.lua:40: Workspace nil final upstream: ide-backend, client: 10.10.58.89, server: XXX, request: "POST /ws/create HTTP/1.1", host: "coding.net", referrer: "https://XXX/ide-new/ws/izkvic"
2015/07/10 09:06:14 [error] 34#0: *8 lua entry thread aborted: runtime error: /var/lib/ide-openresty/lua/sharding.lua:5: attempt to index global 'config' (a nil value)
stack traceback:
coroutine 0:
        /var/lib/ide-openresty/lua/sharding.lua: in function </var/lib/ide-openresty/lua/sharding.lua:1>, client: 10.10.58.89, server: XXX, request: "POST /ws/create HTTP/1.1", upstream: "http://10.10.62.125:8085/ws/create", host: "coding.net", referrer: "https://XXX/ide-new/ws/izkvic"

nginx配置文件:

upstream ide-backend {
    server 192.168.0.1:8080;
   server 192.168.0.2:8080;
}

upstream ide-backend-a {
    server 192.168.0.1:8080;
}

upstream ide-backend-a {
   server 192.168.0.2:8080;
}

lua_package_path "/var/lib/ide-openresty/lua/?.lua;;";

server {
     listen 80;
     index index.html index.htm;
     server_name ide-backend.XXX.me;
     
     location / {
         
        set $backend_upstream "";
         access_by_lua_file/var/lib/ide-openresty/lua/sharding.lua;
         
        proxy_set_header Host $host;
         proxy_set_header X-Forwarded-For $remote_addr;
         proxy_set_header X-Forwarded-Host $host;
         proxy_set_header X-Forwarded-Port $server_port;
         proxy_set_header X-Forwarded-Proto $scheme;

        proxy_pass http://$backend_upstream;
         proxy_http_version 1.1;
         proxy_set_header Upgrade $http_upgrade;
         proxy_set_header Connection $connection_upgrade;
     }
}

lua程序(均在同一文件夹中)：

sharding.lua

require "common"
local spaceKey = ngx.req.get_headers()['X-Space-Key']
local shardingHeader = ngx.req.get_headers()['X-Sharding-Group']

targetGroup = config.defaultGroup
redisCli = init_redis()
local response = redisCli:ping()
mysqlConn = init_mysql()
if shardingHeader then
	targetGroup = shardingHeader
	ngx.log(ngx.ALERT, targetGroup, " From Header")
elseif spaceKey ~= nil then
	local redisKey = "sharding:"..spaceKey
	local shardingGroup, err = redisCli:get(redisKey)
	if shardingGroup ~= nil and shardingGroup ~= ngx.null then 
		targetGroup = shardingGroup
		ngx.log(ngx.ALERT, shardingGroup, " From Redis")
	else
		local sql = "Select T_HOST.F_GROUP shardingGroup "..
					"From T_WORKSPACE "..
					"Left Join T_HOST on T_HOST.F_ID = T_WORKSPACE.F_HOST_ID "..
					"Where T_WORKSPACE.F_SPACE_KEY = "..ngx.quote_sql_str(spaceKey).." LIMIT 1";
		res, err, errno, sqlstate = mysqlConn:query(sql)
		if not res then 
			ngx.log(ngx.ERR, err) 
			ngx.exit(500)
		else
			shardingGroup = res[1].shardingGroup;
			if shardingGroup ~= nil then
				ngx.log(ngx.ALERT, shardingGroup, " From Mysql")
				redisCli:set(redisKey, shardingGroup)
				redisCli:expire(redisKey, 3600*4)
				targetGroup =  shardingGroup
			end
		end
	end
end

ngx.var.backend_upstream = targetGroup
ngx.log(ngx.ALERT, "Workspace ", spaceKey, " final upstream: ",targetGroup)

close_cosock(mysqlConn)
close_cosock(redisCli)

common.lua

cjson = require 'cjson'
config = require 'config'

function close_cosock(cosock)  
	if not cosock then  
		return  
	end    
	local ok, err = cosock:set_keepalive(config.pool.idle_time, config.pool.size)  
	if not ok then  
		ngx.say("set keepalive error : ", err)  
	end  
end  

function init_redis()
	local redis = require 'resty.redis'  
	redisCli, err = redis:new()
	if not redisCli then
		ngx.log(ngx.ERR, "failed to instantiate redis: ", err)
		return nil
	end
	local ok, err = redisCli:connect(config.redis.host, config.redis.port)
	if not ok then
		ngx.log(ngx.ERR, "Failed connect to redis database: ", err)
		close_cosock(redisCli)
		return nil
	end

	redisCli:ping()

	pcall(redisCli.auth ,  config.redis.password)
	return redisCli
end

function init_mysql()
	local mysql = require 'resty.mysql'
	mysqlCli, err = mysql:new()
	if not mysqlCli then
		ngx.log(ngx.ERR, "failed to instantiate mysql: ", err)
		return nil
	end
	local ok, err, errno, sqlstate = mysqlCli:connect(config.mysql)

	if not ok then
	    ngx.log(ngx.ERR, "failed to connect: ", err, ": ", errno, " ", sqlstate)
	    close_cosock(mysqlCli)
	    return nil
	end
	return mysqlCli
end

config.lua

function env(envName, defaultValue)
	local envValue = os.getenv(envName)
	if not envValue then
		envValue = defaultValue
	end
	return envValue
end

local _config = {
	["mysql"] = {
		["host"] = env("MYSQL_SERVER_NAME", "192.168.59.3"),
		["port"] = tonumber(env("MYSQL_SERVER_PORT", 3306)),
		["user"] = env("MYSQL_USER", "coding_ide"),
		["password"] = env("MYSQL_PASSWORD", "coding_ide"),
		["database"] = env("MYSQL_DATABASE", "coding_ide"),
		["max_packet_size"] = 1024 * 1024 
	},
	["redis"] = {
		["host"] = env("REDIS_HOST", "192.168.59.3"),
		["port"] = tonumber(env("REDIS_PORT", 6379)),
		["password"] = env("REDIS_PASSWORD", "")
	},
	["pool"] = {
		["idle_time"] = 10000, --最大等待时间ms
		["size"] = 100 --连接池大小  
	},
	["defaultGroup"] = env("DEFAULT_HOST", "ide-backend")
}

config = _config
return config

xm1994

我已经找到问题了。。打开code_cache后无法使用require。现已改为init_by_lua

On Friday, July 10, 2015 at 7:02:51 PM UTC+8, Yibai Zhang wrote:

我们的服务会通过设置X-Accel-Redirect在X-Accel-Redirect过程中重选upstream。原来的逻辑是通过X-Accel-Redirect结合X-Sharding-Group去选择upstream。但是在X-Accel-Redirect的过程中lua无法require一个公用的文件，导致出错。

错误日志：
2015/07/10 09:06:14 [alert] 34#0: *8 [lua] sharding.lua:40: Workspace nil final upstream: ide-backend, client: 10.10.58.89, server: XXX, request: "POST /ws/create HTTP/1.1", host: "coding.net", referrer: "https://XXX/ide-new/ws/izkvic" 2015/07/10 09:06:14 [error] 34#0: *8 lua entry thread aborted: runtime error: /var/lib/ide-openresty/lua/sharding.lua:5: attempt to index global 'config' (a nil value) stack traceback: coroutine 0: /var/lib/ide-openresty/lua/sharding.lua: in function </var/lib/ide-openresty/lua/sharding.lua:1>, client: 10.10.58.89, server: XXX, request: "POST /ws/create HTTP/1.1", upstream: "http://10.10.62.125:8085/ws/create", host: "coding.net", referrer: "https://XXX/ide-new/ws/izkvic"

nginx配置文件:
upstream ide-backend { server 192.168.0.1:8080; server 192.168.0.2:8080; } upstream ide-backend-a { server 192.168.0.1:8080; } upstream ide-backend-a { server 192.168.0.2:8080; } lua_package_path "/var/lib/ide-openresty/lua/?.lua;;"; server { listen 80; index index.html index.htm; server_name ide-backend.XXX.me; location / { set $backend_upstream ""; access_by_lua_file/var/lib/ide-openresty/lua/sharding.lua; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://$backend_upstream; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; } }

lua程序(均在同一文件夹中)：
sharding.lua
require "common" local spaceKey = ngx.req.get_headers()['X-Space-Key'] local shardingHeader = ngx.req.get_headers()['X-Sharding-Group'] targetGroup = config.defaultGroup redisCli = init_redis() local response = redisCli:ping() mysqlConn = init_mysql() if shardingHeader then targetGroup = shardingHeader ngx.log(ngx.ALERT, targetGroup, " From Header") elseif spaceKey ~= nil then local redisKey = "sharding:"..spaceKey local shardingGroup, err = redisCli:get(redisKey) if shardingGroup ~= nil and shardingGroup ~= ngx.null then targetGroup = shardingGroup ngx.log(ngx.ALERT, shardingGroup, " From Redis") else local sql = "Select T_HOST.F_GROUP shardingGroup ".. "From T_WORKSPACE ".. "Left Join T_HOST on T_HOST.F_ID = T_WORKSPACE.F_HOST_ID ".. "Where T_WORKSPACE.F_SPACE_KEY = "..ngx.quote_sql_str(spaceKey).." LIMIT 1"; res, err, errno, sqlstate = mysqlConn:query(sql) if not res then ngx.log(ngx.ERR, err) ngx.exit(500) else shardingGroup = res[1].shardingGroup; if shardingGroup ~= nil then ngx.log(ngx.ALERT, shardingGroup, " From Mysql") redisCli:set(redisKey, shardingGroup) redisCli:expire(redisKey, 3600*4) targetGroup = shardingGroup end end end end ngx.var.backend_upstream = targetGroup ngx.log(ngx.ALERT, "Workspace ", spaceKey, " final upstream: ",targetGroup) close_cosock(mysqlConn) close_cosock(redisCli)

common.lua
cjson = require 'cjson' config = require 'config' function close_cosock(cosock) if not cosock then return end local ok, err = cosock:set_keepalive(config.pool.idle_time, config.pool.size) if not ok then ngx.say("set keepalive error : ", err) end end function init_redis() local redis = require 'resty.redis' redisCli, err = redis:new() if not redisCli then ngx.log(ngx.ERR, "failed to instantiate redis: ", err) return nil end local ok, err = redisCli:connect(config.redis.host, config.redis.port) if not ok then ngx.log(ngx.ERR, "Failed connect to redis database: ", err) close_cosock(redisCli) return nil end redisCli:ping() pcall(redisCli.auth , config.redis.password) return redisCli end function init_mysql() local mysql = require 'resty.mysql' mysqlCli, err = mysql:new() if not mysqlCli then ngx.log(ngx.ERR, "failed to instantiate mysql: ", err) return nil end local ok, err, errno, sqlstate = mysqlCli:connect(config.mysql) if not ok then ngx.log(ngx.ERR, "failed to connect: ", err, ": ", errno, " ", sqlstate) close_cosock(mysqlCli) return nil end return mysqlCli end

config.lua
function env(envName, defaultValue) local envValue = os.getenv(envName) if not envValue then envValue = defaultValue end return envValue end local _config = { ["mysql"] = { ["host"] = env("MYSQL_SERVER_NAME", "192.168.59.3"), ["port"] = tonumber(env("MYSQL_SERVER_PORT", 3306)), ["user"] = env("MYSQL_USER", "coding_ide"), ["password"] = env("MYSQL_PASSWORD", "coding_ide"), ["database"] = env("MYSQL_DATABASE", "coding_ide"), ["max_packet_size"] = 1024 * 1024 }, ["redis"] = { ["host"] = env("REDIS_HOST", "192.168.59.3"), ["port"] = tonumber(env("REDIS_PORT", 6379)), ["password"] = env("REDIS_PASSWORD", "") }, ["pool"] = { ["idle_time"] = 10000, --最大等待时间ms ["size"] = 100 --连接池大小 }, ["defaultGroup"] = env("DEFAULT_HOST", "ide-backend") } config = _config return config

agentzh

Hello!

On Fri, Jul 10, 2015 at 7:02 PM, Yibai Zhang wrote:
> 2015/07/10 09:06:14 [error] 34#0: *8 lua entry thread aborted: runtime
> error: /var/lib/ide-openresty/lua/sharding.lua:5: attempt to index global
> 'config' (a nil value)

这个错误提示你滥用了 Lua 全局变量。应当尽量避免使用 Lua 全局变量。这一点在官方文档中有详细的说明：

    https://github.com/openresty/lua-nginx-module#lua-variable-scope

> lua程序(均在同一文件夹中)：
> sharding.lua
> require "common"
> local spaceKey = ngx.req.get_headers()['X-Space-Key']
> local shardingHeader = ngx.req.get_headers()['X-Sharding-Group']
>
> targetGroup = config.defaultGroup

你上面给出的错误消息正是在这一行抛出来的。这一行明显有问题，因为你这里读取了全局变量 config，而 Lua
全局变量的生命期是每请求的（见上面引用的文档链接）。

> redisCli = init_redis()

上面这两行你使用了 Lua 全局变量？

> local response = redisCli:ping()
> mysqlConn = init_mysql()

上面这一行也是全局变量？

> res, err, errno, sqlstate = mysqlConn:query(sql)

这里也都是全局变量？

> common.lua
> cjson = require 'cjson'
> config = require 'config'
>

上面这两行都是定义的全局变量？

> function close_cosock(cosock)

这个 function 也是全局变量。正确的写法是

    local function close_cosock(cosock)

> function init_redis()

同上。

> local redis = require 'resty.redis'

一般不应该把 require 语句放在函数内部，因为这样每次函数调用都要执行一遍 require() 函数。虽然 require()
本身有缓存，但多少也会引入额外开销。

> function init_mysql()

同上。

> config.lua
> function env(envName, defaultValue)

同上。

建议按上面文档中介绍的 lua-releng 工具纠正所有你代码中读写 Lua 全局变量的地方。Lua
全局变量不仅容易引入难以调试的副作用，而且还会降低你 Lua 代码的执行效率（相比 local 词法变量）。

> 我已经找到问题了。。打开code_cache后无法使用require。现已改为init_by_lua

使用 init_by_lua 仍然是在滥用 Lua 全局变量 :)

Regards,
-agentzh