openresty redis based url rewrite

ismailyenigul · 2017-03-14T04:02:11+00:00

In the docs for ngx.req.get_body_file, there's the following guidance: "The returned file is read only and is usually cleaned up by Nginx's memor...

openresty redis based url rewrite

ismailyenigul

Hello,

I would like to create dynamic routing based based on request_uri patterns.
I would like to redirect http://mydomain.com/user1/app1/subfolders to https://mydomain.com/?server=$target/user1/app1 if /user1/app1 (first two folders) has a redis key.

# redis-cli
127.0.0.1:6379> set /user/app1 server1.com
OK

In this case the url rewrite will be https://mydomain.com/?server=server1.com/user1/app1
If there is no value for /user/app1 key, It will be redirect to another predefined server .
my below code is based on redis sample openresty webpage.
location / {

        resolver 8.8.4.4; # use Google's open DNS server

         set $target '';

access_by_lua_block {

function split(pString, pPattern)
   local Table = {} -- NOTE: use {n = 0} in Lua-5.0
   local fpat = "(.-)" .. pPattern
   local last_end = 1
   local s, e, cap = pString:find(fpat, 1)
   while s do
      if s ~= 1 or cap ~= "" then
     table.insert(Table,cap)
      end
      last_end = e+1
      s, e, cap = pString:find(fpat, last_end)
   end
   if last_end <= #pString then
      cap = pString:sub(last_end)
      table.insert(Table, cap)
   end
   return Table
end

                local myuri = ngx.var.request_uri
                list=split(myuri,"/")
                userapp="/" .. list[1] .. "/" .. list[2]
               if not userapp then
                    ngx.log(ngx.ERR, "no userapp path found")
                    return ngx.exit(400)
                end

                local redis = require "resty.redis"
                local red = redis:new()

                red:set_timeout(1000) -- 1 second

                local ok, err = red:connect("127.0.0.1", 6379)
                if not ok then
                    ngx.log(ngx.ERR, "failed to connect to redis: ", err)
                    return ngx.exit(500)
                end

                local host, err = red:get(userapp)
                if not host then
                    ngx.log(ngx.ERR, "failed to get redis key: ", err)
                    return ngx.exit(500)
                end

                if host == ngx.null then
                    ngx.log(ngx.ERR, "no aga host found for key ", key)
                    return ngx.exit(400)
                end

             ngx.var.target = host
}

rewrite ^/(.*) https://mydomain.com/?server=$target/$1 permanent;
The problem is here that rewrite line is processed first and I can't do anything on lua block.

How can I use above rewrite line in my lua block.
Or do you have any other suggestion?

btw I could not find a native comand to parse uri / pattern. then I used a lua script to do this.

qleex001

rewrite command works in REWRITE phase, access_by_lua works in ACCESS phase. so "rewrite" would execute earlier than access_by_lua.

There are wo ways can resolve it

1. put lua script in rewite_by_lua*, instead of access_by_lua*

2. use ngx.exec(url, args) API to do internal redirect ,instead of rewrite command,

在 2017年3月14日星期二 UTC+8上午4:02:11，Ismail Yenigul写道：

Hello,

I would like to create dynamic routing based based on request_uri patterns.
I would like to redirect http://mydomain.com/user1/app1/subfolders to https://mydomain.com/?server=$target/user1/app1 if /user1/app1 (first two folders) has a redis key.

# redis-cli
127.0.0.1:6379> set /user/app1 server1.com
OK

In this case the url rewrite will be https://mydomain.com/?server=server1.com/user1/app1
If there is no value for /user/app1 key, It will be redirect to another predefined server .
my below code is based on redis sample openresty webpage.
location / {

        resolver 8.8.4.4; # use Google's open DNS server

         set $target '';

access_by_lua_block {

function split(pString, pPattern)
   local Table = {} -- NOTE: use {n = 0} in Lua-5.0
   local fpat = "(.-)" .. pPattern
   local last_end = 1
   local s, e, cap = pString:find(fpat, 1)
   while s do
      if s ~= 1 or cap ~= "" then
     table.insert(Table,cap)
      end
      last_end = e+1
      s, e, cap = pString:find(fpat, last_end)
   end
   if last_end <= #pString then
      cap = pString:sub(last_end)
      table.insert(Table, cap)
   end
   return Table
end

                local myuri = ngx.var.request_uri
                list=split(myuri,"/")
                userapp="/" .. list[1] .. "/" .. list[2]
               if not userapp then
                    ngx.log(ngx.ERR, "no userapp path found")
                    return ngx.exit(400)
                end

                local redis = require "resty.redis"
                local red = redis:new()

                red:set_timeout(1000) -- 1 second

                local ok, err = red:connect("127.0.0.1", 6379)
                if not ok then
                    ngx.log(ngx.ERR, "failed to connect to redis: ", err)
                    return ngx.exit(500)
                end

                local host, err = red:get(userapp)
                if not host then
                    ngx.log(ngx.ERR, "failed to get redis key: ", err)
                    return ngx.exit(500)
                end

                if host == ngx.null then
                    ngx.log(ngx.ERR, "no aga host found for key ", key)
                    return ngx.exit(400)
                end

             ngx.var.target = host
}

rewrite ^/(.*) https://mydomain.com/?server=$target/$1 permanent;
The problem is here that rewrite line is processed first and I can't do anything on lua block.

How can I use above rewrite line in my lua block.
Or do you have any other suggestion?

btw I could not find a native comand to parse uri / pattern. then I used a lua script to do this.

ismailyenigul

Hi,

Thanks for your reply.

It seems that ng.exec and ng.redirect is redirecting to same host. How can I redirect to another host.

http://mydomain.com/user1/app1/subfolders to http://otherdomain.com/?server=$target/user1/app1

I would like to rewrite all URL from scratch.

for example:
https://mydomain.com/user/app1/ssss -> https://cloud.otherdomain.com/?target=https://$hostvalueinredis/user/app1/ssss

Can I use a host variable(ngx.var.myhost) defined in rewrite_by_lua_block then use this variable in nginx rewrite phase?

rewrite ^/(.*) https://$myhost ....

Thanks.

14 Mart 2017 Salı 04:08:42 UTC+2 tarihinde FQ Liu yazdı:

rewrite command works in REWRITE phase, access_by_lua works in ACCESS phase. so "rewrite" would execute earlier than access_by_lua.
There are wo ways can resolve it

1. put lua script in rewite_by_lua*, instead of access_by_lua*
2. use ngx.exec(url, args) API to do internal redirect ,instead of rewrite command,

在 2017年3月14日星期二 UTC+8上午4:02:11，Ismail Yenigul写道：
Hello,

I would like to create dynamic routing based based on request_uri patterns.
I would like to redirect http://mydomain.com/user1/app1/subfolders to https://mydomain.com/?server=$target/user1/app1 if /user1/app1 (first two folders) has a redis key.

# redis-cli
127.0.0.1:6379> set /user/app1 server1.com
OK

In this case the url rewrite will be https://mydomain.com/?server=server1.com/user1/app1
If there is no value for /user/app1 key, It will be redirect to another predefined server .
my below code is based on redis sample openresty webpage.
location / {

        resolver 8.8.4.4; # use Google's open DNS server

         set $target '';

access_by_lua_block {

function split(pString, pPattern)
   local Table = {} -- NOTE: use {n = 0} in Lua-5.0
   local fpat = "(.-)" .. pPattern
   local last_end = 1
   local s, e, cap = pString:find(fpat, 1)
   while s do
      if s ~= 1 or cap ~= "" then
     table.insert(Table,cap)
      end
      last_end = e+1
      s, e, cap = pString:find(fpat, last_end)
   end
   if last_end <= #pString then
      cap = pString:sub(last_end)
      table.insert(Table, cap)
   end
   return Table
end

                local myuri = ngx.var.request_uri
                list=split(myuri,"/")
                userapp="/" .. list[1] .. "/" .. list[2]
               if not userapp then
                    ngx.log(ngx.ERR, "no userapp path found")
                    return ngx.exit(400)
                end

                local redis = require "resty.redis"
                local red = redis:new()

                red:set_timeout(1000) -- 1 second

                local ok, err = red:connect("127.0.0.1", 6379)
                if not ok then
                    ngx.log(ngx.ERR, "failed to connect to redis: ", err)
                    return ngx.exit(500)
                end

                local host, err = red:get(userapp)
                if not host then
                    ngx.log(ngx.ERR, "failed to get redis key: ", err)
                    return ngx.exit(500)
                end

                if host == ngx.null then
                    ngx.log(ngx.ERR, "no aga host found for key ", key)
                    return ngx.exit(400)
                end

             ngx.var.target = host
}

rewrite ^/(.*) https://mydomain.com/?server=$target/$1 permanent;
The problem is here that rewrite line is processed first and I can't do anything on lua block.

How can I use above rewrite line in my lua block.
Or do you have any other suggestion?

btw I could not find a native comand to parse uri / pattern. then I used a lua script to do this.

qleex001

Maybe you can use ngx.location.capture to issue a Nginx Subrequest to do it

在 2017年3月14日星期二 UTC+8下午2:22:42，Ismail Yenigul写道：

Hi,

Thanks for your reply.

It seems that ng.exec and ng.redirect is redirecting to same host. How can I redirect to another host.

http://mydomain.com/user1/app1/subfolders to http://otherdomain.com/?server=$target/user1/app1

I would like to rewrite all URL from scratch.

for example:
https://mydomain.com/user/app1/ssss -> https://cloud.otherdomain.com/?target=https://$hostvalueinredis/user/app1/ssss

Can I use a host variable(ngx.var.myhost) defined in rewrite_by_lua_block then use this variable in nginx rewrite phase?

rewrite ^/(.*) https://$myhost ....

Thanks.

14 Mart 2017 Salı 04:08:42 UTC+2 tarihinde FQ Liu yazdı:
rewrite command works in REWRITE phase, access_by_lua works in ACCESS phase. so "rewrite" would execute earlier than access_by_lua.
There are wo ways can resolve it

1. put lua script in rewite_by_lua*, instead of access_by_lua*
2. use ngx.exec(url, args) API to do internal redirect ,instead of rewrite command,

在 2017年3月14日星期二 UTC+8上午4:02:11，Ismail Yenigul写道：
Hello,

I would like to create dynamic routing based based on request_uri patterns.
I would like to redirect http://mydomain.com/user1/app1/subfolders to https://mydomain.com/?server=$target/user1/app1 if /user1/app1 (first two folders) has a redis key.

# redis-cli
127.0.0.1:6379> set /user/app1 server1.com
OK

In this case the url rewrite will be https://mydomain.com/?server=server1.com/user1/app1
If there is no value for /user/app1 key, It will be redirect to another predefined server .
my below code is based on redis sample openresty webpage.
location / {

        resolver 8.8.4.4; # use Google's open DNS server

         set $target '';

access_by_lua_block {

function split(pString, pPattern)
   local Table = {} -- NOTE: use {n = 0} in Lua-5.0
   local fpat = "(.-)" .. pPattern
   local last_end = 1
   local s, e, cap = pString:find(fpat, 1)
   while s do
      if s ~= 1 or cap ~= "" then
     table.insert(Table,cap)
      end
      last_end = e+1
      s, e, cap = pString:find(fpat, last_end)
   end
   if last_end <= #pString then
      cap = pString:sub(last_end)
      table.insert(Table, cap)
   end
   return Table
end

                local myuri = ngx.var.request_uri
                list=split(myuri,"/")
                userapp="/" .. list[1] .. "/" .. list[2]
               if not userapp then
                    ngx.log(ngx.ERR, "no userapp path found")
                    return ngx.exit(400)
                end

                local redis = require "resty.redis"
                local red = redis:new()

                red:set_timeout(1000) -- 1 second

                local ok, err = red:connect("127.0.0.1", 6379)
                if not ok then
                    ngx.log(ngx.ERR, "failed to connect to redis: ", err)
                    return ngx.exit(500)
                end

                local host, err = red:get(userapp)
                if not host then
                    ngx.log(ngx.ERR, "failed to get redis key: ", err)
                    return ngx.exit(500)
                end

                if host == ngx.null then
                    ngx.log(ngx.ERR, "no aga host found for key ", key)
                    return ngx.exit(400)
                end

             ngx.var.target = host
}

rewrite ^/(.*) https://mydomain.com/?server=$target/$1 permanent;
The problem is here that rewrite line is processed first and I can't do anything on lua block.

How can I use above rewrite line in my lua block.
Or do you have any other suggestion?

btw I could not find a native comand to parse uri / pattern. then I used a lua script to do this.

ismailyenigul

Thanks,

I decided to use lua-uri at https://github.com/LuaDist/lua-uri to extract uri.

This is old but working ;)

Is there any equivalent of this module in lua or openresty by default?

My use case:

local MYURL = require "uri";
local uri = assert(MYURL:new("http://example.com/usr/foos?u=5"));

userapp= uri:path();
local arguments = uri:query();

2017-03-15 17:45 GMT+02:00 FQ Liu <qle...@gmail.com>:

Maybe you can use ngx.location.capture to issue a Nginx Subrequest to do it

在 2017年3月14日星期二 UTC+8下午2:22:42，Ismail Yenigul写道：
Hi,

Thanks for your reply.

It seems that ng.exec and ng.redirect is redirecting to same host. How can I redirect to another host.

http://mydomain.com/user1/app1/subfolders to http://otherdomain.com/?server=$target/user1/app1

I would like to rewrite all URL from scratch.

for example:
https://mydomain.com/user/app1/ssss -> https://cloud.otherdomain.com/?target=https://$hostvalueinredis/user/app1/ssss

Can I use a host variable(ngx.var.myhost) defined in rewrite_by_lua_block then use this variable in nginx rewrite phase?

rewrite ^/(.*) https://$myhost ....

Thanks.

14 Mart 2017 Salı 04:08:42 UTC+2 tarihinde FQ Liu yazdı:
rewrite command works in REWRITE phase, access_by_lua works in ACCESS phase. so "rewrite" would execute earlier than access_by_lua.
There are wo ways can resolve it

1. put lua script in rewite_by_lua*, instead of access_by_lua*
2. use ngx.exec(url, args) API to do internal redirect ,instead of rewrite command,

在 2017年3月14日星期二 UTC+8上午4:02:11，Ismail Yenigul写道：
Hello,

I would like to create dynamic routing based based on request_uri patterns.
I would like to redirect http://mydomain.com/user1/app1/subfolders to https://mydomain.com/?server=$target/user1/app1 if /user1/app1 (first two folders) has a redis key.

# redis-cli
127.0.0.1:6379> set /user/app1 server1.com
OK

In this case the url rewrite will be https://mydomain.com/?server=server1.com/user1/app1
If there is no value for /user/app1 key, It will be redirect to another predefined server .
my below code is based on redis sample openresty webpage.
location / {

        resolver 8.8.4.4; # use Google's open DNS server

         set $target '';

access_by_lua_block {

function split(pString, pPattern)
   local Table = {} -- NOTE: use {n = 0} in Lua-5.0
   local fpat = "(.-)" .. pPattern
   local last_end = 1
   local s, e, cap = pString:find(fpat, 1)
   while s do
      if s ~= 1 or cap ~= "" then
     table.insert(Table,cap)
      end
      last_end = e+1
      s, e, cap = pString:find(fpat, last_end)
   end
   if last_end <= #pString then
      cap = pString:sub(last_end)
      table.insert(Table, cap)
   end
   return Table
end

                local myuri = ngx.var.request_uri
                list=split(myuri,"/")
                userapp="/" .. list[1] .. "/" .. list[2]
               if not userapp then
                    ngx.log(ngx.ERR, "no userapp path found")
                    return ngx.exit(400)
                end

                local redis = require "resty.redis"
                local red = redis:new()

                red:set_timeout(1000) -- 1 second

                local ok, err = red:connect("127.0.0.1", 6379)
                if not ok then
                    ngx.log(ngx.ERR, "failed to connect to redis: ", err)
                    return ngx.exit(500)
                end

                local host, err = red:get(userapp)
                if not host then
                    ngx.log(ngx.ERR, "failed to get redis key: ", err)
                    return ngx.exit(500)
                end

                if host == ngx.null then
                    ngx.log(ngx.ERR, "no aga host found for key ", key)
                    return ngx.exit(400)
                end

             ngx.var.target = host
}

rewrite ^/(.*) https://mydomain.com/?server=$target/$1 permanent;
The problem is here that rewrite line is processed first and I can't do anything on lua block.

How can I use above rewrite line in my lua block.
Or do you have any other suggestion?

btw I could not find a native comand to parse uri / pattern. then I used a lua script to do this.

.