upstream机制中问号被转义，参数解析失败问题

nucsgy

HI All,

请帮忙看下下面的问题，谢谢！

模块中发起子请求（/mysql_query?name=lisi), 通过upstream（proxy_pass)访问（ location /mysql），但是此时带过来的URL中问号被转义了（mysql%3Fname=lisi), 并且ngx.var.arg_name 和 ngx.var.args参数都为空，不知道怎么修改才能获得参数（ngx.var.arg_name）。

配置如下：

location /mysql_test{

client_body_in_single_buffer on;

sub_request_mysql test;

}

location /mysql { -->此处计划访问数据库的，通过uri传过来的参数构造SQL语句

content_by_lua '

ngx.header.content_type = "text/plain";

ngx.say("hello",ngx.var.request_uri,ngx.var.arg_name,ngx.var.args); --》此处参数arg_name, args都为空

}

location /mysql_query {

proxy_pass http://localhost/mysql;

}

root@iZ2ze3uk97m2bfv681kyx4Z:/usr/local/openresty/nginx/logs# curl -i -X POST -H "'Content-type':'application/json'" -d '{"name":"lisi"}' http://47.93.228.87:80/mysql_test

HTTP/1.1 200 OK

Server: openresty/1.13.6.1

Date: Sun, 06 May 2018 13:00:03 GMT

Content-Type: text/plain; charset=GBK

Content-Length: 30

Connection: keep-alive

hello/mysqlFname=lisinilnil

日志：

2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_sub_req_post_handler: 200 while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", subrequest: "/mysql_query?name=lisi", upstream: "http://127.0.0.1:80/mysql%3Fname=lisi", host: "47.93.228.87"

2018/05/06 20:57:45 [emerg] 19792#0: *1 data: 30, hello/mysql%3Fname=lisinilnil

while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", subrequest: "/mysql_query?name=lisi", upstream: "http://127.0.0.1:80/mysql%3Fname=lisi", host: "47.93.228.87" －－》%3F

2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: status[200] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"

2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: send header ret[0] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"

2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: send body ret[0] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"

PS:

第一次发问题邮件，不确定用中文是否合适.如果要求必须是英文的话，请提醒。

Best Regards,

Shengy

huang_hua2012

hello，

Post的请求体请使用 https://github.com/openresty/lua-nginx-module#ngxreqget_post_args API获取对应的参数

Kwanhur Huang

TL;DR

在 2018年5月7日，下午12:09，sgy <nu...@163.com> 写道：

HI All,

请帮忙看下下面的问题，谢谢！

模块中发起子请求（/mysql_query?name=lisi), 通过upstream（proxy_pass)访问（ location /mysql），但是此时带过来的URL中问号被转义了（mysql%3Fname=lisi), 并且ngx.var.arg_name 和 ngx.var.args参数都为空，不知道怎么修改才能获得参数（ngx.var.arg_name）。

配置如下：
location /mysql_test{
client_body_in_single_buffer on;
sub_request_mysql test;
}

location /mysql { -->此处计划访问数据库的，通过uri传过来的参数构造SQL语句
content_by_lua '
ngx.header.content_type = "text/plain";
ngx.say("hello",ngx.var.request_uri,ngx.var.arg_name,ngx.var.args); --》此处参数arg_name, args都为空
';
}

location /mysql_query {
proxy_pass http://localhost/mysql;
}

root@iZ2ze3uk97m2bfv681kyx4Z:/usr/local/openresty/nginx/logs# curl -i -X POST -H "'Content-type':'application/json'" -d '{"name":"lisi"}' http://47.93.228.87:80/mysql_test
HTTP/1.1 200 OK
Server: openresty/1.13.6.1
Date: Sun, 06 May 2018 13:00:03 GMT
Content-Type: text/plain; charset=GBK
Content-Length: 30
Connection: keep-alive

hello/mysqlFname=lisinilnil

日志：
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_sub_req_post_handler: 200 while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", subrequest: "/mysql_query?name=lisi", upstream: "http://127.0.0.1:80/mysql%3Fname=lisi", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 data: 30, hello/mysql%3Fname=lisinilnil
while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", subrequest: "/mysql_query?name=lisi", upstream: "http://127.0.0.1:80/mysql%3Fname=lisi", host: "47.93.228.87" －－》%3F
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: status[200] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: send header ret[0] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: send body ret[0] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"

PS:

第一次发问题

shengybest

Hello, Kwanhur,

发起初始请求时使用的是post, 子请求(/mysql_query?name=lisi)使用GET方式. mysql_query配置块处理中，通过upstream访问location /mysql部分出问题了，应该和初始请求的post没有关系吧。

请求流程：

在 2018年5月7日星期一 UTC+8下午2:14:08，Kwanhur Huang写道：

hello，

Post的请求体请使用 https://github.com/openresty/lua-nginx-module#ngxreqget_post_args API获取对应的参数

Kwanhur Huang
TL;DR

在 2018年5月7日，下午12:09，sgy <nuc...@163.com> 写道：

HI All,

请帮忙看下下面的问题，谢谢！

模块中发起子请求（/mysql_query?name=lisi), 通过upstream（proxy_pass)访问（ location /mysql），但是此时带过来的URL中问号被转义了（mysql%3Fname=lisi), 并且ngx.var.arg_name 和 ngx.var.args参数都为空，不知道怎么修改才能获得参数（ngx.var.arg_name）。

配置如下：
location /mysql_test{
client_body_in_single_buffer on;
sub_request_mysql test;
}

location /mysql { -->此处计划访问数据库的，通过uri传过来的参数构造SQL语句
content_by_lua '
ngx.header.content_type = "text/plain";
ngx.say("hello",ngx.var.request_uri,ngx.var.arg_name,ngx.var.args); --》此处参数arg_name, args都为空
';
}

location /mysql_query {
proxy_pass http://localhost/mysql;
}

root@iZ2ze3uk97m2bfv681kyx4Z:/usr/local/openresty/nginx/logs# curl -i -X POST -H "'Content-type':'application/json'" -d '{"name":"lisi"}' http://47.93.228.87:80/mysql_test
HTTP/1.1 200 OK
Server: openresty/1.13.6.1
Date: Sun, 06 May 2018 13:00:03 GMT
Content-Type: text/plain; charset=GBK
Content-Length: 30
Connection: keep-alive

hello/mysqlFname=lisinilnil

日志：
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_sub_req_post_handler: 200 while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", subrequest: "/mysql_query?name=lisi", upstream: "http://127.0.0.1:80/mysql%3Fname=lisi", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 data: 30, hello/mysql%3Fname=lisinilnil
while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", subrequest: "/mysql_query?name=lisi", upstream: "http://127.0.0.1:80/mysql%3Fname=lisi", host: "47.93.228.87" －－》%3F
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: status[200] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: send header ret[0] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: send body ret[0] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"

PS:

第一次发问题邮件，不确定用中文是否合适.如果要求必须是英文的话，请提醒。

--
Best Regards,
Shengy

--

nucsgy

Anyone can give me some help? Thank you!

在 2018年5月7日星期一 UTC+8下午4:10:01，shengybest写道：

Hello, Kwanhur,

发起初始请求时使用的是post, 子请求(/mysql_query?name=lisi)使用GET方式. mysql_query配置块处理中，通过upstream访问location /mysql部分出问题了，应该和初始请求的post没有关系吧。

请求流程：
模块中发起子请求（/mysql_query?name=lisi), 通过upstream（proxy_pass)访问（ location /mysql），但是此时带过来的URL中问号被转义了（mysql%3Fname=lisi), 并且ngx.var.arg_name 和 ngx.var.args参数都为空，不知道怎么修改才能获得参数（ngx.var.arg_name）。

在 2018年5月7日星期一 UTC+8下午2:14:08，Kwanhur Huang写道：
hello，

Post的请求体请使用 https://github.com/openresty/lua-nginx-module#ngxreqget_post_args API获取对应的参数

Kwanhur Huang
TL;DR

在 2018年5月7日，下午12:09，sgy <nuc...@163.com> 写道：

HI All,

请帮忙看下下面的问题，谢谢！

模块中发起子请求（/mysql_query?name=lisi), 通过upstream（proxy_pass)访问（ location /mysql），但是此时带过来的URL中问号被转义了（mysql%3Fname=lisi), 并且ngx.var.arg_name 和 ngx.var.args参数都为空，不知道怎么修改才能获得参数（ngx.var.arg_name）。

配置如下：
location /mysql_test{
client_body_in_single_buffer on;
sub_request_mysql test;
}

location /mysql { -->此处计划访问数据库的，通过uri传过来的参数构造SQL语句
content_by_lua '
ngx.header.content_type = "text/plain";
ngx.say("hello",ngx.var.request_uri,ngx.var.arg_name,ngx.var.args); --》此处参数arg_name, args都为空
';
}

location /mysql_query {
proxy_pass http://localhost/mysql;
}

root@iZ2ze3uk97m2bfv681kyx4Z:/usr/local/openresty/nginx/logs# curl -i -X POST -H "'Content-type':'application/json'" -d '{"name":"lisi"}' http://47.93.228.87:80/mysql_test
HTTP/1.1 200 OK
Server: openresty/1.13.6.1
Date: Sun, 06 May 2018 13:00:03 GMT
Content-Type: text/plain; charset=GBK
Content-Length: 30
Connection: keep-alive

hello/mysqlFname=lisinilnil

日志：
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_sub_req_post_handler: 200 while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", subrequest: "/mysql_query?name=lisi", upstream: "http://127.0.0.1:80/mysql%3Fname=lisi", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 data: 30, hello/mysql%3Fname=lisinilnil
while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", subrequest: "/mysql_query?name=lisi", upstream: "http://127.0.0.1:80/mysql%3Fname=lisi", host: "47.93.228.87" －－》%3F
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: status[200] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: send header ret[0] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: send body ret[0] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"

PS:

第一次发问题邮件，不确定用中文是否合适.如果要求必须是英文的话，请提醒。

--
Best Regards,
Shengy

--

nucsgy

我找到原因了，是因为我发起子请求时，ngx_http_subrequest函数传入的参数问题导致的。

原因：发起子请求时，uri和arg是两个参数，我将参数写到uri里面了，被转义了。

转义代码：

if (r->quoted_uri || r->space_in_uri || r->internal) {

escape = 2 * ngx_escape_uri(NULL, r->uri.data + loc_len,

r->uri.len - loc_len, NGX_ESCAPE_URI);

}

错误写法：

sub_uri.len = ngx_strlen("/mysql_query?name=lisi");

sub_uri.data = "">

ngx_snprintf(sub_uri.data, sub_uri.len, "%s",

"/mysql_query?name=lisi");

ngx_http_request_t *sr = NULL;

rc = ngx_http_subrequest(r, &sub_uri, NULL,

&sr, my_sub_req, NGX_HTTP_SUBREQUEST_IN_MEMORY);

if (rc != NGX_OK)

{

正确写法：

sub_uri.len = ngx_strlen("/mysql_query");

sub_uri.data = "">

ngx_snprintf(sub_uri.data, sub_uri.len, "%s",

"/mysql_query");

sub_args.len = ngx_strlen("name=lisi");

sub_args.data = "">

ngx_snprintf(sub_args.data, sub_args.len, "%s",

"name=lisi");

ngx_log_error(NGX_LOG_EMERG, r->connection->log, 0,

"sub request uri: %V", &sub_uri);

ngx_http_request_t *sr = NULL;

rc = ngx_http_subrequest(r, &sub_uri, &sub_args,

&sr, my_sub_req, NGX_HTTP_SUBREQUEST_IN_MEMORY);

if (rc != NGX_OK)

在 2018年5月9日星期三 UTC+8上午9:39:12，shengybest写道：

Anyone can give me some help? Thank you!

在 2018年5月7日星期一 UTC+8下午4:10:01，shengybest写道：
Hello, Kwanhur,

发起初始请求时使用的是post, 子请求(/mysql_query?name=lisi)使用GET方式. mysql_query配置块处理中，通过upstream访问location /mysql部分出问题了，应该和初始请求的post没有关系吧。

请求流程：
模块中发起子请求（/mysql_query?name=lisi), 通过upstream（proxy_pass)访问（ location /mysql），但是此时带过来的URL中问号被转义了（mysql%3Fname=lisi), 并且ngx.var.arg_name 和 ngx.var.args参数都为空，不知道怎么修改才能获得参数（ngx.var.arg_name）。

在 2018年5月7日星期一 UTC+8下午2:14:08，Kwanhur Huang写道：
hello，

Post的请求体请使用 https://github.com/openresty/lua-nginx-module#ngxreqget_post_args API获取对应的参数

Kwanhur Huang
TL;DR

在 2018年5月7日，下午12:09，sgy <nuc...@163.com> 写道：

HI All,

请帮忙看下下面的问题，谢谢！

模块中发起子请求（/mysql_query?name=lisi), 通过upstream（proxy_pass)访问（ location /mysql），但是此时带过来的URL中问号被转义了（mysql%3Fname=lisi), 并且ngx.var.arg_name 和 ngx.var.args参数都为空，不知道怎么修改才能获得参数（ngx.var.arg_name）。

配置如下：
location /mysql_test{
client_body_in_single_buffer on;
sub_request_mysql test;
}

location /mysql { -->此处计划访问数据库的，通过uri传过来的参数构造SQL语句
content_by_lua '
ngx.header.content_type = "text/plain";
ngx.say("hello",ngx.var.request_uri,ngx.var.arg_name,ngx.var.args); --》此处参数arg_name, args都为空
';
}

location /mysql_query {
proxy_pass http://localhost/mysql;
}

root@iZ2ze3uk97m2bfv681kyx4Z:/usr/local/openresty/nginx/logs# curl -i -X POST -H "'Content-type':'application/json'" -d '{"name":"lisi"}' http://47.93.228.87:80/mysql_test
HTTP/1.1 200 OK
Server: openresty/1.13.6.1
Date: Sun, 06 May 2018 13:00:03 GMT
Content-Type: text/plain; charset=GBK
Content-Length: 30
Connection: keep-alive

hello/mysqlFname=lisinilnil

日志：
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_sub_req_post_handler: 200 while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", subrequest: "/mysql_query?name=lisi", upstream: "http://127.0.0.1:80/mysql%3Fname=lisi", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 data: 30, hello/mysql%3Fname=lisinilnil
while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", subrequest: "/mysql_query?name=lisi", upstream: "http://127.0.0.1:80/mysql%3Fname=lisi", host: "47.93.228.87" －－》%3F
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: status[200] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: send header ret[0] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"
2018/05/06 20:57:45 [emerg] 19792#0: *1 extern_db_request_post_handler: send body ret[0] while sending to client, client: 47.93.228.87, server: localhost, request: "POST /mysql_test HTTP/1.1", host: "47.93.228.87"

PS:

第一次发问题邮件，不确定用中文是否合适.如果要求必须是英文的话，请提醒。

--
Best Regards,
Shengy

--