resty.http connection timeout

gmadarm · 2014-07-16T19:58:51+00:00

Hello! On Tue, Jul 15, 2014 at 4:51 PM, Cefan Daniel Rubin wrote: > So if I want to pretty-print syntax errors in my app you suggest I wrap > everythi...

resty.http connection timeout

gmadarm

I use lua-resty-http to perform an external API query. Many times I need to run the script twice. The first time the request times out with the error below, the second time I get the expected response. I tried increasing the resty.http timeout value several times via set_timeout() to no avail. In a situation where the remote host resets the connection, how can I gracefully handle that error?

2014/07/16 07:36:13 [error] 10742#0: *2176281 lua tcp socket read timed out,
client: 1.2.3.4, server: acme.com, request: "GET /script
HTTP/1.1", host: "acme.com"
2014/07/16 07:36:13 [error] 10742#0: *2176281 lua entry thread aborted: runtime
error: /usr/local/src/lua-resty-http/lib/resty/http.lua:601: bad argument #1 to
'pairs' (table expected, got nil)
stack traceback:
coroutine 0:
        [C]: in function 'pairs'
        /usr/local/src/lua-resty-http/lib/resty/http.lua:601: in function
'__index'
        /opt/openresty/nginx/conf/script.lua:171: in function 'HTTPClient'
        /opt/openresty/nginx/conf/script.lua:327: in function 'main'
        /opt/openresty/nginx/conf/script.lua:349: in function
</opt/openresty/nginx/conf/script.lua:1>, client: 1.2.3.4, server:
acme.com, request: "GET /script HTTP/1.1", host: "acme.com"
2014/07/16 07:36:13 [info] 10742#0: *2176281 recv() failed (104: Connection
reset by peer) while sending to client, client: 1.2.3.4, server:
acme.com, request: "GET /script HTTP/1.1", upstream:
"fastcgi://unix:/var/run/php5-fpm.sock", host: "acme.com"

thanks,
-G

manchev

Which of the 3 or even 4 lua-resty-http libraries are you using?

A self-contained code example is always useful to illustrate what you're trying to do and thus get better help from the list.

Anyhow - it sounds like you should rather use pcall/xpcall in these cases to catch the error.

Best,
Vladislav

On Wed, Jul 16, 2014 at 2:58 PM, Gordon Madarm <gma...@gmail.com> wrote:

I use lua-resty-http to perform an external API query. Many times I need to run the script twice. The first time the request times out with the error below, the second time I get the expected response. I tried increasing the resty.http timeout value several times via set_timeout() to no avail. In a situation where the remote host resets the connection, how can I gracefully handle that error?

2014/07/16 07:36:13 [error] 10742#0: *2176281 lua tcp socket read timed out,
client: 1.2.3.4, server: acme.com, request: "GET /script
HTTP/1.1", host: "acme.com"
2014/07/16 07:36:13 [error] 10742#0: *2176281 lua entry thread aborted: runtime
error: /usr/local/src/lua-resty-http/lib/resty/http.lua:601: bad argument #1 to
'pairs' (table expected, got nil)
stack traceback:
coroutine 0:
        [C]: in function 'pairs'
        /usr/local/src/lua-resty-http/lib/resty/http.lua:601: in function
'__index'
        /opt/openresty/nginx/conf/script.lua:171: in function 'HTTPClient'
        /opt/openresty/nginx/conf/script.lua:327: in function 'main'
        /opt/openresty/nginx/conf/script.lua:349: in function
</opt/openresty/nginx/conf/script.lua:1>, client: 1.2.3.4, server:
acme.com, request: "GET /script HTTP/1.1", host: "acme.com"
2014/07/16 07:36:13 [info] 10742#0: *2176281 recv() failed (104: Connection
reset by peer) while sending to client, client: 1.2.3.4, server:
acme.com, request: "GET /script HTTP/1.1", upstream:
"fastcgi://unix:/var/run/php5-fpm.sock", host: "acme.com"

thanks,
-G

.

gmadarm

On Wednesday, July 16, 2014 3:29:33 PM UTC+3, Vladislav Manchev wrote:

Which of the 3 or even 4 lua-resty-http libraries are you using?

Hi Vladislav,

I'm using: https://github.com/pintsized/lua-resty-http

A self-contained code example is always useful to illustrate what you're trying to do and thus get better help from the list.

The issue is in the function below. As per the error log, line 171 corresponds to: Response = r:read_body()

function HTTPClient(APIIP, TargetIP)
    local http = require "resty.http"
    local URI1= "/"
    local URI2 = "/check/" .. TargetIP .. "/"
    local Host = "www.acme2.com"
    local Response = ""
    local httpc = http.new()
    local HTTPClientScore = 0

    httpc:set_timeout(2000)
    httpc:connect(APIIP, 80)

    ngx.log(ngx.ERR, "Connected to ", Host)

    local res, err = httpc:request_pipeline{
        {
            path = URI1,
            method = "POST",
            headers = {
                ["Host"] = Host,
                ["Content-Type"] = "application/x-www-form-urlencoded"
            },
            body = "ip=" .. TargetIP,
        },
        {
            path = URI2,
            headers = {
                ["Host"] = Host,
            },
        }
    }

    if err then
        ngx.log(ngx.ERR, "Error on connection", err)
    end

    if res == nil then
        ngx.log(ngx.ERR, "No response received from acme2.com: ", err)
    else
        for i,r in ipairs(res) do
            Response = r:read_body()
        end
        httpc:set_keepalive()
        return Response
    end
end

thanks,
- G

Anyhow - it sounds like you should rather use pcall/xpcall in these cases to catch the error.

Best,
Vladislav

On Wed, Jul 16, 2014 at 2:58 PM, Gordon Madarm <gma...@gmail.com> wrote:

I use lua-resty-http to perform an external API query. Many times I need to run the script twice. The first time the request times out with the error below, the second time I get the expected response. I tried increasing the resty.http timeout value several times via set_timeout() to no avail. In a situation where the remote host resets the connection, how can I gracefully handle that error?

2014/07/16 07:36:13 [error] 10742#0: *2176281 lua tcp socket read timed out,
client: 1.2.3.4, server: acme.com, request: "GET /script
HTTP/1.1", host: "acme.com"
2014/07/16 07:36:13 [error] 10742#0: *2176281 lua entry thread aborted: runtime
error: /usr/local/src/lua-resty-http/lib/resty/http.lua:601: bad argument #1 to
'pairs' (table expected, got nil)
stack traceback:
coroutine 0:
        [C]: in function 'pairs'
        /usr/local/src/lua-resty-http/lib/resty/http.lua:601: in function
'__index'
        /opt/openresty/nginx/conf/script.lua:171: in function 'HTTPClient'
        /opt/openresty/nginx/conf/script.lua:327: in function 'main'
        /opt/openresty/nginx/conf/script.lua:349: in function
</opt/openresty/nginx/conf/script.lua:1>, client: 1.2.3.4, server:
acme.com, request: "GET /script HTTP/1.1", host: "acme.com"
2014/07/16 07:36:13 [info] 10742#0: *2176281 recv() failed (104: Connection
reset by peer) while sending to client, client: 1.2.3.4, server:
acme.com, request: "GET /script HTTP/1.1", upstream:
"fastcgi://unix:/var/run/php5-fpm.sock", host: "acme.com"

thanks,
-G

.

manchev

I meant your own piece of code where you're trying to use lua-resty-http.

Also, take a look at the two commits from today - https://github.com/pintsized/lua-resty-http/commits/master

Seems that James fixed the problem you're experiencing and added a note in the docs.

Still not a bad idea to take a look at pcall/xpcall for future reference.

Best,

Vladislav

On Wed, Jul 16, 2014 at 3:55 PM, Gordon Madarm <gma...@gmail.com> wrote:

On Wednesday, July 16, 2014 3:29:33 PM UTC+3, Vladislav Manchev wrote:

Which of the 3 or even 4 lua-resty-http libraries are you using?

Hi Vladislav,

I'm using: https://github.com/pintsized/lua-resty-http

A self-contained code example is always useful to illustrate what you're trying to do and thus get better help from the list.

The issue is in the function below. As per the error log, line 171 corresponds to: Response = r:read_body()

function HTTPClient(APIIP, TargetIP)
    local http = require "resty.http"
    local URI1= "/"
    local URI2 = "/check/" .. TargetIP .. "/"
    local Host = "www.acme2.com"
    local Response = ""
    local httpc = http.new()
    local HTTPClientScore = 0

    httpc:set_timeout(2000)
    httpc:connect(APIIP, 80)

    ngx.log(ngx.ERR, "Connected to ", Host)

    local res, err = httpc:request_pipeline{
        {
            path = URI1,
            method = "POST",
            headers = {
                ["Host"] = Host,
                ["Content-Type"] = "application/x-www-form-urlencoded"
            },
            body = "ip=" .. TargetIP,
        },
        {
            path = URI2,
            headers = {
                ["Host"] = Host,
            },
        }
    }

    if err then
        ngx.log(ngx.ERR, "Error on connection", err)
    end

    if res == nil then
        ngx.log(ngx.ERR, "No response received from acme2.com: ", err)
    else
        for i,r in ipairs(res) do
            Response = r:read_body()
        end
        httpc:set_keepalive()
        return Response
    end
end

thanks,
- G

Anyhow - it sounds like you should rather use pcall/xpcall in these cases to catch the error.

Best,
Vladislav

On Wed, Jul 16, 2014 at 2:58 PM, Gordon Madarm <gma...@gmail.com> wrote:

I use lua-resty-http to perform an external API query. Many times I need to run the script twice. The first time the request times out with the error below, the second time I get the expected response. I tried increasing the resty.http timeout value several times via set_timeout() to no avail. In a situation where the remote host resets the connection, how can I gracefully handle that error?

2014/07/16 07:36:13 [error] 10742#0: *2176281 lua tcp socket read timed out,
client: 1.2.3.4, server: acme.com, request: "GET /script
HTTP/1.1", host: "acme.com"
2014/07/16 07:36:13 [error] 10742#0: *2176281 lua entry thread aborted: runtime
error: /usr/local/src/lua-resty-http/lib/resty/http.lua:601: bad argument #1 to
'pairs' (table expected, got nil)
stack traceback:
coroutine 0:
        [C]: in function 'pairs'
        /usr/local/src/lua-resty-http/lib/resty/http.lua:601: in function
'__index'
        /opt/openresty/nginx/conf/script.lua:171: in function 'HTTPClient'
        /opt/openresty/nginx/conf/script.lua:327: in function 'main'
        /opt/openresty/nginx/conf/script.lua:349: in function
</opt/openresty/nginx/conf/script.lua:1>, client: 1.2.3.4, server:
acme.com, request: "GET /script HTTP/1.1", host: "acme.com"
2014/07/16 07:36:13 [info] 10742#0: *2176281 recv() failed (104: Connection
reset by peer) while sending to client, client: 1.2.3.4, server:
acme.com, request: "GET /script HTTP/1.1", upstream:
"fastcgi://unix:/var/run/php5-fpm.sock", host: "acme.com"

thanks,
-G

.

james_9172

Hi,

This is a definitely a timeout error, though the error you're getting is because I failed to test for it (this only applies to pipelined requests). I've made this a bit better: https://github.com/pintsized/lua-resty-http/commit/5bc745ff510d90b5e76dc75121cdbf8fd3cea466

...so it no longer just bails on pairs(), and will log the socket error, and have updated the docs to show you must test the status for a reasonable value before you access the other fields.

That is, you should do this:

for i,r in ipairs(res) do

if r.status then
Response = r:read_body()

else

-- There was an error reading this response.

end
end

But obviously the real issue is because the connection is timing out, so you'll want to investigate / fix that separately.

I would also add, pipelining probably doesn't really buy you much. You could just do two requests normally, and use keepalives. YMMV.

To explain the issue a bit further: Pipelined requests are kinda clumsy, in that the responses simply appear one after the other on the socket. So you have to read everything fully before moving on. In addition, I wanted to make sure the streaming API in this module was still valid for pipelined requests. So the interface is little bit "magic", in that the "responses" table which you get back from "request_pipeline", is just a table of handles, and I use a metamethod to actually read the response as you access the fields themselves. So long as you access the responses in order, it will read status and headers on first "touch" of a field, and the body only when you ask for it (though you *must* ask for it in order to move on).

I toyed with having a "read_response" method, but the trouble is you might want to stream the actual body, and even merge in trailers afterwards, so the interface just looks weird.

James.

On 16 July 2014 13:55, Gordon Madarm <gma...@gmail.com> wrote:

On Wednesday, July 16, 2014 3:29:33 PM UTC+3, Vladislav Manchev wrote:

Which of the 3 or even 4 lua-resty-http libraries are you using?

Hi Vladislav,

I'm using: https://github.com/pintsized/lua-resty-http

A self-contained code example is always useful to illustrate what you're trying to do and thus get better help from the list.

The issue is in the function below. As per the error log, line 171 corresponds to: Response = r:read_body()

function HTTPClient(APIIP, TargetIP)
    local http = require "resty.http"
    local URI1= "/"
    local URI2 = "/check/" .. TargetIP .. "/"
    local Host = "www.acme2.com"
    local Response = ""
    local httpc = http.new()
    local HTTPClientScore = 0

    httpc:set_timeout(2000)
    httpc:connect(APIIP, 80)

    ngx.log(ngx.ERR, "Connected to ", Host)

    local res, err = httpc:request_pipeline{
        {
            path = URI1,
            method = "POST",
            headers = {
                ["Host"] = Host,
                ["Content-Type"] = "application/x-www-form-urlencoded"
            },
            body = "ip=" .. TargetIP,
        },
        {
            path = URI2,
            headers = {
                ["Host"] = Host,
            },
        }
    }

    if err then
        ngx.log(ngx.ERR, "Error on connection", err)
    end

    if res == nil then
        ngx.log(ngx.ERR, "No response received from acme2.com: ", err)
    else
        for i,r in ipairs(res) do
            Response = r:read_body()
        end
        httpc:set_keepalive()
        return Response
    end
end

thanks,
- G

Anyhow - it sounds like you should rather use pcall/xpcall in these cases to catch the error.

Best,
Vladislav

On Wed, Jul 16, 2014 at 2:58 PM, Gordon Madarm <gma...@gmail.com> wrote:

I use lua-resty-http to perform an external API query. Many times I need to run the script twice. The first time the request times out with the error below, the second time I get the expected response. I tried increasing the resty.http timeout value several times via set_timeout() to no avail. In a situation where the remote host resets the connection, how can I gracefully handle that error?

2014/07/16 07:36:13 [error] 10742#0: *2176281 lua tcp socket read timed out,
client: 1.2.3.4, server: acme.com, request: "GET /script
HTTP/1.1", host: "acme.com"
2014/07/16 07:36:13 [error] 10742#0: *2176281 lua entry thread aborted: runtime
error: /usr/local/src/lua-resty-http/lib/resty/http.lua:601: bad argument #1 to
'pairs' (table expected, got nil)
stack traceback:
coroutine 0:
        [C]: in function 'pairs'
        /usr/local/src/lua-resty-http/lib/resty/http.lua:601: in function
'__index'
        /opt/openresty/nginx/conf/script.lua:171: in function 'HTTPClient'
        /opt/openresty/nginx/conf/script.lua:327: in function 'main'
        /opt/openresty/nginx/conf/script.lua:349: in function
</opt/openresty/nginx/conf/script.lua:1>, client: 1.2.3.4, server:
acme.com, request: "GET /script HTTP/1.1", host: "acme.com"
2014/07/16 07:36:13 [info] 10742#0: *2176281 recv() failed (104: Connection
reset by peer) while sending to client, client: 1.2.3.4, server:
acme.com, request: "GET /script HTTP/1.1", upstream:
"fastcgi://unix:/var/run/php5-fpm.sock", host: "acme.com"

thanks,
-G

.

--
James Hurst
ja...@pintsized.co.uk