lua-resty-mysql

ostinelli · 2013-11-11T02:14:27+00:00

Hello! On Sat, Nov 9, 2013 at 7:17 AM, Uroš Gruber wrote: > Currently we just return 504 and when > web user refresh the page it will go to the...

lua-resty-mysql

ostinelli

Dear list,

I'm using the lua-resty-mysql driver in an OpenResty application, and I keep on receiving the error:

, client: 127.0.0.1, server: , request: "POST /users? HTTP/1.1", host: "127.0.0.1"

2013/11/10 13:09:44 [error] 39782#0: *1 lua entry thread aborted: runtime error: attempt to yield across C-call boundary

stack traceback:

coroutine 0:

[C]: in function 'require'

[string "content_by_lua"]:1: in function <[string "content_by_lua"]:1>, client: 127.0.0.1, server: , request: "POST /users? HTTP/1.1", host: "127.0.0.1"

According to the library limitations [1], I should not use it in init_by_lua and similar functions (which I'm not), and the resty.mysql instance is not stored at a module level. This is the connect portion:

local function mysql_connect(options)

-- ini mysql

local mysql = require "resty.mysql"

-- create sql object

local db, err = mysql:new()

if not db then error("failed to instantiate mysql: " .. err) end

-- set 1 second timeout for suqsequent operations

db:set_timeout(timeout_subsequent_ops)

-- connect to db

local db_options = {

host = options.host,

port = options.port,

database = options.database,

user = options.user,

password = options.password,

max_packet_size = max_packet_size

}

local ok, err, errno, sqlstate = db:connect(db_options)

if not ok then error("failed to connect to mysql: " .. err .. ": " .. errno .. " " .. sqlstate) end

-- return

return db

end

I do, however, store the module that calls the mysql_connection here above at global level.

Any ideas?

Thanks,

[1] https://github.com/agentzh/lua-resty-mysql#limitations

agentzh

Hello!

On Sun, Nov 10, 2013 at 10:14 AM, Roberto Ostinelli wrote:
> 2013/11/10 13:09:44 [error] 39782#0: *1 lua entry thread aborted: runtime
> error: attempt to yield across C-call boundary
> stack traceback:
> coroutine 0:
> [C]: in function 'require'

You should not call the connect() method on the top level of your Lua
modules because the require() builtin function is implemented as a C
function in the current LuaJIT implementation which prohibits yields
required by nonblocking I/O operations.

There are two options:

1. put your DB initialization code calling connect() into your own
initialization Lua function, say, init(), and ask your module user to
call init() explicitly after require().

2. ask (or sponsor) Mike Pall to rewrite the require() Lua builtin
function with LuaJIT (embedded) bytecode on the LuaJIT mailing list
such that we can yield across its boundary without problems :)

Best regards,
-agentzh

ostinelli

After additional digging it looks like the ngx_lua cosocket API is not available, and this is what is causing the lib to blow up with this message.

I'm not really sure how the cosocket cannot be available, basically this is what I have:

in nginx.conf

location / {

content_by_lua 'require(\"router\").handler(ngx)';

}

In router.lua, at the very top of the module I call the library (with the previously defined function):

-- get db object

local db_conn = mysql_connect(options)

-- execute query

local res, err, errno, sqlstate = db_conn:query(sql)

if not res then error("bad mysql result: " .. err .. ": " .. errno .. " " .. sqlstate) end

-- keepalive

mysql_keepalive(db_conn, options)

This is when it blows up.

On Sunday, November 10, 2013 1:14:27 PM UTC-5, Roberto Ostinelli wrote:

Dear list,

I'm using the lua-resty-mysql driver in an OpenResty application, and I keep on receiving the error:

, client: 127.0.0.1, server: , request: "POST /users? HTTP/1.1", host: "127.0.0.1"
2013/11/10 13:09:44 [error] 39782#0: *1 lua entry thread aborted: runtime error: attempt to yield across C-call boundary
stack traceback:
coroutine 0:
[C]: in function 'require'
[string "content_by_lua"]:1: in function <[string "content_by_lua"]:1>, client: 127.0.0.1, server: , request: "POST /users? HTTP/1.1", host: "127.0.0.1"

According to the library limitations [1], I should not use it in init_by_lua and similar functions (which I'm not), and the resty.mysql instance is not stored at a module level. This is the connect portion:

local function mysql_connect(options)
-- ini mysql
local mysql = require "resty.mysql"
-- create sql object
local db, err = mysql:new()
if not db then error("failed to instantiate mysql: " .. err) end
-- set 1 second timeout for suqsequent operations
db:set_timeout(timeout_subsequent_ops)
-- connect to db
local db_options = {
host = options.host,
port = options.port,
database = options.database,
user = options.user,
password = options.password,
max_packet_size = max_packet_size
}
local ok, err, errno, sqlstate = db:connect(db_options)
if not ok then error("failed to connect to mysql: " .. err .. ": " .. errno .. " " .. sqlstate) end
-- return
return db
end

I do, however, store the module that calls the mysql_connection here above at global level.

Any ideas?

Thanks,

r.

[1] https://github.com/agentzh/lua-resty-mysql#limitations

ostinelli

We posted at almost the same time, didn't see your reply.

Can you please give me an example of what you mean with 1?

As usual, thank you :)

On Sunday, November 10, 2013 2:59:32 PM UTC-5, agentzh wrote:

Hello!

On Sun, Nov 10, 2013 at 10:14 AM, Roberto Ostinelli wrote:
> 2013/11/10 13:09:44 [error] 39782#0: *1 lua entry thread aborted: runtime
> error: attempt to yield across C-call boundary
> stack traceback:
> coroutine 0:
> [C]: in function 'require'

You should not call the connect() method on the top level of your Lua
modules because the require() builtin function is implemented as a C
function in the current LuaJIT implementation which prohibits yields
required by nonblocking I/O operations.

There are two options:

1. put your DB initialization code calling connect() into your own
initialization Lua function, say, init(), and ask your module user to
call init() explicitly after require().

2. ask (or sponsor) Mike Pall to rewrite the require() Lua builtin
function with LuaJIT (embedded) bytecode on the LuaJIT mailing list
such that we can yield across its boundary without problems :)

Best regards,
-agentzh

agentzh

Hello!

On Sun, Nov 10, 2013 at 12:02 PM, Roberto Ostinelli wrote:
> In router.lua, at the very top of the module I call the library (with the
> previously defined function):
>
>     -- get db object
>     local db_conn = mysql_connect(options)
>     -- execute query
>     local res, err, errno, sqlstate = db_conn:query(sql)
>     if not res then error("bad mysql result: " .. err .. ": " .. errno .. "
> " .. sqlstate) end
>     -- keepalive
>     mysql_keepalive(db_conn, options)
>

This piece of Lua code should be put into your router module's
handler() function rather than the toplevel code chunk of your router
module. That's because Lua modules are loaded only once and the
toplevel code chunk will only be executed once, unless you turn off
lua_code_cache.

Putting the DB connect()/query() stuff into handler() will also solve
your runtime error "attempt to yield across C-call boundary".

Regards,
-agentzh

ostinelli

I already tried that, unfortunately it is not working.

By the way I also tried turning code cache off, and this error still happens..

On Sunday, November 10, 2013 3:07:20 PM UTC-5, agentzh wrote:

Hello!

On Sun, Nov 10, 2013 at 12:02 PM, Roberto Ostinelli wrote:
> In router.lua, at the very top of the module I call the library (with the
> previously defined function):
>
> -- get db object
> local db_conn = mysql_connect(options)
> -- execute query
> local res, err, errno, sqlstate = db_conn:query(sql)
> if not res then error("bad mysql result: " .. err .. ": " .. errno .. "
> " .. sqlstate) end
> -- keepalive
> mysql_keepalive(db_conn, options)
>

This piece of Lua code should be put into your router module's
handler() function rather than the toplevel code chunk of your router
module. That's because Lua modules are loaded only once and the
toplevel code chunk will only be executed once, unless you turn off
lua_code_cache.

Putting the DB connect()/query() stuff into handler() will also solve
your runtime error "attempt to yield across C-call boundary".

Regards,
-agentzh

ostinelli

Do you mean after the resty.mysql require?

On Sunday, November 10, 2013 2:59:32 PM UTC-5, agentzh wrote:

1. put your DB initialization code calling connect() into your own
initialization Lua function, say, init(), and ask your module user to
call init() explicitly after require().

agentzh

Hello!

On Sun, Nov 10, 2013 at 12:14 PM, Roberto Ostinelli wrote:
> I already tried that, unfortunately it is not working.
>

Tried what?

> By the way I also tried turning code cache off, and this error still
> happens..
>

As I've said, that particular error, "attempt to yield across C-call
boundary", is due to the fact, your require() call triggers cosocket
connect() (and other) calls directly or indirectly. Fix your code path
to avoid that. This has nothing to do with lua_code_cache.

BTW, you'd better provide a minimal and *complete* example to prevent
potential misunderstanding and confusion here.

> Do you mean after the resty.mysql require?

I meant something like this:

    local router = require "router"
    router.init()   -- connect to DB and etc
    router.handler()

such that DB connect() and etc do not yield across the boundary of require().

Regards,
-agentzh

ostinelli

On Sunday, November 10, 2013 3:57:54 PM UTC-5, agentzh wrote:

Hello!

On Sun, Nov 10, 2013 at 12:14 PM, Roberto Ostinelli wrote:
> I already tried that, unfortunately it is not working.
>

Tried what?

As per your suggestion, putting the init code in the router handler function.

> By the way I also tried turning code cache off, and this error still
> happens..
>

As I've said, that particular error, "attempt to yield across C-call
boundary", is due to the fact, your require() call triggers cosocket
connect() (and other) calls directly or indirectly. Fix your code path
to avoid that. This has nothing to do with lua_code_cache.

Indeed. I was referring to:

> This piece of Lua code should be put into your router module's

> handler() function rather than the toplevel code chunk of your router
> module. That's because Lua modules are loaded only once and the
> toplevel code chunk will only be executed once, unless you turn off
> lua_code_cache.

BTW, you'd better provide a minimal and *complete* example to prevent
potential misunderstanding and confusion here.

You're indeed very right. Will do if needed.

> Do you mean after the resty.mysql require?

I meant something like this:

local router = require "router"
router.init() -- connect to DB and etc
router.handler()

such that DB connect() and etc do not yield across the boundary of require().

Regards,
-agentzh

Thank you.