Variables set in lua are escaped in access log

calin

If a variable value is set by a lua script it's value is escaped when it's
send to the access log.

Example:

log_format lua $data;
server {
    set $data '{\'foo\':\'bar\'}';
    access_log lua.log lua;
}

Renders {'foo':'bar'} as log entry.

If instead it's value is modified by lua by doing ngx.var.data =
cjson.encode({foo='bar'}) the log entry woud contain
{\x22foo\x22:\x22bar\x22}

calin

My mistake \x22 is " charater. It gets escaped in both cases.

Anyway, it would be still nice to be able to set this escape flag on variables.

On Wed, Jul 3, 2013 at 11:25 PM, Calin Don <ca...@presslabs.com> wrote:

If a variable value is set by a lua script it's value is escaped when it's send to the access log.

Example:

log_format lua $data;
server {

set $data '{\'foo\':\'bar\'}';
access_log lua.log lua;
}

Renders {'foo':'bar'} as log entry.

If instead it's value is modified by lua by doing ngx.var.data = "" the log entry woud contain {\x22foo\x22:\x22bar\x22}

From what I understand by reading nginx source code, variables have a escape flag which I guess it is not set or set to true by the lua module. A function to set this flag would be handy for sending encoded data to the access log.

--
Have fun,
Calin Don
+40 724 010898
http://www.presslabs.com

agentzh

Hello!

On Wed, Jul 3, 2013 at 1:25 PM, Calin Don wrote:
> If a variable value is set by a lua script it's value is escaped when it's
> send to the access log.
>

No, escaping is done by the standard ngx_http_log_module rather than
ngx_lua. It's not in the control of ngx_lua at all. You should ask on
the nginx mailing list instead.

Also, you could consider logging directly from within Lua. Then you
can bypass the automatic variable value escaping of the standard
access_log writer.

Best regards,
-agentzh

calin

Writing the log from lua looks like a good idea. Though I have few
questions about how to implement this.

Is there any way to tap into the USR1 in order to reopen the log files
as nginx would.

Also if I open a file in init_by_lua would there be any race
conditions if multiple workers write to the same file?

On 04.07.2013, at 01:52, "Yichun Zhang (agentzh)" <age...@gmail.com> wrote:

> Hello!
>
> On Wed, Jul 3, 2013 at 1:25 PM, Calin Don wrote:
>> If a variable value is set by a lua script it's value is escaped when it's
>> send to the access log.
>
> No, escaping is done by the standard ngx_http_log_module rather than
> ngx_lua. It's not in the control of ngx_lua at all. You should ask on
> the nginx mailing list instead.
>
> Also, you could consider logging directly from within Lua. Then you
> can bypass the automatic variable value escaping of the standard
> access_log writer.
>
> Best regards,
> -agentzh.
>
>

agentzh

Hello!

On Thu, Jul 4, 2013 at 1:22 AM, Călin Don wrote:
> Is there any way to tap into the USR1 in order to reopen the log files
> as nginx would.
>

Yeah, this is a good question. We need to find a way :)

> Also if I open a file in init_by_lua would there be any race
> conditions if multiple workers write to the same file?
>

As long as you use a single write buffer and a single write() syscall
to do the write. You'd need to use LuaJIT FFI to allocate the buffer
and call libc's wrapper for the write() syscall (but not libc's fwrite
API or something like that!). Then atomicity *should* be guaranteed by
the OS for normal file systems.

Best regards,
-agentzh