non blocking call to write in a file

gg.lafon · 2014-07-01T19:41:01+00:00

On Wednesday, July 2, 2014 4:05:23 PM UTC-4, agentzh wrote:Hello! On Wed, Jul 2, 2014 at 9:22 AM, jon.keys wrote: > I'm curious if it would be p...

non blocking call to write in a file

gg.lafon

Hello,

How could I do to write in a file using a non blocking call?

Can I use io.open() directly?

Jérôme

agentzh

Hello!

On Tue, Jul 1, 2014 at 4:41 AM, Jerome Lafon wrote:
> How could I do to write in a file using a non blocking call?

There is no such thing as nonblocking mode on normal file descriptors
for many POSIX-style operating systems including Linux.

We could use the kernel AIO feature but it has many limitations on
Linux which is of little practice merits.

A common workaround is to introduce extra OS threads but OS threads
add quite some complexity and have their own (relatively large)
overhead.

> Can I use io.open() directly?
>

You can. But better cache and share the file descriptor yourself on
the Lua land (to save the open and close syscalls) and prevent
flushing big volume of data onto file that would block the nginx event
loop significantly.

File I/O in nginx is usually blocking (unless with AIO enabled, but
AIO has its own drawbacks) and better keep the file operations small.
You can always quantitatively measure the actual blocking effect in a
live nginx worker process via tools like epoll-loop-blocking-distr
(https://github.com/openresty/stapxx#epoll-loop-blocking-distr ) and
off-CPU flame graphs
(https://github.com/openresty/nginx-systemtap-toolkit#sample-bt-off-cpu
).

Best regards,
-agentzh

gg.lafon

Thank you for your clear explanation.

.Perhaps I will use lua-resty-shell in order to run a bash script with some arguments to write in append mode in a file. This will not block openresty I believe.

any comments on this workaround?

Jérôme

Le mardi 1 juillet 2014 13:41:00 UTC+2, Jerome Lafon a écrit :

Hello,
How could I do to write in a file using a non blocking call?
Can I use io.open() directly?

Jérôme

agentzh

Hello!

On Wed, Jul 2, 2014 at 4:38 AM, Jerome Lafon wrote:
> .Perhaps I will use lua-resty-shell in order to run a bash script with some
> arguments to write in append mode in a file. This will not block openresty I
> believe.

Always do active benchmark and compare because your workaround may be
more expensive.

Regards,
-agentzh