Upload module for reference

sagar.dixit · 2014-04-02T01:55:08+00:00

Hello,Can you give me more details on the work to do and on your company please?Best regards,JérômeLe mardi 1 avril 2014 19:34:44 UTC+2, ming zhang...

Upload module for reference

sagar.dixit

Hi,

I am writing an nginx module that serves static files and also supports upload of large files ( 100 KBs ~ 16 MB ).
One of the requirements I have is, I cannot afford to write small chunks as I get them from network to disk and instead, need to accumulate them until entire buffer is assembled in memory and then write out to underlying disk. Is there any openresty module that is close to what I am looking for ? Even if there no such, could you please point any reference to an openresty module that has similar functionality whose code I can refer to ?

Thanks

agentzh

Hello!

On Tue, Apr 1, 2014 at 10:55 AM, SSD wrote:
> I am writing an nginx module that serves static files and also supports
> upload of large files ( 100 KBs ~ 16 MB ).
> One of the requirements I have is,  I cannot afford to write small chunks as
> I get them from network to disk and instead,  need to accumulate them until
> entire buffer is assembled in memory and then write out to underlying disk.
> Is there any openresty module that is close to what I am looking for ?
> Even if there no such,  could you please point any reference to an openresty
> module that has similar functionality whose code I can refer to ?
>

NGINX's builtin request body reader always fully buffers the request
body. And in ngx_lua, it's just as simple as

    ngx.req.read_body()
    local data = ngx.req.get_body_data()
    write_to_file(data)

See

    https://github.com/chaoslawful/lua-nginx-module#ngxreqget_body_data

    https://github.com/chaoslawful/lua-nginx-module#ngxreqget_body_data

But a more efficient way is to interleave network reading and disk
writing as much as possible. You can use larger data chunks anyway.
See

    https://github.com/chaoslawful/lua-nginx-module#ngxreqsocket

and

    https://github.com/agentzh/lua-resty-upload

Regards,
-agentzh