How do you issue a PUT subrequest with a file transfer

james.marlowe · 2014-03-11T08:28:15+00:00

It seems my problem is related to variable scope. In nginx.conf I have location /no_cookies { access_b...

How do you issue a PUT subrequest with a file transfer

james.marlowe

I see that you can set the method of a subrequest like so:

res = ngx.location.capture(
    '/foo/bar',
    { method = ngx.HTTP_POST, body = 'hello, world' }
)

I could change the method to put, but how would I transfer a file? essentially the -T option for curl as found at http://curl.haxx.se/docs/manpage.html#-T

If thats not possible, can it be done inline? The file I want to transfer is text anyway, so that may be fine (assuming the recieving end can tell whats happening, I'm not in control of it)

agentzh

Hello!

On Mon, Mar 10, 2014 at 5:28 PM, James Marlowe wrote:
> res = ngx.location.capture(
>     '/foo/bar',
>     { method = ngx.HTTP_POST, body = 'hello, world' }
> )
>
> I could change the method to put, but how would I transfer a file?

Well, for now, you need to read the whole file in Lua to a Lua string
and pass it to the "body" option. Yes, this is quite inefficient for
large files.

We could add a "body_file" opiton to ngx.location.capture* functions,
just as the -f option in the echo_subreqruest directive in the
ngx_echo module. Patches welcome :)

Alternatively, you can try pure Lua implementations based on ngx_lua's
cosocket API (like lua-resty-http or lua-resty-fastcgi, depending on
the subrequest target here).

Regards,
-agentzh

james.marlowe

In case anyone who comes here was wondering, I was connecting to aws s3 to upload a file. Setting the body to a string does indeed get written to the destination file in s3. Pretty cool, and for my use case (which is sort of like a proxy) setting it as a string rather than a file worked out fine. I also tried using io.read() to set the string for the body which seems to work fine too.

Thanks for the help

david.birdsong

On Mon, Mar 10, 2014 at 9:42 PM, Yichun Zhang (agentzh) <age...@gmail.com> wrote:

Hello!

On Mon, Mar 10, 2014 at 5:28 PM, James Marlowe wrote:
> res = ngx.location.capture(
> '/foo/bar',
> { method = ngx.HTTP_POST, body = 'hello, world' }
> )
>
> I could change the method to put, but how would I transfer a file?

Well, for now, you need to read the whole file in Lua to a Lua string
and pass it to the "body" option. Yes, this is quite inefficient for
large files.

I've actually used this for a long time. I had to rip it out, so I might be missing something, but the original request was a POST and I convert to a PUT:

https://gist.github.com/davidbirdsong/9502254

We could add a "body_file" opiton to ngx.location.capture* functions,
just as the -f option in the echo_subreqruest directive in the
ngx_echo module. Patches welcome :)

Alternatively, you can try pure Lua implementations based on ngx_lua's
cosocket API (like lua-resty-http or lua-resty-fastcgi, depending on
the subrequest target here).

Regards,
-agentzh

.

agentzh

Hello!

On Wed, Mar 12, 2014 at 12:16 AM, David Birdsong wrote:
> I've actually used this for a long time. I had to rip it out, so I might be
> missing something, but the original request was a POST and I convert to a
> PUT:
> https://gist.github.com/davidbirdsong/9502254
>

This ngx.req.set_body_file trick is clever :) Thanks for sharing!

Best regards,
-agentzh

david.birdsong

On Wed, Mar 12, 2014 at 10:40 AM, Yichun Zhang (agentzh) <age...@gmail.com> wrote:

Hello!

On Wed, Mar 12, 2014 at 12:16 AM, David Birdsong wrote:
> I've actually used this for a long time. I had to rip it out, so I might be
> missing something, but the original request was a POST and I convert to a
> PUT:
> https://gist.github.com/davidbirdsong/9502254
>

This ngx.req.set_body_file trick is clever :) Thanks for sharing!

I learned it from you, agentzh.

Best regards,

-agentzh.