How to access raw bytes in body_filter_by_lua when proxy_pass remote server responds with gzip data?

gauravphoenix · 2014-11-19T14:43:48+00:00

Sure, it's even in the documentation [1]:That is, the ngx.ctx.foo entry persists across the rewrite, access, and content phases of a request.Every reques...

How to access raw bytes in body_filter_by_lua when proxy_pass remote server responds with gzip data?

gauravphoenix

Refer to following sample code -

local chunk, eof = ngx.arg[1], ngx.arg[2]
local buffered = ngx.ctx.buffered
if not buffered then
buffered = {} -- XXX we can use table.new here
ngx.ctx.buffered = buffered
end
if chunk ~= "" then
buffered[#buffered + 1] = chunk
ngx.arg[1] = nil
end
if eof then
local whole = table.concat(buffered)
ngx.ctx.buffered = nil
ngx.arg[1] = whole
end

I am finding that value of ngx.arg[1] after execution of code contains plaintext HTML when server responded with gzip encoded data. If there a way I can access full raw bytes in body_filter_by_lua? It is obvious that above code is working on chunked data but I am wondering if there is a way to access raw data.

agentzh

Hello!

On Tue, Nov 18, 2014 at 10:43 PM, Gaurav Kumar wrote:
> I am finding that value of ngx.arg[1] after execution of code contains
> plaintext HTML when server responded with gzip encoded data. If there a way
> I can access full raw bytes in body_filter_by_lua?

Two options:

1. disable gzip compression in your *backend server* by configuring

    proxy_set_header Accept-Encoding "";

or

2. de-compress the gzip data chunk in your filter via a Lua binding to
zlib (like lua-zlib).

Regards,
-agentzh