http2 POST requests are not buffered properly?

pavel.zhuk · 2016-04-26T05:18:50+00:00

> Do you think I should supply lua-resty-jit-uuid in my project as well, or just to check for a fallback (e.g. pcall require)? Yes, I would suggest y...

http2 POST requests are not buffered properly?

pavel.zhuk

Hello,

We are trying to implement simple HTTP endpoint with the lua code which will handle HTTP/2 POST requests (with actual HTTP body payload).

However, every time when incoming request is coming

2016/04/25 20:48:11 [error] 4536#0: *4 lua entry thread aborted: runtime error: /etc/nginx/lua/test.lua:33: requesty body in temp file not supported

Of course, I've read previous questions about this error, and tried to tune client_body_buffer_size and client_max_body_size which is not helping in any way.

Request size is really small - about 1k only.

Any ideas on possible workarounds?

server {
listen 8082 http2 ssl;

ssl_certificate /etc/nginx/ssl/ourcert.crt;

ssl_certificate_key /etc/nginx/ssl/ourcert.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers 'ECDH+AESGCM:ECDH+AES256:ECDH+AES128:!DH+3DES:!ADH:!AECDH:!MD5';

location /lfsdk/1.0/purchase_list {
   lua_need_request_body on;
   client_body_buffer_size 1m;
   client_max_body_size 5m;

   content_by_lua_file lua/test.lua;
}

Code in the file is supper simple and fails on the very beginning of executable code:

32 ngx.req.read_body()

33 local body = ngx.req.get_post_args()

Nginx build details:

nginx version: openresty/1.9.3.1
built with OpenSSL 1.0.2d 9 Jul 2015
TLS SNI support enabled
configure arguments: --prefix=/opt/openresty/nginx --with-cc-opt=-O2 --add-module=../ngx_devel_kit-0.2.19 --add-module=../echo-nginx-module-0.58 --add-module=../xss-nginx-module-0.05 --add-
module=../ngx_coolkit-0.2rc3 --add-module=../set-misc-nginx-module-0.29 --add-module=../form-input-nginx-module-0.11 --add-module=../encrypted-session-nginx-module-0.04 --add-module=../srca
che-nginx-module-0.30 --add-module=../ngx_lua-0.9.16 --add-module=../ngx_lua_upstream-0.03 --add-module=../headers-more-nginx-module-0.26 --add-module=../array-var-nginx-module-0.04 --add-m
odule=../memc-nginx-module-0.16 --add-module=../redis2-nginx-module-0.12 --add-module=../redis-nginx-module-0.3.7 --add-module=../rds-json-nginx-module-0.14 --add-module=../rds-csv-nginx-mo
dule-0.06 --with-ld-opt=-Wl,-rpath,/opt/openresty/luajit/lib --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-
path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib
/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --with-http_dav_module --with-http_flv_module --with-http_geoip_module
--with-http_gzip_static_module --with-http_image_filter_module --with-http_realip_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_modul
e --with-ipv6 --with-sha1=/usr/include/openssl --with-md5=/usr/include/openssl --with-mail --with-mail_ssl_module --with-http_stub_status_module --with-http_secure_link_module --with-http_s
ub_module --with-http_v2_module

jenders

On Monday, April 25, 2016 at 2:18:50 PM UTC-7, Pavlo Zhuk wrote:

Hello,

We are trying to implement simple HTTP endpoint with the lua code which will handle HTTP/2 POST requests (with actual HTTP body payload).
However, every time when incoming request is coming

2016/04/25 20:48:11 [error] 4536#0: *4 lua entry thread aborted: runtime error: /etc/nginx/lua/test.lua:33: requesty body in temp file not supported

Of course, I've read previous questions about this error, and tried to tune client_body_buffer_size and client_max_body_size which is not helping in any way.
Request size is really small - about 1k only.

Any ideas on possible workarounds?

server {
listen 8082 http2 ssl;

  ssl_certificate /etc/nginx/ssl/ourcert.crt;
ssl_certificate_key /etc/nginx/ssl/ourcert.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers 'ECDH+AESGCM:ECDH+AES256:ECDH+AES128:!DH+3DES:!ADH:!AECDH:!MD5';

location /lfsdk/1.0/purchase_list {
   lua_need_request_body on;
   client_body_buffer_size 1m;
   client_max_body_size 5m;

   content_by_lua_file lua/test.lua;
}

Code in the file is supper simple and fails on the very beginning of executable code:

32 ngx.req.read_body()
33   local body = ngx.req.get_post_args()

Nginx build details:
nginx version: openresty/1.9.3.1
built with OpenSSL 1.0.2d 9 Jul 2015
TLS SNI support enabled
configure arguments: --prefix=/opt/openresty/nginx --with-cc-opt=-O2 --add-module=../ngx_devel_kit-0.2.19 --add-module=../echo-nginx-module-0.58 --add-module=../xss-nginx-module-0.05 --add-
module=../ngx_coolkit-0.2rc3 --add-module=../set-misc-nginx-module-0.29 --add-module=../form-input-nginx-module-0.11 --add-module=../encrypted-session-nginx-module-0.04 --add-module=../srca
che-nginx-module-0.30 --add-module=../ngx_lua-0.9.16 --add-module=../ngx_lua_upstream-0.03 --add-module=../headers-more-nginx-module-0.26 --add-module=../array-var-nginx-module-0.04 --add-m
odule=../memc-nginx-module-0.16 --add-module=../redis2-nginx-module-0.12 --add-module=../redis-nginx-module-0.3.7 --add-module=../rds-json-nginx-module-0.14 --add-module=../rds-csv-nginx-mo
dule-0.06 --with-ld-opt=-Wl,-rpath,/opt/openresty/luajit/lib --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-
path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib
/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --with-http_dav_module --with-http_flv_module --with-http_geoip_module
--with-http_gzip_static_module --with-http_image_filter_module --with-http_realip_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_modul
e --with-ipv6 --with-sha1=/usr/include/openssl --with-md5=/usr/include/openssl --with-mail --with-mail_ssl_module --with-http_stub_status_module --with-http_secure_link_module --with-http_s
ub_module --with-http_v2_module

Per, https://github.com/openresty/lua-nginx-module#lua_need_request_body:

if request body data is required, then this directive should be turned on or the ngx.req.read_body function should be called within the Lua code.

Either use lua_need_request_body in your nginx.conf OR ngx.req.read_body() in content_by_lua_file but not both.

pavel.zhuk

Hello Josh,

Thanks for reply - but I tried at 3 possible combinations (only lua_need_request_body on in the config, only ngx.req.read_body() in the lua file, and finally both this options enabled).

In all 3 configurations - error is the same.

When I tried to disable http2 module - everything works fine on http 1.1.
Looks like HTTP2 POST is never buffered properly and always goes to the temrrary file.
Additionaly, I tried to ramp up http2_recv_buffer_size in server configuraiton - also has no any effect.

Any ideas?

Thanks a lot

BR,
Pavlo

On Tuesday, April 26, 2016 at 4:51:39 AM UTC+3, Josh Enders wrote:

On Monday, April 25, 2016 at 2:18:50 PM UTC-7, Pavlo Zhuk wrote:
Hello,

We are trying to implement simple HTTP endpoint with the lua code which will handle HTTP/2 POST requests (with actual HTTP body payload).
However, every time when incoming request is coming

2016/04/25 20:48:11 [error] 4536#0: *4 lua entry thread aborted: runtime error: /etc/nginx/lua/test.lua:33: requesty body in temp file not supported

Of course, I've read previous questions about this error, and tried to tune client_body_buffer_size and client_max_body_size which is not helping in any way.
Request size is really small - about 1k only.

Any ideas on possible workarounds?

server {
listen 8082 http2 ssl;

  ssl_certificate /etc/nginx/ssl/ourcert.crt;
ssl_certificate_key /etc/nginx/ssl/ourcert.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers 'ECDH+AESGCM:ECDH+AES256:ECDH+AES128:!DH+3DES:!ADH:!AECDH:!MD5';

location /lfsdk/1.0/purchase_list {
   lua_need_request_body on;
   client_body_buffer_size 1m;
   client_max_body_size 5m;

   content_by_lua_file lua/test.lua;
}

Code in the file is supper simple and fails on the very beginning of executable code:

32 ngx.req.read_body()
33   local body = ngx.req.get_post_args()

Nginx build details:
nginx version: openresty/1.9.3.1
built with OpenSSL 1.0.2d 9 Jul 2015
TLS SNI support enabled
configure arguments: --prefix=/opt/openresty/nginx --with-cc-opt=-O2 --add-module=../ngx_devel_kit-0.2.19 --add-module=../echo-nginx-module-0.58 --add-module=../xss-nginx-module-0.05 --add-
module=../ngx_coolkit-0.2rc3 --add-module=../set-misc-nginx-module-0.29 --add-module=../form-input-nginx-module-0.11 --add-module=../encrypted-session-nginx-module-0.04 --add-module=../srca
che-nginx-module-0.30 --add-module=../ngx_lua-0.9.16 --add-module=../ngx_lua_upstream-0.03 --add-module=../headers-more-nginx-module-0.26 --add-module=../array-var-nginx-module-0.04 --add-m
odule=../memc-nginx-module-0.16 --add-module=../redis2-nginx-module-0.12 --add-module=../redis-nginx-module-0.3.7 --add-module=../rds-json-nginx-module-0.14 --add-module=../rds-csv-nginx-mo
dule-0.06 --with-ld-opt=-Wl,-rpath,/opt/openresty/luajit/lib --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-
path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib
/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --with-http_dav_module --with-http_flv_module --with-http_geoip_module
--with-http_gzip_static_module --with-http_image_filter_module --with-http_realip_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_modul
e --with-ipv6 --with-sha1=/usr/include/openssl --with-md5=/usr/include/openssl --with-mail --with-mail_ssl_module --with-http_stub_status_module --with-http_secure_link_module --with-http_s
ub_module --with-http_v2_module

Per, https://github.com/openresty/lua-nginx-module#lua_need_request_body:

if request body data is required, then this directive should be turned on or the ngx.req.read_body function should be called within the Lua code.

Either use lua_need_request_body in your nginx.conf OR ngx.req.read_body() in content_by_lua_file but not both.

agentzh

Hello!

On Tue, Apr 26, 2016 at 3:00 AM, Pavlo Zhuk wrote:
> Thanks for reply - but I tried at 3 possible combinations (only
> lua_need_request_body on in the config, only ngx.req.read_body() in the lua
> file, and finally both this options enabled).
> In all 3 configurations - error is the same.
>
> When I tried to disable http2 module - everything works fine on http 1.1.
> Looks like HTTP2 POST is never buffered properly and always goes to the
> temrrary file.
> Additionaly, I tried to ramp up http2_recv_buffer_size in server
> configuraiton - also has no any effect.

I guess (temporarily) enabling the nginx debugging logs can give you
some hints. See

    http://nginx.org/en/docs/debugging_log.html

The same procedure applies equally well to OpenResty BTW.

Well, HTTP 2.0 is not officially supported by ngx_lua yet. So you're
mostly on your own. We'll eventually work on it.

Regards,
-agentzh