ngx.req with SPDY enabled. Should I use HAProxy in front of Openresty?

xfalcox · 2013-11-22T12:58:50+00:00

GIN v0.1.1 just released.It now has PostgreSQL support. Looking forward to agentzh's postgresql driver. ^^_r.On Friday, November 22, 2013 7:39:48 AM UTC-5, R...

ngx.req with SPDY enabled. Should I use HAProxy in front of Openresty?

xfalcox

Guys please give me some insight.

We're getting close to deploy openresty in front of our intranet services.

It will handle authentication through Lua subrequests (like on http://www.stavros.io/posts/writing-an-nginx-authentication-module-in-lua/ ), serve static content and reverse proxy to our Java Glassfish legacy apps, and soon our Scala Play! apps.

But since we will have clients from all the country, and most with bad (even sub 1MB) link speeds, I think enabling SPDY will give us a huge boost.

The problem is, we cant use ngx.req and SPDY togheter due to bugs (https://github.com/chaoslawful/lua-nginx-module/issues/142 and https://github.com/chaoslawful/lua-nginx-module/issues/173).

Should we use HAProxy in front of OpenResty just to get SPDY? What you guys think?

PS.: As you can see, english is not my mother tongue.

agentzh

Hello!

On Thu, Nov 21, 2013 at 5:17 AM,  xfalcox wrote:
>
> The problem is, we cant use ngx.req and SPDY togheter due to bugs
> (https://github.com/chaoslawful/lua-nginx-module/issues/142 and
> https://github.com/chaoslawful/lua-nginx-module/issues/173).
>

AFAIK, only the ngx.location.capture*() and ngx.req.socket() APIs are
known to have compatibility issues with SPDY. Are you really using
these two APIs in your code?

Best regards,
-agentzh

xfalcox

Hello,

We built the entire authentication logic in Lua with OpenResty, so we read the auth cookie (external service OpenAM), and use ngx.location.capture to get data about the authenticated user and store it on our own cookie and the rest at redis.

This is a HUGE gain for us, since our Java/Scala apps don't need to care about auth logic anymore.

But we want SPDY too =(

Were thinking in putting another layer in front, maybe HAProxy, but were making our stack more complicated that way.

Em sexta-feira, 22 de novembro de 2013 03h03min49s UTC-2, agentzh escreveu:

Hello!

On Thu, Nov 21, 2013 at 5:17 AM, xfalcox wrote:
>
> The problem is, we cant use ngx.req and SPDY togheter due to bugs
> (https://github.com/chaoslawful/lua-nginx-module/issues/142 and
> https://github.com/chaoslawful/lua-nginx-module/issues/173).
>

AFAIK, only the ngx.location.capture*() and ngx.req.socket() APIs are
known to have compatibility issues with SPDY. Are you really using
these two APIs in your code?

Best regards,
-agentzh

peter_booth

Here's my two cents:

Openresty was the single most valuable web technology I used in 2012, deploying it in front if a highly trafficked retail site. I used it fir complex caching logic but it quickly became the Swiss Army Knife that fixed lots of tiny issues.

If I were you, I would separate my site and serve all static content from different host (like yahoo, microsift do) without any auth logic. That static server would be Nginx SpDY

Sent from my iPhone

On Nov 21, 2013, at 8:17 AM, xfa...@gmail.com wrote:

Guys please give me some insight.

We're getting close to deploy openresty in front of our intranet services.

It will handle authentication through Lua subrequests (like on http://www.stavros.io/posts/writing-an-nginx-authentication-module-in-lua/ ), serve static content and reverse proxy to our Java Glassfish legacy apps, and soon our Scala Play! apps.

But since we will have clients from all the country, and most with bad (even sub 1MB) link speeds, I think enabling SPDY will give us a huge boost.

The problem is, we cant use ngx.req and SPDY togheter due to bugs (https://github.com/chaoslawful/lua-nginx-module/issues/142 and https://github.com/chaoslawful/lua-nginx-module/issues/173).

Should we use HAProxy in front of OpenResty just to get SPDY? What you guys think?

PS.: As you can see, english is not my mother tongue.

.

xfalcox

Sorry if I didnt expressed myself well,

We're deploying an intranet website, and the users dont have access to the internet.

Our stack will be OpenResty handling all incoming requests, auth logic, serving static content, and making reverse proxy requests for dynamic content (built in Java/Scala).

Since we cant have SPDY and ngx.location.capture at the same time, and we really want to, were trying to figure the best thing to do.

Thanks!

Em sexta-feira, 22 de novembro de 2013 11h29min33s UTC-2, Peter Booth escreveu:

Here's my two cents:

Openresty was the single most valuable web technology I used in 2012, deploying it in front if a highly trafficked retail site. I used it fir complex caching logic but it quickly became the Swiss Army Knife that fixed lots of tiny issues.

If I were you, I would separate my site and serve all static content from different host (like yahoo, microsift do) without any auth logic. That static server would be Nginx SpDY

Sent from my iPhone

On Nov 21, 2013, at 8:17 AM, xfa...@gmail.com wrote:

Guys please give me some insight.

We're getting close to deploy openresty in front of our intranet services.

It will handle authentication through Lua subrequests (like on http://www.stavros.io/posts/writing-an-nginx-authentication-module-in-lua/ ), serve static content and reverse proxy to our Java Glassfish legacy apps, and soon our Scala Play! apps.

But since we will have clients from all the country, and most with bad (even sub 1MB) link speeds, I think enabling SPDY will give us a huge boost.

The problem is, we cant use ngx.req and SPDY togheter due to bugs (https://github.com/chaoslawful/lua-nginx-module/issues/142 and https://github.com/chaoslawful/lua-nginx-module/issues/173).

Should we use HAProxy in front of OpenResty just to get SPDY? What you guys think?

PS.: As you can see, english is not my mother tongue.

.

peter_booth

Being an intranet app makes your app a special case. Are the users in the same location/ region? Do they have high or low bandwidth connections to your servers? Are they all using chrome? Is the app a rich content one, with heavy web pages?

Depending upon these things, your expected benefit from SPDY might be very different from a typical internet website with diverse user base with different connection quality.

Sent from my iPhone

On Nov 22, 2013, at 8:42 AM, Falco <xfa...@gmail.com> wrote:

Sorry if I didnt expressed myself well,

We're deploying an intranet website, and the users dont have access to the internet.

Our stack will be OpenResty handling all incoming requests, auth logic, serving static content, and making reverse proxy requests for dynamic content (built in Java/Scala).

Since we cant have SPDY and ngx.location.capture at the same time, and we really want to, were trying to figure the best thing to do.

Thanks!

Em sexta-feira, 22 de novembro de 2013 11h29min33s UTC-2, Peter Booth escreveu:
Here's my two cents:

Openresty was the single most valuable web technology I used in 2012, deploying it in front if a highly trafficked retail site. I used it fir complex caching logic but it quickly became the Swiss Army Knife that fixed lots of tiny issues.

If I were you, I would separate my site and serve all static content from different host (like yahoo, microsift do) without any auth logic. That static server would be Nginx SpDY

Sent from my iPhone

On Nov 21, 2013, at 8:17 AM, xfa...@gmail.com wrote:

Guys please give me some insight.

We're getting close to deploy openresty in front of our intranet services.

It will handle authentication through Lua subrequests (like on http://www.stavros.io/posts/writing-an-nginx-authentication-module-in-lua/ ), serve static content and reverse proxy to our Java Glassfish legacy apps, and soon our Scala Play! apps.

But since we will have clients from all the country, and most with bad (even sub 1MB) link speeds, I think enabling SPDY will give us a huge boost.

The problem is, we cant use ngx.req and SPDY togheter due to bugs (https://github.com/chaoslawful/lua-nginx-module/issues/142 and https://github.com/chaoslawful/lua-nginx-module/issues/173).

Should we use HAProxy in front of OpenResty just to get SPDY? What you guys think?

PS.: As you can see, english is not my mother tongue.

.

xfalcox

Our users are on the entire country (Brazil is huge), all use Firefox 14.0.1 (yay no IE!) and our pages today, are really non-optimized (our index has 179 requests and 1MB). There are as many as 6.000 places using our website, mostly with sub 1Mbit/s speeds.

We're trying to deploy our new stack, and then go refactor all apps.

Em sexta-feira, 22 de novembro de 2013 14h29min54s UTC-2, Peter Booth escreveu:

Being an intranet app makes your app a special case. Are the users in the same location/ region? Do they have high or low bandwidth connections to your servers? Are they all using chrome? Is the app a rich content one, with heavy web pages?

Depending upon these things, your expected benefit from SPDY might be very different from a typical internet website with diverse user base with different connection quality.

Sent from my iPhone

On Nov 22, 2013, at 8:42 AM, Falco <xfa...@gmail.com> wrote:

Sorry if I didnt expressed myself well,

We're deploying an intranet website, and the users dont have access to the internet.

Our stack will be OpenResty handling all incoming requests, auth logic, serving static content, and making reverse proxy requests for dynamic content (built in Java/Scala).

Since we cant have SPDY and ngx.location.capture at the same time, and we really want to, were trying to figure the best thing to do.

Thanks!

Em sexta-feira, 22 de novembro de 2013 11h29min33s UTC-2, Peter Booth escreveu:
Here's my two cents:

Openresty was the single most valuable web technology I used in 2012, deploying it in front if a highly trafficked retail site. I used it fir complex caching logic but it quickly became the Swiss Army Knife that fixed lots of tiny issues.

If I were you, I would separate my site and serve all static content from different host (like yahoo, microsift do) without any auth logic. That static server would be Nginx SpDY

Sent from my iPhone

On Nov 21, 2013, at 8:17 AM, xfa...@gmail.com wrote:

Guys please give me some insight.

We're getting close to deploy openresty in front of our intranet services.

It will handle authentication through Lua subrequests (like on http://www.stavros.io/posts/writing-an-nginx-authentication-module-in-lua/ ), serve static content and reverse proxy to our Java Glassfish legacy apps, and soon our Scala Play! apps.

But since we will have clients from all the country, and most with bad (even sub 1MB) link speeds, I think enabling SPDY will give us a huge boost.

The problem is, we cant use ngx.req and SPDY togheter due to bugs (https://github.com/chaoslawful/lua-nginx-module/issues/142 and https://github.com/chaoslawful/lua-nginx-module/issues/173).

Should we use HAProxy in front of OpenResty just to get SPDY? What you guys think?

PS.: As you can see, english is not my mother tongue.

.

agentzh

Hello!

On Fri, Nov 22, 2013 at 5:42 AM, Falco wrote:
>
> Since we cant have SPDY and ngx.location.capture at the same time, and we
> really want to, were trying to figure the best thing to do.
>

At CloudFlare, we run an extra Nginx (with SPDY and SSL enabled) in
front of the main Nginx doing the heavy lifting (with Lua). The
frontend SSL/SPDY Nginx uses proxy_pass to forward the traffic to the
main Nginx (with proxy_buffering turning off here).

Making ngx.location.capture work with SPDY has its own merits and has
been on my TODO list for quite a while. I'll try to give you a patch
to try later :)

Best regards,
-agentzh

xfalcox

Thanks agentzh,

I'll do the same here then.

The communication between the two nginxs are through unix sockets? I'm now researching how to create a unix socket for OpenResty.

Thanks,

Em sexta-feira, 22 de novembro de 2013 17h46min24s UTC-2, agentzh escreveu:

Hello!

On Fri, Nov 22, 2013 at 5:42 AM, Falco wrote:
>
> Since we cant have SPDY and ngx.location.capture at the same time, and we
> really want to, were trying to figure the best thing to do.
>

At CloudFlare, we run an extra Nginx (with SPDY and SSL enabled) in
front of the main Nginx doing the heavy lifting (with Lua). The
frontend SSL/SPDY Nginx uses proxy_pass to forward the traffic to the
main Nginx (with proxy_buffering turning off here).

Making ngx.location.capture work with SPDY has its own merits and has
been on my TODO list for quite a while. I'll try to give you a patch
to try later :)

Best regards,
-agentzh

agentzh

Hello!

On Mon, Nov 25, 2013 at 8:40 AM, Falco wrote:
> The communication between the two nginxs are through unix sockets?

Yes sure.

Because SPDY requires SSL, you can only direct SSL traffic to your
frontend Nginx (by listening on the 443 port only). Normal HTTP
traffic can directly go to the 80 port listened by your main Nginx.

Just remember disabling proxy_buffering in your frontend SPDY/SSL
Nginx server, which should help performance for slow clients.

Regards,
-agentzh

mat999

Did you ever get around to this?

I would also be happy to test this patch.

On Saturday, November 23, 2013 6:46:24 AM UTC+11, agentzh wrote:

Hello!

On Fri, Nov 22, 2013 at 5:42 AM, Falco wrote:
>
> Since we cant have SPDY and ngx.location.capture at the same time, and we
> really want to, were trying to figure the best thing to do.
>

At CloudFlare, we run an extra Nginx (with SPDY and SSL enabled) in
front of the main Nginx doing the heavy lifting (with Lua). The
frontend SSL/SPDY Nginx uses proxy_pass to forward the traffic to the
main Nginx (with proxy_buffering turning off here).

Making ngx.location.capture work with SPDY has its own merits and has
been on my TODO list for quite a while. I'll try to give you a patch
to try later :)

Best regards,
-agentzh

agentzh

Hello!

On Thu, Feb 27, 2014 at 6:12 PM,  mat999 wrote:
> Did you ever get around to this?
>
> I would also be happy to test this patch.
>

Please try out the latest git master HEAD of lua-nginx-module. I
committed a patch to fix (hopefully) ngx.location.capture*() in SPDY
mode.

Thanks!
-agentzh

xfalcox

Oh this is great.

We are running a pure nginx in front of openresty in the same machine, since the original message cuz we need both ngx.req and spdy.

I'm not sure I can try this patch soon, ( Brazilian Carnaval its 4 And half days off and starts today) but will report if I find something.

On Feb 28, 2014 10:50 PM, "Yichun Zhang (agentzh)" <age...@gmail.com> wrote:

Hello!

On Thu, Feb 27, 2014 at 6:12 PM, mat999 wrote:
> Did you ever get around to this?
>
> I would also be happy to test this patch.
>

Please try out the latest git master HEAD of lua-nginx-module. I
committed a patch to fix (hopefully) ngx.location.capture*() in SPDY
mode.

Thanks!
-agentzh.

agentzh

Hello!

On Fri, Feb 28, 2014 at 6:03 PM, Falco wrote:
> We are running a pure nginx in front of openresty in the same machine, since
> the original message cuz we need both ngx.req and spdy.
>

Please note that not the whole ngx.req API cannot work with SPDY, just
the ngx.req.socket() function has issues because SPDY uses its own
request body reading facility that is parallel to the standard request
body reader in the NGINX core. My latest patch will not help the
ngx.req.socket() API in the SPDY mode; it only tries to fix the SPDY
compatibility issue of the ngx.location.capture* API.

Because you're putting another NGINX in front of your ngx_lua app, it
also makes little sense to use ngx.req.socket() at all because the
request body is already fully buffered by your frontend NGINX anyway.

Regards,
-agentzh

googlproxer

You ever get this working?