how to manage a pool of websockets shared among the whole server

gg.lafon · 2014-04-17T15:34:44+00:00

if you use spdy, it might be a bug there http://forum.nginx.org/read.php?2,240278,240278#msg-240278 2014-04-16 19:24 GMT+02:00 Miguel Pousa Freire <migue...

how to manage a pool of websockets shared among the whole server

gg.lafon

Hello,

I am working on a project in which I have to manage a lot small chat for only 2 users per chat.

My first idea was to use one websocket per user to do the communication between nginx and the browser.

I need to get a spool of websockets for each user so I could send messages between users by selecting the right websocket.

But How could I have a spool of websockets shared among the whole server?

Jérôme

femaref

Do you actually need a websocket per user? There are several libraries implementing a pub/sub queue system, where you'd be able to get along with one websocket endpoint in total (at least at the beginning), but would have to deal with authentication.

One such library is Faye: http://faye.jcoglan.com/

gg.lafon

This could be a solution but if I can use only openresty with the non blocking websocket module and PHP without an other server nearby I would prefer. And the communication would be only between 2 users for each chat, not really a broadcast for several users.

Thanks for this track I keep it in my mind.

Le jeudi 17 avril 2014 10:17:04 UTC+2, Femaref a écrit :

Do you actually need a websocket per user? There are several libraries implementing a pub/sub queue system, where you'd be able to get along with one websocket endpoint in total (at least at the beginning), but would have to deal with authentication.

One such library is Faye: http://faye.jcoglan.com/

manchev

Take a look at the code here:
https://github.com/agentzh/lua-resty-websocket/issues/1#issuecomment-24816008

I would say this is the way to go, not tunneling two websockets as you suggest.

For this code to work you'll need to use patches in this fork: https://github.com/aviramc/lua-nginx-module/tree/socket-changes

Or use the lua-nginx-module directly from my repo which has the socket-changes branch merged into the newest master: https://github.com/blablacio/lua-nginx-module

PS: there are many solutions similar to Faye which Famaref pointed out, but not quite sure how well that will match your needs, maybe it's worth investigating.

On 17.04.2014 11:42, Jerome Lafon wrote:

This could be a solution but if I can use only openresty with the non blocking websocket module and PHP without an other server nearby I would prefer. And the communication would be only between 2 users for each chat, not really a broadcast for several users.
Thanks for this track I keep it in my mind.

Le jeudi 17 avril 2014 10:17:04 UTC+2, Femaref a écrit :

Do you actually need a websocket per user? There are several libraries implementing a pub/sub queue system, where you'd be able to get along with one websocket endpoint in total (at least at the beginning), but would have to deal with authentication.

One such library is Faye: http://faye.jcoglan.com/

.

gg.lafon

Traduire le message en français

I read the code provided by bungle for the chat server and it seems to be a great idea using the pub/sub mechanism of redis with the websockets.

By using for each new chat a new REDIS channel several chats could be created at the same time.

If I understand correctly this code,

- a light thread is created in order receive the new messages from REDIS via the subscribe mechanism and to send these new messages to the browser using the websocket.

- The main loop while is waiting for new messages from the browser via the websocket and publish to the right channel of REDIS.

So several users using the same REDIS channel can communicate using this architecture.

Am I right?

So each user would use one cosocket to send and retrieve messages.

I don't know well the light threads and I am wondering when the light thread will be dead?

Last question:

Currently the ability to use websockets for full duplex communication is only possible using the ua-nginx-module from https://github.com/blablacio/lua-nginx-module?

Why the patches are not included in the official lua-nginx-module?

Jérôme

Le jeudi 17 avril 2014 11:06:33 UTC+2, Vladislav Manchev a écrit :

Take a look at the code here:
https://github.com/agentzh/lua-resty-websocket/issues/1#issuecomment-24816008

I would say this is the way to go, not tunneling two websockets as you suggest.

For this code to work you'll need to use patches in this fork: https://github.com/aviramc/lua-nginx-module/tree/socket-changes

Or use the lua-nginx-module directly from my repo which has the socket-changes branch merged into the newest master: https://github.com/blablacio/lua-nginx-module

PS: there are many solutions similar to Faye which Famaref pointed out, but not quite sure how well that will match your needs, maybe it's worth investigating.

On 17.04.2014 11:42, Jerome Lafon wrote:

This could be a solution but if I can use only openresty with the non blocking websocket module and PHP without an other server nearby I would prefer. And the communication would be only between 2 users for each chat, not really a broadcast for several users.
Thanks for this track I keep it in my mind.

Le jeudi 17 avril 2014 10:17:04 UTC+2, Femaref a écrit :

Do you actually need a websocket per user? There are several libraries implementing a pub/sub queue system, where you'd be able to get along with one websocket endpoint in total (at least at the beginning), but would have to deal with authentication.

One such library is Faye: http://faye.jcoglan.com/

.

manchev

On 17.04.2014 16:33, Jerome Lafon wrote:

Traduire le message en français

I read the code provided by bungle for the chat server and it seems to be a great idea using the pub/sub mechanism of redis with the websockets.
By using for each new chat a new REDIS channel several chats could be created at the same time.

If I understand correctly this code,

- a light thread is created in order receive the new messages from REDIS via the subscribe mechanism and to send these new messages to the browser using the websocket.

Correct

- The main loop while is waiting for new messages from the browser via the websocket and publish to the right channel of REDIS.

Correct

So several users using the same REDIS channel can communicate using this architecture.

Am I right?

Correct

So each user would use one cosocket to send and retrieve messages.

I don't know well the light threads and I am wondering when the light thread will be dead?

As you can see that's an endless loop, usually you'll want to check for a close frame from the client by checking the type of the received frame and then break the loop which will effectively terminate the light thread.

Last question:

Currently the ability to use websockets for full duplex communication is only possible using the ua-nginx-module from https://github.com/blablacio/lua-nginx-module?

Not sure if this is the only repo, but as I wanted to use the patches (4 months old) with newest openresty code, I had to merge these patches manually to the upstream master.

You can merge aviramc/lua-nginx-module/tree/socket-changes into the upstream master manually if you want, that's all I did in my fork actually.

Why the patches are not included in the official lua-nginx-module?

The author does not have the time to review the patches and I guess he prefers to keep the bundle relatively stable.

Jérôme

Le jeudi 17 avril 2014 11:06:33 UTC+2, Vladislav Manchev a écrit :

Take a look at the code here:
https://github.com/agentzh/lua-resty-websocket/issues/1#issuecomment-24816008

I would say this is the way to go, not tunneling two websockets as you suggest.

For this code to work you'll need to use patches in this fork: https://github.com/aviramc/lua-nginx-module/tree/socket-changes

Or use the lua-nginx-module directly from my repo which has the socket-changes branch merged into the newest master: https://github.com/blablacio/lua-nginx-module

PS: there are many solutions similar to Faye which Famaref pointed out, but not quite sure how well that will match your needs, maybe it's worth investigating.

On 17.04.2014 11:42, Jerome Lafon wrote:

This could be a solution but if I can use only openresty with the non blocking websocket module and PHP without an other server nearby I would prefer. And the communication would be only between 2 users for each chat, not really a broadcast for several users.
Thanks for this track I keep it in my mind.

Le jeudi 17 avril 2014 10:17:04 UTC+2, Femaref a écrit :

Do you actually need a websocket per user? There are several libraries implementing a pub/sub queue system, where you'd be able to get along with one websocket endpoint in total (at least at the beginning), but would have to deal with authentication.

One such library is Faye: http://faye.jcoglan.com/

.

gg.lafon

Thanks a lot for your help.

Every point is clear now.

Can I ask you a last question:

With a lot of connected users at the same time, let's say 2000 for example, is it better in term of memory/CPU usage and performance to use 2000 websockets at the same time or AJAX polling every 5 seconds for each user (this is an acceptable latency time) or long polling for each user?

Jérôme

Le jeudi 17 avril 2014 15:45:20 UTC+2, Vladislav Manchev a écrit :

On 17.04.2014 16:33, Jerome Lafon wrote:

Traduire le message en français

I read the code provided by bungle for the chat server and it seems to be a great idea using the pub/sub mechanism of redis with the websockets.
By using for each new chat a new REDIS channel several chats could be created at the same time.

If I understand correctly this code,

- a light thread is created in order receive the new messages from REDIS via the subscribe mechanism and to send these new messages to the browser using the websocket.

Correct

- The main loop while is waiting for new messages from the browser via the websocket and publish to the right channel of REDIS.

Correct

So several users using the same REDIS channel can communicate using this architecture.

Am I right?

Correct

So each user would use one cosocket to send and retrieve messages.

I don't know well the light threads and I am wondering when the light thread will be dead?

As you can see that's an endless loop, usually you'll want to check for a close frame from the client by checking the type of the received frame and then break the loop which will effectively terminate the light thread.

Last question:

Currently the ability to use websockets for full duplex communication is only possible using the ua-nginx-module from https://github.com/blablacio/lua-nginx-module?

Not sure if this is the only repo, but as I wanted to use the patches (4 months old) with newest openresty code, I had to merge these patches manually to the upstream master.

You can merge aviramc/lua-nginx-module/tree/socket-changes into the upstream master manually if you want, that's all I did in my fork actually.

Why the patches are not included in the official lua-nginx-module?

The author does not have the time to review the patches and I guess he prefers to keep the bundle relatively stable.

Jérôme

Le jeudi 17 avril 2014 11:06:33 UTC+2, Vladislav Manchev a écrit :

Take a look at the code here:
https://github.com/agentzh/lua-resty-websocket/issues/1#issuecomment-24816008

I would say this is the way to go, not tunneling two websockets as you suggest.

For this code to work you'll need to use patches in this fork: https://github.com/aviramc/lua-nginx-module/tree/socket-changes

Or use the lua-nginx-module directly from my repo which has the socket-changes branch merged into the newest master: https://github.com/blablacio/lua-nginx-module

PS: there are many solutions similar to Faye which Famaref pointed out, but not quite sure how well that will match your needs, maybe it's worth investigating.

On 17.04.2014 11:42, Jerome Lafon wrote:

This could be a solution but if I can use only openresty with the non blocking websocket module and PHP without an other server nearby I would prefer. And the communication would be only between 2 users for each chat, not really a broadcast for several users.
Thanks for this track I keep it in my mind.

Le jeudi 17 avril 2014 10:17:04 UTC+2, Femaref a écrit :

Do you actually need a websocket per user? There are several libraries implementing a pub/sub queue system, where you'd be able to get along with one websocket endpoint in total (at least at the beginning), but would have to deal with authentication.

One such library is Faye: http://faye.jcoglan.com/

.

manchev

Well, as always it depends on your use case, but for a chat server I would definitely use websockets.

Think about it - these are two different paradigms, so the two solutions would be very different.

In the case of websockets, you'll usually want to have some callback or pub/sub implementation and only send out data to clients when it arrives on the server.

In the case of AJAX you'll need to initiate and close a connection every time you want to check for data and usually do some possibly expensive calls in the back end - i.e. database queries to determine if there's new data, etc.

Long polling should be a better solution than AJAX as there won't be connection overhead, but still you'll have to do some possibly expensive calls in the back end if not using some kind of pub/sub.

I think you should test how these options work out for your case and decide.

PS: you might be interested in WAMP too for your scenario - http://wamp.ws

On 17.04.2014 17:34, Jerome Lafon wrote:

Thanks a lot for your help.
Every point is clear now.

Can I ask you a last question:

With a lot of connected users at the same time, let's say 2000 for example, is it better in term of memory/CPU usage and performance to use 2000 websockets at the same time or AJAX polling every 5 seconds for each user (this is an acceptable latency time) or long polling for each user?

Jérôme

Le jeudi 17 avril 2014 15:45:20 UTC+2, Vladislav Manchev a écrit :

On 17.04.2014 16:33, Jerome Lafon wrote:

Traduire le message en français

I read the code provided by bungle for the chat server and it seems to be a great idea using the pub/sub mechanism of redis with the websockets.
By using for each new chat a new REDIS channel several chats could be created at the same time.

If I understand correctly this code,

- a light thread is created in order receive the new messages from REDIS via the subscribe mechanism and to send these new messages to the browser using the websocket.

Correct

- The main loop while is waiting for new messages from the browser via the websocket and publish to the right channel of REDIS.

Correct

So several users using the same REDIS channel can communicate using this architecture.

Am I right?

Correct

So each user would use one cosocket to send and retrieve messages.

I don't know well the light threads and I am wondering when the light thread will be dead?

As you can see that's an endless loop, usually you'll want to check for a close frame from the client by checking the type of the received frame and then break the loop which will effectively terminate the light thread.

Last question:

Currently the ability to use websockets for full duplex communication is only possible using the ua-nginx-module from https://github.com/blablacio/lua-nginx-module?

Not sure if this is the only repo, but as I wanted to use the patches (4 months old) with newest openresty code, I had to merge these patches manually to the upstream master.

You can merge aviramc/lua-nginx-module/tree/socket-changes into the upstream master manually if you want, that's all I did in my fork actually.

Why the patches are not included in the official lua-nginx-module?

The author does not have the time to review the patches and I guess he prefers to keep the bundle relatively stable.

Jérôme

Le jeudi 17 avril 2014 11:06:33 UTC+2, Vladislav Manchev a écrit :

Take a look at the code here:
https://github.com/agentzh/lua-resty-websocket/issues/1#issuecomment-24816008

I would say this is the way to go, not tunneling two websockets as you suggest.

For this code to work you'll need to use patches in this fork: https://github.com/aviramc/lua-nginx-module/tree/socket-changes

Or use the lua-nginx-module directly from my repo which has the socket-changes branch merged into the newest master: https://github.com/blablacio/lua-nginx-module

PS: there are many solutions similar to Faye which Famaref pointed out, but not quite sure how well that will match your needs, maybe it's worth investigating.

On 17.04.2014 11:42, Jerome Lafon wrote:

This could be a solution but if I can use only openresty with the non blocking websocket module and PHP without an other server nearby I would prefer. And the communication would be only between 2 users for each chat, not really a broadcast for several users.
Thanks for this track I keep it in my mind.

Le jeudi 17 avril 2014 10:17:04 UTC+2, Femaref a écrit :

Do you actually need a websocket per user? There are several libraries implementing a pub/sub queue system, where you'd be able to get along with one websocket endpoint in total (at least at the beginning), but would have to deal with authentication.

One such library is Faye: http://faye.jcoglan.com/

.

gg.lafon

Thanks a lot for all your explanations.

Regards

Jerome

Le vendredi 18 avril 2014 17:31:24 UTC+2, Vladislav Manchev a écrit :

Well, as always it depends on your use case, but for a chat server I would definitely use websockets.

Think about it - these are two different paradigms, so the two solutions would be very different.

In the case of websockets, you'll usually want to have some callback or pub/sub implementation and only send out data to clients when it arrives on the server.

In the case of AJAX you'll need to initiate and close a connection every time you want to check for data and usually do some possibly expensive calls in the back end - i.e. database queries to determine if there's new data, etc.

Long polling should be a better solution than AJAX as there won't be connection overhead, but still you'll have to do some possibly expensive calls in the back end if not using some kind of pub/sub.

I think you should test how these options work out for your case and decide.

PS: you might be interested in WAMP too for your scenario - http://wamp.ws

On 17.04.2014 17:34, Jerome Lafon wrote:

Thanks a lot for your help.
Every point is clear now.

Can I ask you a last question:

With a lot of connected users at the same time, let's say 2000 for example, is it better in term of memory/CPU usage and performance to use 2000 websockets at the same time or AJAX polling every 5 seconds for each user (this is an acceptable latency time) or long polling for each user?

Jérôme

Le jeudi 17 avril 2014 15:45:20 UTC+2, Vladislav Manchev a écrit :

On 17.04.2014 16:33, Jerome Lafon wrote:

Traduire le message en français

I read the code provided by bungle for the chat server and it seems to be a great idea using the pub/sub mechanism of redis with the websockets.
By using for each new chat a new REDIS channel several chats could be created at the same time.

If I understand correctly this code,

- a light thread is created in order receive the new messages from REDIS via the subscribe mechanism and to send these new messages to the browser using the websocket.

Correct

- The main loop while is waiting for new messages from the browser via the websocket and publish to the right channel of REDIS.

Correct

So several users using the same REDIS channel can communicate using this architecture.

Am I right?

Correct

So each user would use one cosocket to send and retrieve messages.

I don't know well the light threads and I am wondering when the light thread will be dead?

As you can see that's an endless loop, usually you'll want to check for a close frame from the client by checking the type of the received frame and then break the loop which will effectively terminate the light thread.

Last question:

Currently the ability to use websockets for full duplex communication is only possible using the ua-nginx-module from https://github.com/blablacio/lua-nginx-module?

Not sure if this is the only repo, but as I wanted to use the patches (4 months old) with newest openresty code, I had to merge these patches manually to the upstream master.

You can merge aviramc/lua-nginx-module/tree/socket-changes into the upstream master manually if you want, that's all I did in my fork actually.

Why the patches are not included in the official lua-nginx-module?

The author does not have the time to review the patches and I guess he prefers to keep the bundle relatively stable.

Jérôme

Le jeudi 17 avril 2014 11:06:33 UTC+2, Vladislav Manchev a écrit :

Take a look at the code here:
https://github.com/agentzh/lua-resty-websocket/issues/1#issuecomment-24816008

I would say this is the way to go, not tunneling two websockets as you suggest.

For this code to work you'll need to use patches in this fork: https://github.com/aviramc/lua-nginx-module/tree/socket-changes

Or use the lua-nginx-module directly from my repo which has the socket-changes branch merged into the newest master: https://github.com/blablacio/lua-nginx-module

PS: there are many solutions similar to Faye which Famaref pointed out, but not quite sure how well that will match your needs, maybe it's worth investigating.

On 17.04.2014 11:42, Jerome Lafon wrote:

This could be a solution but if I can use only openresty with the non blocking websocket module and PHP without an other server nearby I would prefer. And the communication would be only between 2 users for each chat, not really a broadcast for several users.
Thanks for this track I keep it in my mind.

Le jeudi 17 avril 2014 10:17:04 UTC+2, Femaref a écrit :

Do you actually need a websocket per user? There are several libraries implementing a pub/sub queue system, where you'd be able to get along with one websocket endpoint in total (at least at the beginning), but would have to deal with authentication.

One such library is Faye: http://faye.jcoglan.com/

.

gg.lafon

Hello,
I tried to compile the latest version of Openresty with the lua-nginx-module from https://github.com/blablacio/lua-nginx-module
But I have an error running resty postgres module:
resty/postgres.lua:183: bad argument #3 to 'connect' (bad\
"ssl" option type: nil)
The line 183 in postgres.lua is: ok, err = sock:connect(host, port, { pool = pool })

Could you help me please?

Thank you
Jérôme

Le vendredi 18 avril 2014 17:31:24 UTC+2, Vladislav Manchev a écrit :

Well, as always it depends on your use case, but for a chat server I would definitely use websockets.

Think about it - these are two different paradigms, so the two solutions would be very different.

In the case of websockets, you'll usually want to have some callback or pub/sub implementation and only send out data to clients when it arrives on the server.

In the case of AJAX you'll need to initiate and close a connection every time you want to check for data and usually do some possibly expensive calls in the back end - i.e. database queries to determine if there's new data, etc.

Long polling should be a better solution than AJAX as there won't be connection overhead, but still you'll have to do some possibly expensive calls in the back end if not using some kind of pub/sub.

I think you should test how these options work out for your case and decide.

PS: you might be interested in WAMP too for your scenario - http://wamp.ws

On 17.04.2014 17:34, Jerome Lafon wrote:

Thanks a lot for your help.
Every point is clear now.

Can I ask you a last question:

With a lot of connected users at the same time, let's say 2000 for example, is it better in term of memory/CPU usage and performance to use 2000 websockets at the same time or AJAX polling every 5 seconds for each user (this is an acceptable latency time) or long polling for each user?

Jérôme

Le jeudi 17 avril 2014 15:45:20 UTC+2, Vladislav Manchev a écrit :

On 17.04.2014 16:33, Jerome Lafon wrote:

Traduire le message en français

I read the code provided by bungle for the chat server and it seems to be a great idea using the pub/sub mechanism of redis with the websockets.
By using for each new chat a new REDIS channel several chats could be created at the same time.

If I understand correctly this code,

- a light thread is created in order receive the new messages from REDIS via the subscribe mechanism and to send these new messages to the browser using the websocket.

Correct

- The main loop while is waiting for new messages from the browser via the websocket and publish to the right channel of REDIS.

Correct

So several users using the same REDIS channel can communicate using this architecture.

Am I right?

Correct

So each user would use one cosocket to send and retrieve messages.

I don't know well the light threads and I am wondering when the light thread will be dead?

As you can see that's an endless loop, usually you'll want to check for a close frame from the client by checking the type of the received frame and then break the loop which will effectively terminate the light thread.

Last question:

Currently the ability to use websockets for full duplex communication is only possible using the ua-nginx-module from https://github.com/blablacio/lua-nginx-module?

Not sure if this is the only repo, but as I wanted to use the patches (4 months old) with newest openresty code, I had to merge these patches manually to the upstream master.

You can merge aviramc/lua-nginx-module/tree/socket-changes into the upstream master manually if you want, that's all I did in my fork actually.

Why the patches are not included in the official lua-nginx-module?

The author does not have the time to review the patches and I guess he prefers to keep the bundle relatively stable.

Jérôme

Le jeudi 17 avril 2014 11:06:33 UTC+2, Vladislav Manchev a écrit :

Take a look at the code here:
https://github.com/agentzh/lua-resty-websocket/issues/1#issuecomment-24816008

I would say this is the way to go, not tunneling two websockets as you suggest.

For this code to work you'll need to use patches in this fork: https://github.com/aviramc/lua-nginx-module/tree/socket-changes

Or use the lua-nginx-module directly from my repo which has the socket-changes branch merged into the newest master: https://github.com/blablacio/lua-nginx-module

PS: there are many solutions similar to Faye which Famaref pointed out, but not quite sure how well that will match your needs, maybe it's worth investigating.

On 17.04.2014 11:42, Jerome Lafon wrote:

This could be a solution but if I can use only openresty with the non blocking websocket module and PHP without an other server nearby I would prefer. And the communication would be only between 2 users for each chat, not really a broadcast for several users.
Thanks for this track I keep it in my mind.

Le jeudi 17 avril 2014 10:17:04 UTC+2, Femaref a écrit :

Do you actually need a websocket per user? There are several libraries implementing a pub/sub queue system, where you'd be able to get along with one websocket endpoint in total (at least at the beginning), but would have to deal with authentication.

One such library is Faye: http://faye.jcoglan.com/

.