Re: lua memcached / couchbase

agentzh · 2014-04-05T03:21:44+00:00

web: https://testssl.sh/ repo: https://bitbucket.org/nginx-goodies/testssl.sh testssl.sh is a free Unix command line tool which checks a server's service on...

Re: lua memcached / couchbase

agentzh

Hello!

On Fri, Apr 4, 2014 at 7:59 AM, kay wrote:
> I'm trying to use https://github.com/chrhlnd/lualibcbase with lua module
>
> Actually I already got results, but I have few questions:
>
> * there are two types of lua binding library: sync and async. which
> should I use?

Neither. This library will always block the NGINX worker process
because it knows nothing about ngx_lua's cosocket API:

    https://github.com/chaoslawful/lua-nginx-module#ngxsockettcp

Because the cosocket API is mostly compatibilty to LuaSocket's API,
existing Lua libraries based on the LuaSocket API can be made
nonblocking in openresty with minor adjustments.

> * where should I create a connection? In init_by_lua? Or where? I
> should not create new connection for each request.
>

Nonblocking cosocket objects in openresty can only be per-request.
Though you can share and reuse the underlying TCP connections via the
cosocket pool mechanism:

    https://github.com/chaoslawful/lua-nginx-module#tcpsocksetkeepalive

I'm cc'ing the openresty-en mailing list:
https://groups.google.com/group/openresty-en And some people there
might know a read-to-use nonblocking Lua driver for openresty +
couchbase. You're also recommended to join the list and ask such
questions there.

I'm not sure if couchbase supports the memcached wire protocol. If
yes, then you can use the standard lua-resty-memcached library
directly:

    https://github.com/agentzh/lua-resty-memcached#readme

Regards,
-agentzh

kay.diam

Thank you very much for detailed answer!

I'll try to modify this library as you suggested.

As for memcached, yes, couchbase supports memcached protocol, but with
moxi software which is not ready for production. That is why I decided
to use pure couchbase client.

As for the nginx-libcocubase library
(https://github.com/couchbaselabs/couchbase-nginx-module), it is very
buggy. I guess developers still didn't manage to implement
libcouchbase library with nginx event model.

Regards!

2014-04-04 23:21 GMT+04:00 Yichun Zhang (agentzh) <age...@gmail.com>:
> Hello!
>
> On Fri, Apr 4, 2014 at 7:59 AM, kay wrote:
>> I'm trying to use https://github.com/chrhlnd/lualibcbase with lua module
>>
>> Actually I already got results, but I have few questions:
>>
>> * there are two types of lua binding library: sync and async. which
>> should I use?
>
> Neither. This library will always block the NGINX worker process
> because it knows nothing about ngx_lua's cosocket API:
>
>     https://github.com/chaoslawful/lua-nginx-module#ngxsockettcp
>
> Because the cosocket API is mostly compatibilty to LuaSocket's API,
> existing Lua libraries based on the LuaSocket API can be made
> nonblocking in openresty with minor adjustments.
>
>> * where should I create a connection? In init_by_lua? Or where? I
>> should not create new connection for each request.
>>
>
> Nonblocking cosocket objects in openresty can only be per-request.
> Though you can share and reuse the underlying TCP connections via the
> cosocket pool mechanism:
>
>     https://github.com/chaoslawful/lua-nginx-module#tcpsocksetkeepalive
>
> I'm cc'ing the openresty-en mailing list:
> https://groups.google.com/group/openresty-en And some people there
> might know a read-to-use nonblocking Lua driver for openresty +
> couchbase. You're also recommended to join the list and ask such
> questions there.
>
> I'm not sure if couchbase supports the memcached wire protocol. If
> yes, then you can use the standard lua-resty-memcached library
> directly:
>
>     https://github.com/agentzh/lua-resty-memcached#readme
>
> Regards,
> -agentzh

agentzh

Hello!

On Fri, Apr 4, 2014 at 12:36 PM, kay wrote:
> As for the nginx-libcocubase library
> (https://github.com/couchbaselabs/couchbase-nginx-module), it is very
> buggy. I guess developers still didn't manage to implement
> libcouchbase library with nginx event model.
>

According to my own experience, it should be much easier to implement
a client library in Lua with the ngx_lua cosocket API than to write a
proper NGINX C upstream module. The former approach will also be much
more flexible (and also much faster if you will process the result in
Lua anyway).

Good luck with your lua-resty-couchbase library :)

Regards,
-agentzh

anton.jouline

this is not an answer for your original question, but i think it's still worth mentioning:

Couchbase wire protocol is 100% compatible with Memcached - they even use the same default port: 11211, so you can use

lua-resty-memcached to talk to Couchbase, for the typical get/set/incr/etc. operations that you'd want from a key/value store.

For some newer features, such as views, you can always do an internal subrequest via http, because all of that is available via a REST-like api on port 8092.

Anton.

On Friday, April 4, 2014 2:07:20 PM UTC-7, agentzh wrote:

Hello!

On Fri, Apr 4, 2014 at 12:36 PM, kay wrote:
> As for the nginx-libcocubase library
> (https://github.com/couchbaselabs/couchbase-nginx-module), it is very
> buggy. I guess developers still didn't manage to implement
> libcouchbase library with nginx event model.
>

According to my own experience, it should be much easier to implement
a client library in Lua with the ngx_lua cosocket API than to write a
proper NGINX C upstream module. The former approach will also be much
more flexible (and also much faster if you will process the result in
Lua anyway).

Good luck with your lua-resty-couchbase library :)

Regards,
-agentzh

kay.diam

Dear Anton,

Memcached compatible protocol - it is moxi, which is additional layer. According to my tests, pure Couchbase protocol is x2 faster than moxi.

"REST-like api on port 8092" has worse test results.

понедельник, 7 апреля 2014 г., 22:18:43 UTC+4 пользователь Anton Jouline написал:

this is not an answer for your original question, but i think it's still worth mentioning:

Couchbase wire protocol is 100% compatible with Memcached - they even use the same default port: 11211, so you can use
lua-resty-memcached to talk to Couchbase, for the typical get/set/incr/etc. operations that you'd want from a key/value store.

For some newer features, such as views, you can always do an internal subrequest via http, because all of that is available via a REST-like api on port 8092.

Anton.

On Friday, April 4, 2014 2:07:20 PM UTC-7, agentzh wrote:
Hello!

On Fri, Apr 4, 2014 at 12:36 PM, kay wrote:
> As for the nginx-libcocubase library
> (https://github.com/couchbaselabs/couchbase-nginx-module), it is very
> buggy. I guess developers still didn't manage to implement
> libcouchbase library with nginx event model.
>

According to my own experience, it should be much easier to implement
a client library in Lua with the ngx_lua cosocket API than to write a
proper NGINX C upstream module. The former approach will also be much
more flexible (and also much faster if you will process the result in
Lua anyway).

Good luck with your lua-resty-couchbase library :)

Regards,
-agentzh