We used to enhance our API latency by openresty resty-redis, and result is such good. (150000+ TPS, which based on with 3 redis instances, haproxy+sentinel master/slave mode + single openresty, 24 core cpu). But redis has capacity issue and hard to scale out, so we are going to migrate to redis cluster.
Currently redis has no official redis cluster client. We have looked over and compared some 3rd party clients, and we found
https://github.com/cuiweixie/lua-resty-redis-cluster is mostly close to our requirement. This client has great ideas for caching slot->node mapping with FFI,
pipelining. But, it still has some issues and we cannot use at production yet:
1. While redis cluster is resharding slots, smart client should support both MOVED and ASK redirection. (please refer to
https://redis.io/topics/cluster-spec, Redirection and resharding part)
2. Correspondingly, pipeline should also support both MOVED and ASK redirection.
3. slotcache will cache the mapping between 16384 slots and real node, so the cache volume is big. While we are getting high throughputs, for cosockets will suspend current coroutine and switch other coroutine in, and if coroutine still has reference with the slotcache, then with multiple(maybe 200) coroutine switch, we will get Lua VM crashed, reason: not enough memory error.
4. While in the case node is down/cluster is down, we need to refresh slot cahce.
1. read from slave node. Basically by default only master node can be read, but in low consistency scenario we can also make use of slave nodes to read. (readonly mode)
Feel free to give any suggestions!
Link is: https://github.com/steve0511/resty-redis-cluster
