shdict locking woes

brian

the spinlock used for shdicts is causing me headaches. I actually get better overall performance in one of my apps by using a "hashed" set of redis servers because we don't block on network accesses. The cpu usage is much higher, but the overall throughput is better because the shdict locks cause nginx to block. I was already using a hashed set of shdicts, but I may give up and just use redis and/or memcache.

Anyone else run into similar issues?

agentzh

Hello!

On Wed, Apr 17, 2013 at 5:29 PM, Brian Akins wrote:
> the spinlock used for shdicts is causing me headaches. I actually get better
> overall performance in one of my apps by using a "hashed" set of redis
> servers because we don't block on network accesses. The cpu usage is much
> higher, but the overall throughput is better because the shdict locks cause
> nginx to block.  I was already using a hashed set of shdicts, but I may give
> up and just use redis and/or memcache.
>

It'll be great if you can share some flamegraphs for your busiest
Nginx worker processes:

    https://github.com/agentzh/nginx-systemtap-toolkit#ngx-sample-bt

And the output of the ngx-shm tool should be interesting as well:

    https://github.com/agentzh/nginx-systemtap-toolkit#ngx-shm

And I'm also going to add a tool to my Nginx Systemtap Toolkit to
directly measure the locking time for shdict in a running nginx
process.

Thanks!
-agentzh

brian

It seems our version of Linux we run in production is too old for that. I may can stand up a tests server. FWIW, I can recreate "stalled" processes easily by reducing the number of "hashed" shdicts we use.