Surprising benchmark results

brinkema · 2016-03-26T09:34:55+00:00

cool, that was it, thanks for jogging my memory.On Fri, Mar 25, 2016 at 10:39 AM Robert Paprocki <rpap...@fearnothingproductions.net> wrote:stappxx may...

Surprising benchmark results

brinkema

Hi all,

I am new to OpenResty. To learn it I followed the directions on the OpenResty web page including setting up and running the http_load benchmark. The"surprising thing" is that the results I got are about 50% *SLOWER* than angentzh reported. Angentzh ran his test via localhost on a Levono T400; I ran my test on a desktop with a modern AMD processor. *I know that this is a garbage comparison*, but: the T400 runs about 4800 bogoMIPS; the CPU I use runs at about 7800 bogoMIPS. I assumed that the nginx server in the benchmark configuration would be almost 100% computer-bound and thus my results should be greater. ???

The http_load results from angentzh are 27923.9 fetches/sec, 335087 bytes/sec

My results are 13475.1 fetches/sec, 256028 bytes/sec

Both were done on the localhost although I get similar results via a network.

Any idea of what is going on? What http_load benchmarks results have you gotten?

/jb

agentzh

Hello!

On Fri, Mar 25, 2016 at 6:34 PM,  brinkema wrote:
> I am new to OpenResty.  To learn it I followed the directions on the
> OpenResty web page including setting up and running the http_load benchmark.
> The"surprising thing" is that the results I got are about 50% *SLOWER* than
> angentzh reported.  Angentzh ran his test via localhost on a Levono T400;  I
> ran my test on a desktop with a modern AMD processor. *I know that this is a
> garbage comparison*, but: the T400 runs about 4800 bogoMIPS; the CPU I use
> runs at about 7800 bogoMIPS.  I assumed that the nginx server in the
> benchmark configuration would be almost 100% computer-bound and thus my
> results should be greater.  ???
>

Please do not capitalize my nick. Also my nick is agentzh. Please do
not reinvent my nick. Thank you.

While doing benchmark, ensure you have a spare CPU core for your
client tool itself. Also observe the CPU usage of each relevant
processes in tools like top. If nginx workers cannot max out the CPU,
then you may have some problems or misconfigurations. If your nginx
workers are indeed at 100% CPU usage, then you can further use the
on-CPU flame graph tools to analyze if there's anything you can
improve further:

https://openresty.org/en/profiling.html

If your client tool http_load is maxing out the CPU, then you should
analyze bottlenecks in your client tool instead.

Regards,
-agentzh

brinkema

Some more information:

The benchmark was run under Fedora 23, right out of the box ... nochanges from defaults.

I ran the benchmark under Ubuntu, also out of the box, and got theexpected results. Specifically:


173628 fetches, 10 max parallel, 2.25716e+06 bytes, in 5.00002 seconds
13 mean bytes/connection
34725.5 fetches/sec, 451431 bytes/sec
msecs/connect: 0.0381365 mean, 0.336 max, 0.01 min

msecs/first-response: 0.219433 mean, 0.865 m173628 fetches, 10 maxparallel, 2.25716e+06 bytes, in 5.00002 seconds

13 mean bytes/connection
34725.5 fetches/sec, 451431 bytes/sec
msecs/connect: 0.0381365 mean, 0.336 max, 0.01 min
msecs/first-response: 0.219433 mean, 0.865 max, 0.084 min
HTTP response codes:
  code 200 -- 173628
ax, 0.084 min
HTTP response codes:
  code 200 -- 173628

Both systems are hardware similar, 6G of memory, with dual processorAMDs at 3.5GHz.

As agentzh suggested, there seems to be a configuration issue with myFedora platform. More to come.



On 03/27/2016 03:02 PM, Yichun Zhang (agentzh) wrote:

Hello!

On Fri, Mar 25, 2016 at 6:34 PM,  brinkema wrote:

I am new to OpenResty.  To learn it I followed the directions on the
OpenResty web page including setting up and running the http_load benchmark.
The"surprising thing" is that the results I got are about 50% *SLOWER* than
angentzh reported.  Angentzh ran his test via localhost on a Levono T400;  I
ran my test on a desktop with a modern AMD processor. *I know that this is a
garbage comparison*, but: the T400 runs about 4800 bogoMIPS; the CPU I use
runs at about 7800 bogoMIPS.  I assumed that the nginx server in the
benchmark configuration would be almost 100% computer-bound and thus my
results should be greater.  ???

While doing benchmark, ensure you have a spare CPU core for your
client tool itself. Also observe the CPU usage of each relevant
processes in tools like top. If nginx workers cannot max out the CPU,
then you may have some problems or misconfigurations. If your nginx
workers are indeed at 100% CPU usage, then you can further use the
on-CPU flame graph tools to analyze if there's anything you can
improve further:

https://openresty.org/en/profiling.html

If your client tool http_load is maxing out the CPU, then you should
analyze bottlenecks in your client tool instead.

Regards,
-agentzh