[ANN] lua-resty-jit-uuid

thibault.angelo · 2016-04-21T10:02:30+00:00

I will look into it, thanks!On Wednesday, April 27, 2016 at 2:57:14 AM UTC+3, agentzh wrote:Hello! On Mon, Apr 25, 2016 at 5:57 AM, <al...@chamel...

[ANN] lua-resty-jit-uuid

thibault.angelo

Hi everyone,

A few weeks ago I started playing with the UUID libraries available out

there for Lua/LuaJIT and benchmarking them. I was convinced that there

was a way to efficiently generate UUIDs with LuaJIT without being

dependent on libuuid like the C and FFI bindings.

What started as a LuaJIT script to generate UUIDs became lua-resty-

jit-uuid:

https://github.com/thibaultCha/lua-resty-jit-uuid

According to the benchmarks I ran, it appears to be more efficient than the

FFI and C bindings. It uses math.random() (which as we know is enhanced

by LuaJIT) and the BitOp module, and is optimized for use in ngx_lua.

As a bonus it also provides UUID validation and generation of v3 UUIDs

(name-based).

I would love for someone to give feedback, suggest improvements or even

prove the benchmarks wrong!

Regards,
Thibault

aapo.talvensaari

On Thursday, 21 April 2016 05:02:30 UTC+3, Thibault Charbonnier wrote:

What started as a LuaJIT script to generate UUIDs became lua-resty-
jit-uuid:

https://github.com/thibaultCha/lua-resty-jit-uuid

According to the benchmarks I ran, it appears to be more efficient than the
FFI and C bindings. It uses math.random() (which as we know is enhanced
by LuaJIT) and the BitOp module, and is optimized for use in ngx_lua.

Thanks Thibault,

I can confirm your benchmarks, your lib is in general faster than everything else.

libuuid supports 3 different algorightms. On my tests it seems that your lib is

faster in generation except with generate_time function where your lib and

the FFI version are about the same (I modified the bench code to use

generate_time with FFI binding instead, here are results from my mac):

LuaJIT 2.1.0-beta1 with 1e+06 UUIDs
UUID v4 (random) generation
1. FFI binding took: 0.100180s -1%
2. resty-jit-uuid took: 0.101588s +0%

I also changed the order a bit of that bench table, and that changed a results a bit

(I think they are still inside the error margin):

1. FFI binding took: 0.100896s -10%
2. resty-jit-uuid took: 0.113080s +0%

Great work indeed, maybe I should add this as a fallback to lua-resty-uuid for

Windows platforms where the libuuid dependency is usually a problem.

Thanks for contributing this!

thibault.angelo

Hey Aapo,

Thank you for taking the time to try it out and for playing with thebenchmarks. I am glad you were able to confirm them.


> faster in generation except with generate_time function

I believe generation of v1 UUIDs would be faster than v4 indeed. I willevaluate implementing v1 UUID generation as well in resty-jit-uuid, I amnot yet sure about the feasibility/efficiency of it.

> I also changed the order a bit of that bench table, and that changed> a results a bit

Interesting, I will give this a try and maybe improve the benchmarkingscript.


> maybe I should add this as a fallback to lua-resty-uuid for
> Windows platforms where the libuuid dependency is usually a problem.

That is a good use-case indeed, precisely one of those I had in mind :)

Regards,
Thibault

On 4/21/16 5:53 AM, Aapo Talvensaari wrote:

On Thursday, 21 April 2016 05:02:30 UTC+3, Thibault Charbonnier wrote:

    What started as a LuaJIT script to generate UUIDs became lua-resty-
    jit-uuid:

    https://github.com/thibaultCha/lua-resty-jit-uuid
    <https://github.com/thibaultCha/lua-resty-jit-uuid>

    According to the benchmarks I ran, it appears to be more efficient
    than the
    FFI and C bindings. It uses math.random() (which as we know is enhanced
      by LuaJIT) and the BitOp module, and is optimized for use in ngx_lua.


Thanks Thibault,

I can confirm your benchmarks, your lib is in general faster than
everything else.
libuuid supports 3 different algorightms. On my tests it seems that your
lib is
faster in generation except with generate_time function where your lib and
the FFI version are about the same (I modified the bench code to use
generate_time with FFI binding instead, here are results from my mac):

LuaJIT 2.1.0-beta1 with 1e+06 UUIDs
UUID v4 (random) generation
1. FFI binding took: 0.100180s -1%
2. resty-jit-uuid took: 0.101588s +0%

I also changed the order a bit of that bench table, and that changed a
results a bit
(I think they are still inside the error margin):

1. FFI binding took: 0.100896s -10%
2. resty-jit-uuid took: 0.113080s +0%

Great work indeed, maybe I should add this as a fallback to
lua-resty-uuid for
Windows platforms where the libuuid dependency is usually a problem.

Thanks for contributing this!.

aapo.talvensaari

On Friday, 22 April 2016 01:44:27 UTC+3, Thibault Charbonnier wrote:

Interesting, I will give this a try and maybe improve the benchmarking
script.

There could be a lot of reasons, e.g. GC kicking in, or memory allocation problems etc. But well, I think the results are still clear. If you really want to make it better, you maybe should do it something like this:

1. have benchmarks on different libs in different scripts, and run them independently

2. warm up the code before benchmarking, so that LuaJIT etc. can kick in

> maybe I should add this as a fallback to lua-resty-uuid for
> Windows platforms where the libuuid dependency is usually a problem.

That is a good use-case indeed, precisely one of those I had in mind :)

I will look forward to adding that. Do you think I should supply lua-resty-jit-uuid in my project as well, or just to check for a fallback (e.g. pcall require)? I think I will keep it as a fallback to maintain default compability with libuuid. I think libuuid also takes in account network card's MAC address, and it uses /dev/urandom as a default random provider.

Regards

Aapo

thibault.angelo

 > Do you think I should supply

lua-resty-jit-uuid in my project as well, or just to check for a
fallback (e.g. pcall require)?

Yes, I would suggest you stick to libuuid if and when you can. As yousaid, libuuid generates v1 UUIDs (time + MAC address) which gives betterrandomness and overall a smaller chance of collision. v4 UUIDs, therandom ones, are designed for use cases where the MAC address is notaccessible, although the chance of a collision happening is still very low.

The main interest of this lib is its portability, when shippingOpenResty/LuaJIT software or running on Windows. But if you can affordlibuuid, I'd say it's the safest bet :)


Regards
Thibault