Feedback for HMAC module

jon.keys · 2014-07-18T01:03:29+00:00

One request can never block other requests so nothing to worry about.You'll only need to (probably) block the current request while getting the result fr...

Feedback for HMAC module

jon.keys

In our search for an OpenResty HMAC module (specifically hmac-sha256) we found a few possible solutions:

https://github.com/jamesmarlowe/lua-resty-hmac
https://gist.github.com/siddontang/5603849

Unfortunately #1 doesn't appear to be LuaJIT compatible.

While #2 works well it doesn't have any tests or list a license (it is just a gist after all). It also uses the builtin module function and requires explicit resource cleanup.

So, we decided to write our own HMAC module to meet our needs: https://github.com/jkeys089/lua-resty-hmac

We've tried to learn and follow modern best practices from other modules (e.g. https://github.com/openresty/lua-resty-string) but we're still very new to OpenResty and Lua so we're asking for feedback on this module.

All comments / feedback / criticism / pull requests are welcome.

Thanks!

ddragosd

FYI, there's an open pull request to support HMAC-SHA256 and other HMAC-SHA* algorithms in lua-resty-string module.

Until it gets merged, you can use the hmac.lua directly if you want:

https://github.com/ddragosd/lua-resty-string/blob/b0d7918619d8f3de3dc2304be79c7e7faae8ab61/lib/resty/hmac.lua

And for the usage: https://github.com/ddragosd/lua-resty-string/commit/126006ab1d10fa4172f180a3777f2d9845f26cb3

Hoping this helps,

dragos

On Thursday, July 17, 2014 10:03:28 AM UTC-7, Jon Keys wrote:

In our search for an OpenResty HMAC module (specifically hmac-sha256) we found a few possible solutions:
https://github.com/jamesmarlowe/lua-resty-hmac
https://gist.github.com/siddontang/5603849
Unfortunately #1 doesn't appear to be LuaJIT compatible.
While #2 works well it doesn't have any tests or list a license (it is just a gist after all). It also uses the builtin module function and requires explicit resource cleanup.

So, we decided to write our own HMAC module to meet our needs: https://github.com/jkeys089/lua-resty-hmac

We've tried to learn and follow modern best practices from other modules (e.g. https://github.com/openresty/lua-resty-string) but we're still very new to OpenResty and Lua so we're asking for feedback on this module.
All comments / feedback / criticism / pull requests are welcome.

Thanks!

jon.keys

FYI, there's an open pull request to support HMAC-SHA256 and other HMAC-SHA* algorithms in lua-resty-string module.

Nice! Thanks for the info.

Until it gets merged, you can use the hmac.lua directly if you want:
https://github.com/ddragosd/lua-resty-string/blob/b0d7918619d8f3de3dc2304be79c7e7faae8ab61/lib/resty/hmac.lua
And for the usage: https://github.com/ddragosd/lua-resty-string/commit/126006ab1d10fa4172f180a3777f2d9845f26cb3

I took a look and overall it looks very good!

Unfortunately, for our needs I don't think it is optimal since it doesn't allow for re-use of the hmac context and doesn't allow the MAC calculation to be updated. I left a comment about this on the PR.

I appreciate the info. It is good to see our approach is very similar to yours and we're not way off track :-)

Thanks!

aapo.talvensaari

On Thursday, July 17, 2014 8:03:28 PM UTC+3, Jon Keys wrote:

In our search for an OpenResty HMAC module (specifically hmac-sha256) we found a few possible solutions:
https://github.com/jamesmarlowe/lua-resty-hmac
https://gist.github.com/siddontang/5603849
Unfortunately #1 doesn't appear to be LuaJIT compatible.
While #2 works well it doesn't have any tests or list a license (it is just a gist after all). It also uses the builtin module function and requires explicit resource cleanup.

So, we decided to write our own HMAC module to meet our needs: https://github.com/jkeys089/lua-resty-hmac

Great! OpenSSL is usually preferred as it ships with Nginx by default. OpenSSL API is quite horrible though. I have been writing bindings with LuaJIT FFI for Nettle (http://www.lysator.liu.se/~nisse/nettle/) that has following HMAC implementations: md5, sha1, sha224, sha256, sha384, sha512, and ripemd160. Nettle is documented here:

http://www.lysator.liu.se/~nisse/nettle/nettle.html

I have already written bindings for everything else except public-key algorithms (described here: http://www.lysator.liu.se/~nisse/nettle/nettle.html#Public_002dkey-algorithms).

My hmac bindings are here:

https://github.com/bungle/lua-resty-nettle/blob/master/lib/resty/nettle/hmac.lua

Others can be found from here (all the bindings there is, do work):

https://github.com/bungle/lua-resty-nettle/tree/master/lib/resty/nettle

The library is not yet documented but it works fine and is usable. You need to download and compile Nettle and place it in package.cpath (something you don't need to do with OpenSSL).

Regards

Aapo

jon.keys

On Tuesday, July 22, 2014 12:29:09 PM UTC-4, Aapo Talvensaari wrote:

On Thursday, July 17, 2014 8:03:28 PM UTC+3, Jon Keys wrote:
In our search for an OpenResty HMAC module (specifically hmac-sha256) we found a few possible solutions:
https://github.com/jamesmarlowe/lua-resty-hmac
https://gist.github.com/siddontang/5603849
Unfortunately #1 doesn't appear to be LuaJIT compatible.
While #2 works well it doesn't have any tests or list a license (it is just a gist after all). It also uses the builtin module function and requires explicit resource cleanup.

So, we decided to write our own HMAC module to meet our needs: https://github.com/jkeys089/lua-resty-hmac

Great! OpenSSL is usually preferred as it ships with Nginx by default. OpenSSL API is quite horrible though. I have been writing bindings with LuaJIT FFI for Nettle (http://www.lysator.liu.se/~nisse/nettle/) that has following HMAC implementations: md5, sha1, sha224, sha256, sha384, sha512, and ripemd160. Nettle is documented here:
http://www.lysator.liu.se/~nisse/nettle/nettle.html

Yes, the OpenSSL API is not great and the HMAC API changed fairly recently (v1.0.0).

I've not heard of Nettle before. It looks really nice!

Is there any reason to choose Nettle over OpenSSL aside from a nicer / more stable API? I did a quick google search and didn't find anything recent that compares the two.

I have already written bindings for everything else except public-key algorithms (described here: http://www.lysator.liu.se/~nisse/nettle/nettle.html#Public_002dkey-algorithms).

My hmac bindings are here:
https://github.com/bungle/lua-resty-nettle/blob/master/lib/resty/nettle/hmac.lua

Others can be found from here (all the bindings there is, do work):
https://github.com/bungle/lua-resty-nettle/tree/master/lib/resty/nettle

The library is not yet documented but it works fine and is usable. You need to download and compile Nettle and place it in package.cpath (something you don't need to do with OpenSSL).

I'll definitely check it out. Thanks for sharing!

aapo.talvensaari

On Tuesday, July 22, 2014 7:45:41 PM UTC+3, Jon Keys wrote:

I've not heard of Nettle before. It looks really nice!
Is there any reason to choose Nettle over OpenSSL aside from a nicer / more stable API?

I hadn't heard about it until this June when I saw annoucement of the version 3.0 at Phoronix. I looked it quickly, and it looked a very straight forward to implement LuaJIT bindings to it, so that's what I ended up doing. I think that it is a default crypto library in GNU projects (like GNUTLS). Its performance is comparable to OpenSSLs. The code is cleanly laid out, and easy to follow.

No particular reason. A great learning experience. I also wanted to have nice AES/Camellia libraries with different modes like (ECB, CBC, CTR, EAX, GCM, and CCM), and PBKDF2 lib. It's been a lot of fun too. Piece by piece I ended up implementing a nice Lua API for Nettle.

Lua doesn't come batteries included, and while OpenResty has some batteries, the more building blocks are needed. I like this OpenResty project, so as a hobby I end up implementing this and that to support the project, and make it better for web development. There are common libs that you usually need when developing web apps:

- crypto

- image manipulation / conversion

- movie transcoding

- authentication (+ extenal authentication like openid, google, ms, facebook, twitter, github etc. logins, and app-to-app auth like oauth)

- session management

- cookie handling

- localization

- access to backend systems like dbs, caches, and search services

- excel / pdf report generation

- html templating

- input validation

- email/sms handling

- string manipulation, encodings

- a lot of saas services to bind to

- ...

I just tend to pick something that I think at this phase I will need. Right now I feel the core of OpenResty is quite alright, and there are libs

already for many things (thanks to agentz and others).

I like the way how OpenResty is constructed. It uses Nginx cleverly where it fits. It's not an opionated framework, and more like a nice Lua library layer on top of Nginx. Loosely coupled small libraries is a way to go, in my opinion (I think it is better to copy paste small amounts of code than it is to create dependencies between libs - not to mention fullstack opionated frameworks where all your code belongs to framework). I think what OpenResty needs is just more libs (for everything) and Luafied APIs for FFI-bindings, and a package management for them (something agentz has been talking about).

So lua-resty-nettle is just a one component to have (and I'm pretty sure some others will find it useful as well).

Regards

Aapo