Resty Hmac Library

james_357f · 2014-03-20T07:53:35+00:00

@agentzhThank you very much for this, at least I now know where to focus my attention and your patch should prevent me from making the same mistake twice.Gla...

Resty Hmac Library

james_357f

I was setting up a project to connect to AWS from openresty recently and thought it would be cool to share it.

https://github.com/jamesmarlowe/lua-resty-hmac

I rewrote it to heavily follow agentzh's examples since he's the best :)

Do you have any suggestions or pointers? Also do you have an example of how to run these tests?

brian

To run the tests, you use prove -- it's a perl thing.

james_357f

Thanks for pointing me in the right direction,

I see the make test in my lib will run prove and that will understand "use Test::Nginx::Socket::Lua;" in my test file. I believe that references https://github.com/agentzh/test-nginx/tree/master/lib/Test/Nginx/Socket so I put test-nginx in my ~ dir, and ran perl Makefile.PL and then make and sudo make install.

Now when running make test in my lib, it says Can't locate Test/Base.pm ... /test-nginx/lib/Test/Nginx/Socket.pm line 6.

Wandering around cpan, I found http://search.cpan.org/~ingy/Test-Base-0.62/lib/Test/Base.pm http://search.cpan.org/~ingy/Spiffy-0.31/lib/Spiffy.pm which I wrote into the test-nginx dir so now it complains "Can't find a class for Filter at..." I assume that just copying the files in isn't the correct way to install those, can you give me another hint?

agentzh

Hello!

On Wed, Mar 19, 2014 at 4:53 PM, James Marlowe wrote:
> Do you have any suggestions or pointers? Also do you have an example of how
> to run these tests?
>

Just run the following commands:

    sudo cpan Test::Nginx   # for installing Test::Nginx's all
dependencies automatically
    mkdir -p ~/work
    cd ~/work
    git clone https://github.com/agentzh/test-nginx.git
    cd /path/to/your/lua-resty-library
    prove -I ~/work/test-nginx/lib -r t

The reason for using the git repos of Test::Nginx is that Test::Nginx
is a fast-moving project and the latest CPAN release can never catch
up ;)

Test your Test::Nginx installation with my test suites first, like the
one in lua-resty-string.

Regards,
-agentzh

james_357f

That is very helpful, I haven't done much work in perl. I tried running sudo cpan Test::Nginx but got:

Test Summary Report

-------------------

t/syntax.t (Wstat: 256 Tests: 2 Failed: 1)

Failed test: 1

Non-zero exit status: 1

Files=7, Tests=24, 1 wallclock secs ( 0.04 usr 0.05 sys + 0.87 cusr 0.26 csys = 1.22 CPU)

Result: FAIL

Failed 1/7 test programs. 1/24 subtests failed.

make: *** [test_dynamic] Error 1

AGENT/Test-Nginx-0.22.tar.gz

one dependency not OK (LWP::UserAgent); additionally test harness failed

/usr/bin/make test -- NOT OK

//hint// to see the cpan-testers results for installing this module, try:

reports AGENT/Test-Nginx-0.22.tar.gz

Running make install

make test had returned bad status, won't install without force

So then I tried sudo cpan LWP::UserAgent with the same failure

agentzh

Hello!

On Fri, Mar 21, 2014 at 8:54 AM, James Marlowe wrote:
> That is very helpful, I haven't done much work in perl. I tried running sudo
> cpan Test::Nginx but got:
>
> Test Summary Report
> -------------------
> t/syntax.t             (Wstat: 256 Tests: 2 Failed: 1)

Please provide the test failure details output before this "Test
Summary Report". This test summary is useless for diagnosing the
problem.

BTW, you don't really need to use the cpan utility to install
Test::Nginx itself. That "cpan" command is just for installing all the
dependencies automatically. Now that you're already running
Test::Nginx's test suite, you have already had all the dependencies
installed. You can just proceed to the subsequent commands like
checking out the git repos of test-nginx. But anyway, I'd like to see
the test failures of Test::Nginx on your side.

Regards,
-agentzh

james_357f

I omitted it because it was incredibly long, I did get it to work on a different box though.

Afterwards I wanted to get it to work in the original environment and was able to fix it with:

apt-get update
apt-get upgrade
apt-get install perl build-essential curl

I'm not sure what I was missing since I was pretty sure I had all of those, but maybe it was recently added to apt (I'm running Ubuntu 13.10 on that box). I successfully got it working with cpanm also (not that that's really relevant)

Most of my tests pass which was cool to see. The last, which sets up multiple location blocks, fails though. Can you only set one location block in the --config section of the test?

On Friday, March 21, 2014 6:28:06 PM UTC-5, agentzh wrote:

Hello!

On Fri, Mar 21, 2014 at 8:54 AM, James Marlowe wrote:
> That is very helpful, I haven't done much work in perl. I tried running sudo
> cpan Test::Nginx but got:
>
> Test Summary Report
> -------------------
> t/syntax.t (Wstat: 256 Tests: 2 Failed: 1)

Please provide the test failure details output before this "Test
Summary Report". This test summary is useless for diagnosing the
problem.

BTW, you don't really need to use the cpan utility to install
Test::Nginx itself. That "cpan" command is just for installing all the
dependencies automatically. Now that you're already running
Test::Nginx's test suite, you have already had all the dependencies
installed. You can just proceed to the subsequent commands like
checking out the git repos of test-nginx. But anyway, I'd like to see
the test failures of Test::Nginx on your side.

Regards,
-agentzh

agentzh

Hello!

On Wed, Mar 26, 2014 at 7:37 AM, James Marlowe wrote:
> I omitted it because it was incredibly long, I did get it to work on a
> different box though.
>

If there're a lot of failures, then I just need the first few.

> Most of my tests pass which was cool to see. The last, which sets up
> multiple location blocks, fails though. Can you only set one location block
> in the --config section of the test?
>

Yes, you can. For example,

    https://github.com/agentzh/echo-nginx-module/blob/master/t/subrequest.t#L18

Best regards,
-agentzh

james_357f

Ah, I see, thank you.

The error was from trying to proxy to an internal location which wasn't working out.

agentzh

Hello!

On Thu, Mar 27, 2014 at 7:46 AM, James Marlowe wrote:
> The error was from trying to proxy to an internal location which wasn't
> working out.
>

You have bugs in your test cases (and maybe also your library being
tested). By applying the following patch to your lua-resty-hmac
master,

diff --git a/t/sanity.t b/t/sanity.t
index 2033b7b..0208ed4 100644
--- a/t/sanity.t
+++ b/t/sanity.t
@@ -98,7 +98,7 @@ signature matches
 --- request
 GET /t
 --- response_body_like chop
-^\S+, \d+ \S+ \d+ \d+:\d+:\d+ \+0000 AWS
AccessKeyId\:Jk9\+HAezinRE9cTmGVpdzcmA1WU\=$
+^\S+, \d+ \S+ \d+ \d+:\d+:\d+ \+0000\nAWS
AccessKeyId\:Jk9\+HAezinRE9cTmGVpdzcmA1WU\=$
 --- no_error_log
 [error]

@@ -117,7 +117,7 @@ GET /t
             if ok then
                 ngx.say("authentication complete")
             else
-                ngx.say("authentication failed")
+                ngx.say("authentication failed: ", err)
             end
         ';
     }
@@ -129,7 +129,7 @@ GET /t
         proxy_set_header Authorization $auth;
         proxy_pass_request_headers off;

-        proxy_pass http://localhost/v;
+        proxy_pass http://127.0.0.1:$server_port/v;
     }
     location /t {
         content_by_lua '

TEST 3 now passes and TEST 4 shows an error returned by your
check_headers() function:

t/sanity.t .. 9/24
#   Failed test 'TEST 4: generate a request and check the auth headers
- response_body - response is expected'
#   at /home/agentzh/git/lua-resty-hmac/../test-nginx/lib/Test/Nginx/Socket.pm
line 1150.
#          got: 'authentication failed: missing headers
#
# '
#     expected: 'authentication complete
# '

#   Failed test 'TEST 4: generate a request and check the auth headers
- response_body - response is expected'
#   at /home/agentzh/git/lua-resty-hmac/../test-nginx/lib/Test/Nginx/Socket.pm
line 1150.
#          got: 'authentication failed: missing headers
#
# '
#     expected: 'authentication complete
# '
# Looks like you failed 2 tests of 24.
t/sanity.t .. Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/24 subtests

Test Summary Report
-------------------
t/sanity.t (Wstat: 512 Tests: 24 Failed: 2)
  Failed tests:  20, 23
  Non-zero exit status: 2
Files=1, Tests=24,  1 wallclock secs ( 0.03 usr  0.00 sys +  0.23 cusr
 0.09 csys =  0.35 CPU)
Result: FAIL

I think you should fix your test case and/or your check_headers()
function yourself. The check_headers() function looks very wrong in
that it tests against uninitialized global Lua variables "method" and
"uri" for header checking. Also it checks the response headers while
your test case is actually sending request headers.

Regards,
-agentzh

james_357f

You have bugs in your test cases (and maybe also your library being
tested). By applying the following patch to your lua-resty-hmac
master,

Yes, I switched some ngx.prints to ngx.says which threw off the newlines and the fix for the internal proxy was great, I was wondering about the server port.

I think you should fix your test case and/or your check_headers()
function yourself. The check_headers() function looks very wrong in
that it tests against uninitialized global Lua variables "method" and
"uri" for header checking. Also it checks the response headers while
your test case is actually sending request headers.

I was previously checking against request method and uri and decided to remove them, but forgot to fix that line. You are right I was completely checking against the wrong headers, fixed that.

Now it still fails test 4 but for a different[better?] reason. Adding debugging output all over the place, I see that when check_headers() calls check_signature() calls generate_signature() it does so with the args mismatched. Is there some kind of caveat to calling self:methods nested? I tested check_signature in test2 so it must be some kind of issue that arises from the nested self call but I don't understand what is happening.

agentzh

Hello!

On Fri, Mar 28, 2014 at 1:01 PM, James Marlowe wrote:
> Now it still fails test 4 but for a different[better?] reason. Adding
> debugging output all over the place, I see that when check_headers() calls
> check_signature() calls generate_signature() it does so with the args
> mismatched. Is there some kind of caveat to calling self:methods nested? I
> tested check_signature in test2 so it must be some kind of issue that arises
> from the nested self call but I don't understand what is happening.
>

Well, please check out the Lua 5.1 language manual for such things:

    http://www.lua.org/manual/5.1/manual.html

Essentially,

    self:method(a, b)

is equivalent to

    self.method(self, a, b)

Regards,
-agentzh

james_357f

Essentially,

self:method(a, b)

is equivalent to

self.method(self, a, b)

Haha! Good show! I knew that rule, but continually overlooked it in that last call assuming I had done it correctly. Even upon reading your response I figured that wasn't the problem, but you were right :)

Everything passes now thanks!

Is it featured enough to be useful for other people now? Things I should add to make it better?