Parsing JSON within OpenResty tests

suchitpuri · 2015-01-06T21:24:33+00:00

but I am missing a feature, I'd like to do QResult = ngx.shared.iworkcomproto:get("IWCP_MSG_PD%") but obviously it doesn't work, other then going through all...

Parsing JSON within OpenResty tests

suchitpuri

Hi guys,

We are using openresty for our dynamic CDN network, we usually have to connect to mysql, redis and work using JSON. We also are trying to build our test suite so that openresty code also becomes part of our CI/CD pipeline.

Unfortunately i am stuck at a very silly issue involving json and escaping strings.

If you look at the code here https://gist.github.com/suchitpuri/02be88e23da404e7368c

The code throws 500 error saying

the code which is throwing this error is

            res, err, errno, sqlstate = db:query("insert into some_table (a_id,s_id,q) VALUES (1234,1234,\"{\"key\":\"value\",\"k\":\"v\"}\")")

I am puzzled as to how to correctly escape the json string in my test as it is blocking us from writing full fledged test, testing real world values.


Regards
Suchit Puri


if you are too lazy to click on the gist like , please find the full gist code below.

use Test::Nginx::Socket;
plan tests => $Test::Nginx::Socket::RepeatEach * 2 * blocks();
run_tests();
 
 
__DATA__
=== TEST 1.1: Should fetch the questions from memcache first

--- http_config
    lua_shared_dict                 URLS 50m;
    lua_shared_dict                 SETTINGS 250m;
    lua_shared_dict                 APPS 10m;
    lua_shared_dict                 APPSETTINGS 50m;
    lua_shared_dict                 STATS 10m;
    lua_shared_dict                 HASH 1m;
    lua_package_path                '../lua/?.lua;;';
    lua_package_cpath               '../lua/?.so;;';

--- config
    location = /questions {
          content_by_lua '
            local common = require "common"
            local somelib = require "somelib"
            local db = common.getMySQL()
            local memc=common.getMemcache()
            local cjson = require "cjson"

            db:query("delete from some_table where account_id=something and survey_id=something")
            memc:delete("some memcache key")

            local sqlParams = cjson.encode({key = 1, anotherKey=2})
            local err, errno, sqlstate
            res, err, errno, sqlstate = db:query("insert into some_table (a_id,s_id,q) VALUES (1234,1234,\"{\"key\":\"value\",\"k\":\"v\"}\")")
            
            if not res then
                ngx.log(ngx.ERR, "Bad result from MySQL: ", err, ": ", errno, ": ", sqlstate, ".")
            end

            ngx.say(somelib.someMethod(1234,1234))

            local res , err = memc:get("some memcache key")
            ngx.say(res)

          ';

    }
--- request
    GET /questions
--- response_body
{"key":"value","k":"v"}
{"key":"value","k":"v"}

hamish.forbes

You're going to need more slashes.

Its a similar issue to escaping PCRE sequences: https://github.com/openresty/lua-nginx-module#special-pcre-sequences

Check out the bit about both the nginx and lua parsers are stripping backslashes.

You could also use the double bracket for delimiting strings too.

e.g.

local json_string = [[{key: "value"}]]

On Tuesday, 6 January 2015 13:24:32 UTC, Suchit Puri wrote:

Hi guys,

Unfortunately i am stuck at a very silly issue involving json and escaping strings.

If you look at the code here https://gist.github.com/suchitpuri/02be88e23da404e7368c

The code throws 500 error saying

the code which is throwing this error is

            res, err, errno, sqlstate = db:query("insert into some_table (a_id,s_id,q) VALUES (1234,1234,\"{\"key\":\"value\",\"k\":\"v\"}\")")

I am puzzled as to how to correctly escape the json string in my test as it is blocking us from writing full fledged test, testing real world values.


Regards
Suchit Puri


if you are too lazy to click on the gist like , please find the full gist code below.

use Test::Nginx::Socket;
plan tests => $Test::Nginx::Socket::RepeatEach * 2 * blocks();
run_tests();
 
 
__DATA__
=== TEST 1.1: Should fetch the questions from memcache first

--- http_config
    lua_shared_dict                 URLS 50m;
    lua_shared_dict                 SETTINGS 250m;
    lua_shared_dict                 APPS 10m;
    lua_shared_dict                 APPSETTINGS 50m;
    lua_shared_dict                 STATS 10m;
    lua_shared_dict                 HASH 1m;
    lua_package_path                '../lua/?.lua;;';
    lua_package_cpath               '../lua/?.so;;';

--- config
    location = /questions {
          content_by_lua '
            local common = require "common"
            local somelib = require "somelib"
            local db = common.getMySQL()
            local memc=common.getMemcache()
            local cjson = require "cjson"

            db:query("delete from some_table where account_id=something and survey_id=something")
            memc:delete("some memcache key")

            local sqlParams = cjson.encode({key = 1, anotherKey=2})
            local err, errno, sqlstate
            res, err, errno, sqlstate = db:query("insert into some_table (a_id,s_id,q) VALUES (1234,1234,\"{\"key\":\"value\",\"k\":\"v\"}\")")
            
            if not res then
                ngx.log(ngx.ERR, "Bad result from MySQL: ", err, ": ", errno, ": ", sqlstate, ".")
            end

            ngx.say(somelib.someMethod(1234,1234))

            local res , err = memc:get("some memcache key")
            ngx.say(res)

          ';

    }
--- request
    GET /questions
--- response_body
{"key":"value","k":"v"}
{"key":"value","k":"v"}

suchitpuri

Hi Hamish,

Thanks a lot for the explanation. This worked just fine.

Cheers

Suchit

On Tue Jan 06 2015 at 10:27:29 PM Hamish Forbes <hamish...@gmail.com> wrote:

You're going to need more slashes.

Its a similar issue to escaping PCRE sequences: https://github.com/openresty/lua-nginx-module#special-pcre-sequences

Check out the bit about both the nginx and lua parsers are stripping backslashes.

You could also use the double bracket for delimiting strings too.

e.g.

local json_string = [[{key: "value"}]]

On Tuesday, 6 January 2015 13:24:32 UTC, Suchit Puri wrote:

Hi guys,

Unfortunately i am stuck at a very silly issue involving json and escaping strings.

If you look at the code here https://gist.github.com/suchitpuri/02be88e23da404e7368c

The code throws 500 error saying

the code which is throwing this error is

            res, err, errno, sqlstate = db:query("insert into some_table (a_id,s_id,q) VALUES (1234,1234,\"{\"key\":\"value\",\"k\":\"v\"}\")")

I am puzzled as to how to correctly escape the json string in my test as it is blocking us from writing full fledged test, testing real world values.


Regards
Suchit Puri


if you are too lazy to click on the gist like , please find the full gist code below.

use Test::Nginx::Socket;
plan tests => $Test::Nginx::Socket::RepeatEach * 2 * blocks();
run_tests();
 
 
__DATA__
=== TEST 1.1: Should fetch the questions from memcache first

--- http_config
    lua_shared_dict                 URLS 50m;
    lua_shared_dict                 SETTINGS 250m;
    lua_shared_dict                 APPS 10m;
    lua_shared_dict                 APPSETTINGS 50m;
    lua_shared_dict                 STATS 10m;
    lua_shared_dict                 HASH 1m;
    lua_package_path                '../lua/?.lua;;';
    lua_package_cpath               '../lua/?.so;;';

--- config
    location = /questions {
          content_by_lua '
            local common = require "common"
            local somelib = require "somelib"
            local db = common.getMySQL()
            local memc=common.getMemcache()
            local cjson = require "cjson"

            db:query("delete from some_table where account_id=something and survey_id=something")
            memc:delete("some memcache key")

            local sqlParams = cjson.encode({key = 1, anotherKey=2})
            local err, errno, sqlstate
            res, err, errno, sqlstate = db:query("insert into some_table (a_id,s_id,q) VALUES (1234,1234,\"{\"key\":\"value\",\"k\":\"v\"}\")")
            
            if not res then
                ngx.log(ngx.ERR, "Bad result from MySQL: ", err, ": ", errno, ": ", sqlstate, ".")
            end

            ngx.say(somelib.someMethod(1234,1234))

            local res , err = memc:get("some memcache key")
            ngx.say(res)

          ';

    }
--- request
    GET /questions
--- response_body
{"key":"value","k":"v"}
{"key":"value","k":"v"}