How to add javascript in response body as DOM

sunny.k · 2015-09-07T19:24:14+00:00

Thanks DeJiang, its working now.. But what about my actual server endpoint. that is something like http://servername.com/getdata ; do i need to mention the i...

How to add javascript in response body as DOM

sunny.k

worker_processes  4;
error_log logs/error.log;
events {
    worker_connections 1024;
}
http {
    lua_package_path "./lua/?.lua;;";

    #Basic reverse proxy server
    upstream backend{
        #server "mention IP of hostname or hostname"
        server   0.0.0.0:5000;
    }


    server {
        listen 8080;


        lua_code_cache off; # for development only

        location / {
            #default_type text/html;
            #content_by_lua_file ./lua/nginx_ss.lua;

            proxy_pass   http://backend/api/v1/ssjs;

            header_filter_by_lua 'ngx.header.content_length = nil';

            body_filter_by_lua_file  ./lua/filterbody.lua;


        }
    }
}

############################## filterbody.lua  contains

replacestr = "<script>here you go</script></head>";
    ngx.arg[1] = ngx.re.sub(ngx.arg[1],"</head>", replacestr);

What is the role of ngx.arg[1] and ngx.arg[2] ? 

Please provide me solution in lua and nginx only. I cant add include anymore dependency ! Thanks.

agentzh

Hello!

On Mon, Sep 7, 2015 at 7:24 PM,  <sun...@shieldsquare.com> wrote:
> ############################## filterbody.lua  contains
>
> replacestr = "<script>here you go</script></head>";
>     ngx.arg[1] = ngx.re.sub(ngx.arg[1],"</head>", replacestr);
>
> What is the role of ngx.arg[1] and ngx.arg[2] ?
>

Regarding the role of ngx.arg[1] and ngx.arg[2], they are actually documented:

    https://github.com/openresty/lua-nginx-module#body_filter_by_lua

BTW, I cannot understand your question in the mail title. And you
didn't provide any details regarding that in your mail body. Alas.

Regards,
-agentzh

sunny.k

Sorry for unclear problem statement. What i wanted to do is, i need to add _javascript_ back in DOM of response html body. it could be in <head> tag or in <body> tag. My js is

<script type="text/_javascript_">
       var __uzdbm_a = "asdasd-asdsad-1232asd-12312--asdasd";
</script> 

But i am trying above method of catching ngx.arg[1] that contains response body in chunk and check for that <head> tag and replace with whatever DOM or html u want to append but its not working. My http flow is like this :

I am catching a request at any block level with Lua and setting cookies and other parameter in header of request.
Passing request new created body to
ngx.location.capture("api/call_remote_server", { method = ngx.HTTP_POST, body = final_json ).
"api/call_remote_server" block has proxy_pass to upstream directive and getting the response back from remote server.
whatever the response i got i need to append that JS in body before rendering it back to browser.
for that m calling filter_body directive and passing filtering logic but that is not working.
I tried to assign ngx.arg[1] = nil and then assigned my JS. It rendering two <script> tag instead of one back to browser and ofcourse loosing all the response body.

So suggest me how to correct it if i am doing anything wrong?

On Monday, September 7, 2015 at 4:54:15 PM UTC+5:30, sun...@shieldsquare.com wrote:

worker_processes 4; error_log logs/error.log; events { worker_connections 1024; } http { lua_package_path "./lua/?.lua;;"; #Basic reverse proxy server upstream backend{ #server "mention IP of hostname or hostname" server 0.0.0.0:5000; } server { listen 8080; lua_code_cache off; # for development only location / { #default_type text/html; #content_by_lua_file ./lua/nginx_ss.lua; proxy_pass http://backend/api/v1/ssjs; header_filter_by_lua 'ngx.header.content_length = nil'; body_filter_by_lua_file ./lua/filterbody.lua; } } } ############################## filterbody.lua contains replacestr = "<script>here you go</script></head>"; ngx.arg[1] = ngx.re.sub(ngx.arg[1],"</head>", replacestr); What is the role of ngx.arg[1] and ngx.arg[2] ?

Please provide me solution in lua and nginx only. I cant add include anymore dependency ! Thanks.

agentzh

Hello!

On Tue, Sep 8, 2015 at 4:24 PM,  <sun...@shieldsquare.com> wrote:
> Sorry for unclear problem statement. What i wanted to do is, i need to add
> Javascript back in DOM of response html body. it could be in <head> tag or
> in  <body> tag. My js is
> <script type="text/javascript">
>        var __uzdbm_a = "asdasd-asdsad-1232asd-12312--asdasd";
> </script>

Seems like this is easiest with the standard ngx_sub module:

    http://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter

> "api/call_remote_server" block has proxy_pass to upstream directive and
> getting the response back from remote server.
> whatever the response i got i need to append that JS in body before
> rendering it back to browser.
> for that m calling filter_body directive and passing filtering logic but
> that is not working.
> I tried to assign ngx.arg[1] = nil and then assigned my JS. It rendering two
> <script> tag instead of one back to browser and ofcourse loosing all the
> response body.
>
> So suggest me how to correct it if i am doing anything wrong?

Without seeing your actual code and configurations, I cannot really
help by just reading your (vague) descriptions. Try providing a
minimal and standalone example that I (and others) and easily run and
reproduce the issue on my side.

Regards,
-agentzh

sunny.k

Minimal Code is included one very first post. Nginx.conf and filterbody.lua for filtering body response. See if you can get any error in that!

On Monday, September 7, 2015 at 4:54:15 PM UTC+5:30, sun...@shieldsquare.com wrote:

worker_processes 4; error_log logs/error.log; events { worker_connections 1024; } http { lua_package_path "./lua/?.lua;;"; #Basic reverse proxy server upstream backend{ #server "mention IP of hostname or hostname" server 0.0.0.0:5000; } server { listen 8080; lua_code_cache off; # for development only location / { #default_type text/html; #content_by_lua_file ./lua/nginx_ss.lua; proxy_pass http://backend/api/v1/ssjs; header_filter_by_lua 'ngx.header.content_length = nil'; body_filter_by_lua_file ./lua/filterbody.lua; } } } ############################## filterbody.lua contains replacestr = "<script>here you go</script></head>"; ngx.arg[1] = ngx.re.sub(ngx.arg[1],"</head>", replacestr); What is the role of ngx.arg[1] and ngx.arg[2] ?

Please provide me solution in lua and nginx only. I cant add include anymore dependency ! Thanks.

agentzh

Hello!

On Tue, Sep 8, 2015 at 4:52 PM,  <sun...@shieldsquare.com> wrote:
> Minimal Code is included one very first post. Nginx.conf and filterbody.lua
> for filtering body response. See if you can get any error in that!
>

Your first example contains one obvious mistake that you assume the
response body only invokes your body_filter_by_lua_file once. That's
not true and for large enough response bodies, the body will receive
as multiple data chunks, each of which invokes body_filter_by_lua_file
once. See the official documentation for body_filter_by_lua for more
details:

    https://github.com/openresty/lua-nginx-module#body_filter_by_lua

Another problem in your code is that the data chunk boundary MAY split
the "</head>" string in the data stream and in that case, your naive
string.sub() call will definitely fail to match. You have to take that
special case into account. And that's also why I recommend ngx_sub
module for this since ngx_sub takes care of such cases already (so
does the more powerful ngx_replace_filter module that supports Perl
style regexes as patterns).

Regards,
-agentzh

sunnygupta.kr

--------------Nginx.conf---------------

        location / {
            access_by_lua_file  ./lua/nginx_ss.lua;
            #echo "Demonic World";

            proxy_pass  http://127.0.0.1:8000/$request_uri;

            sub_filter '<script type="text/_javascript_">var i = 10;</script></body>'  '</body>';
            sub_filter_once on;
            sub_filter_types text/html;

            #header_filter_by_lua '
            #ngx.header.content_length = nil
            #';
            #body_filter_by_lua_file ./lua/js_ss.lua;

        }

---------------------- js_ss.lua -------------------------------

local ss_script = "<script type=\"text/_javascript_\">var i = 10;</script></body>"

--print(ss_script);

local chunk, eof = ngx.arg[1], ngx.arg[2]
local buffered = ngx.ctx.buffered
if not buffered then
    buffered = {}  -- XXX we can use table.new here
    ngx.ctx.buffered = buffered
end
if chunk ~= "" then
    buffered[#buffered + 1] = chunk
    ngx.arg[1] = nil
end
if eof then
    local whole = table.concat(buffered)
    ngx.ctx.buffered = nil
    whole = string.gsub(whole, ss_script, "</body>")
    ngx.arg[1] = whole
end

I have tried both method but i am not able to append the JS to response body.. Please point my errors here and i cant add any other dependency it has to be in lua openresty domain! Thanks.

agentzh

Hello!

On Sun, Sep 13, 2015 at 8:31 AM, Sunny GUPTA wrote:
>         location / {
>             access_by_lua_file  ./lua/nginx_ss.lua;
>             #echo "Demonic World";
>
>             proxy_pass  http://127.0.0.1:8000/$request_uri;
>
>             sub_filter '<script type="text/javascript">var i =
> 10;</script></body>'  '</body>';

I think you confused the order of the arguments of the sub_filter directive. See

    http://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter

You're removing JS snippets here instead of appending one.

Could you be a little bit more careful? It can save both of our time. Thank you.

> local chunk, eof = ngx.arg[1], ngx.arg[2]
> local buffered = ngx.ctx.buffered
> if not buffered then
>     buffered = {}  -- XXX we can use table.new here
>     ngx.ctx.buffered = buffered
> end
> if chunk ~= "" then
>     buffered[#buffered + 1] = chunk
>     ngx.arg[1] = nil

I really don't understand why you have to buffer the whole response
body just fore inserting something before </body>. It just doesn't
make sense.

Regards,
-agentzh

sunny.k

Thanks agentzh, ofcourse yeah first string then replacement that's an order. How did i did silly mistakes. Stupid huh!

But in 2nd using filter body in lua, I need to append the _javascript_ in response DOM, it is dynamic in every call. It will be better if I can add js in <head> tag of response body.

So I followed some of early threads regarding response changing and found your chunk buffering n then applying replacement to get guaranteed response change. But its also not working. Help me out.

sunnygupta.kr

Sub_filter only filters and allow change inside <body> tag elements not with <head> and <body> replacement itself.

<head>

</head>

<body>

.................

................

</body>

I tried with body filter too. no success. just illustrate any example if that is possible along with suitable code. I needed to append js in lua land because i needed to set some cookie variable into js part too, so working on location block will be messy. Do u have any suggestions with body filter directive or any other without adding any dependency? Thanks