combine multiple file in one location

anton.heryanto · 2014-07-18T11:48:45+00:00

On Tuesday, July 22, 2014 7:45:41 PM UTC+3, Jon Keys wrote:I've not heard of Nettle before. It looks really nice!Is there any reason to choose Nettle over Op...

combine multiple file in one location

anton.heryanto

Hello,

i have try to uses openresty for api and client side _javascript_ development, for code to be more readable js code is separated into multiple files, but in order to live test the code the js need to combine into single dev.js file in order to simplify development. i know it can be handle using other tool such nodejs gaze, but if will be more simpler if only openresty is required.

in index.html

<script src="dev.js"></script>

in nginx.conf

location /dev.js {
      default_type "application/_javascript_; charset=UTF-8";
      content_by_lua_file "api/dev.lua";
    }

dev.lua

local popen = io.popen
local open = io.open
local say = ngx.say
local path = 'app'

local cmd = 'find '.. path ..' -name "*.js"'
local handler,err = popen(cmd)
if not handler then return end

for file in handler:lines() do
  local h = open(file,'r')
  say(h:read("*all"))
  h:close()
end

handler:close()

this code often get interupt system call error, i have read few about blocking nature of io.popen in openresty which cause error interrupt system call. so i have reduce io access by changes multiple file open handler into subrequest instead

dev.lua

local popen = io.popen
local open = io.open
local say = ngx.say
local capture = ngx.location.capture
local path = 'app'

local cmd = 'find '.. path ..' -name "*.js"'
local handler,err = popen(cmd)
if not handler then return end

local result = handler:read('*a')
handler:close()

if not result then ngx.log(ngx.WARN, "files not found") return end

local files = split(result,'\n') -- using custom string split function
for i=1, #files in do
  local file = files[i]
  local res = capture('/'.. file)
  if res.status == 200 then say(res.body) end
end

the interrupt system call error gone, but the popen call often return nil result. the core function need to achieve actually how to get certain files in folder which later can be combined.

is there any other way to achieve this. trying using lua ffi to uses posix opendir, readdir to reduce io issue with openresty without luck aside of its cross platform issue, which we have to implement for every possible platform. or is there any way we can get list of files in directory like auto_index module and get the value in lua so we can filter and combine it.

regard

Anton Heryanto

manchev

Hi Anton,

You should really use the echo module: http://wiki.nginx.org/HttpEchoModule#echo_foreach_split

Best,
Vladislav

On Fri, Jul 18, 2014 at 6:48 AM, Anton Heryanto <anton....@gmail.com> wrote:

Hello,

i have try to uses openresty for api and client side _javascript_ development, for code to be more readable js code is separated into multiple files, but in order to live test the code the js need to combine into single dev.js file in order to simplify development. i know it can be handle using other tool such nodejs gaze, but if will be more simpler if only openresty is required.

in index.html
<script src="dev.js"></script>

in nginx.conf

location /dev.js { default_type "application/_javascript_; charset=UTF-8"; content_by_lua_file "api/dev.lua"; }

dev.lua
local popen = io.popen local open = io.open local say = ngx.say local path = 'app' local cmd = 'find '.. path ..' -name "*.js"' local handler,err = popen(cmd) if not handler then return end for file in handler:lines() do local h = open(file,'r') say(h:read("*all")) h:close() end handler:close()

this code often get interupt system call error, i have read few about blocking nature of io.popen in openresty which cause error interrupt system call. so i have reduce io access by changes multiple file open handler into subrequest instead

dev.lua
local popen = io.popen local open = io.open local say = ngx.say local capture = ngx.location.capture local path = 'app' local cmd = 'find '.. path ..' -name "*.js"' local handler,err = popen(cmd) if not handler then return end local result = handler:read('*a') handler:close() if not result then ngx.log(ngx.WARN, "files not found") return end local files = split(result,'\n') -- using custom string split function for i=1, #files in do local file = files[i] local res = capture('/'.. file) if res.status == 200 then say(res.body) end end

the interrupt system call error gone, but the popen call often return nil result. the core function need to achieve actually how to get certain files in folder which later can be combined.

is there any other way to achieve this. trying using lua ffi to uses posix opendir, readdir to reduce io issue with openresty without luck aside of its cross platform issue, which we have to implement for every possible platform. or is there any way we can get list of files in directory like auto_index module and get the value in lua so we can filter and combine it.

regard

Anton Heryanto

.

anton.heryanto

Hi Vladislav,

thanks, its handy if we manually specified the file in query string, but how to dynamic list the files we uses in development, which currently i solve by using linux command find (possible js file is in subfolder)

Regard

Anton

On Friday, July 18, 2014 12:08:34 PM UTC+8, Vladislav Manchev wrote:

Hi Anton,

You should really use the echo module: http://wiki.nginx.org/HttpEchoModule#echo_foreach_split

Best,
Vladislav

On Fri, Jul 18, 2014 at 6:48 AM, Anton Heryanto <anton.h...@gmail.com> wrote:

Hello,

i have try to uses openresty for api and client side _javascript_ development, for code to be more readable js code is separated into multiple files, but in order to live test the code the js need to combine into single dev.js file in order to simplify development. i know it can be handle using other tool such nodejs gaze, but if will be more simpler if only openresty is required.

in index.html
<script src="dev.js"></script>

in nginx.conf

location /dev.js { default_type "application/_javascript_; charset=UTF-8"; content_by_lua_file "api/dev.lua"; }

dev.lua
local popen = io.popen local open = io.open local say = ngx.say local path = 'app' local cmd = 'find '.. path ..' -name "*.js"' local handler,err = popen(cmd) if not handler then return end for file in handler:lines() do local h = open(file,'r') say(h:read("*all")) h:close() end handler:close()

this code often get interupt system call error, i have read few about blocking nature of io.popen in openresty which cause error interrupt system call. so i have reduce io access by changes multiple file open handler into subrequest instead

dev.lua
local popen = io.popen local open = io.open local say = ngx.say local capture = ngx.location.capture local path = 'app' local cmd = 'find '.. path ..' -name "*.js"' local handler,err = popen(cmd) if not handler then return end local result = handler:read('*a') handler:close() if not result then ngx.log(ngx.WARN, "files not found") return end local files = split(result,'\n') -- using custom string split function for i=1, #files in do local file = files[i] local res = capture('/'.. file) if res.status == 200 then say(res.body) end end

the interrupt system call error gone, but the popen call often return nil result. the core function need to achieve actually how to get certain files in folder which later can be combined.

is there any other way to achieve this. trying using lua ffi to uses posix opendir, readdir to reduce io issue with openresty without luck aside of its cross platform issue, which we have to implement for every possible platform. or is there any way we can get list of files in directory like auto_index module and get the value in lua so we can filter and combine it.

regard

Anton Heryanto

.

lancelijade

When got nil, just retry. For development I think it is enough. Maybe a short time cache, such as 10 secs, can do better.

Lance

On Fri, Jul 18, 2014 at 12:53 PM, Anton Heryanto <anton....@gmail.com> wrote:

Hi Vladislav,

thanks, its handy if we manually specified the file in query string, but how to dynamic list the files we uses in development, which currently i solve by using linux command find (possible js file is in subfolder)

Regard

Anton

On Friday, July 18, 2014 12:08:34 PM UTC+8, Vladislav Manchev wrote:

Hi Anton,

You should really use the echo module: http://wiki.nginx.org/HttpEchoModule#echo_foreach_split

Best,
Vladislav

On Fri, Jul 18, 2014 at 6:48 AM, Anton Heryanto <anton.h...@gmail.com> wrote:

Hello,

i have try to uses openresty for api and client side _javascript_ development, for code to be more readable js code is separated into multiple files, but in order to live test the code the js need to combine into single dev.js file in order to simplify development. i know it can be handle using other tool such nodejs gaze, but if will be more simpler if only openresty is required.

in index.html
<script src="dev.js"></script>

in nginx.conf

location /dev.js { default_type "application/_javascript_; charset=UTF-8"; content_by_lua_file "api/dev.lua"; }

dev.lua
local popen = io.popen local open = io.open local say = ngx.say local path = 'app' local cmd = 'find '.. path ..' -name "*.js"' local handler,err = popen(cmd) if not handler then return end for file in handler:lines() do local h = open(file,'r') say(h:read("*all")) h:close() end handler:close()

this code often get interupt system call error, i have read few about blocking nature of io.popen in openresty which cause error interrupt system call. so i have reduce io access by changes multiple file open handler into subrequest instead

dev.lua
local popen = io.popen local open = io.open local say = ngx.say local capture = ngx.location.capture local path = 'app' local cmd = 'find '.. path ..' -name "*.js"' local handler,err = popen(cmd) if not handler then return end local result = handler:read('*a') handler:close() if not result then ngx.log(ngx.WARN, "files not found") return end local files = split(result,'\n') -- using custom string split function for i=1, #files in do local file = files[i] local res = capture('/'.. file) if res.status == 200 then say(res.body) end end

the interrupt system call error gone, but the popen call often return nil result. the core function need to achieve actually how to get certain files in folder which later can be combined.

is there any other way to achieve this. trying using lua ffi to uses posix opendir, readdir to reduce io issue with openresty without luck aside of its cross platform issue, which we have to implement for every possible platform. or is there any way we can get list of files in directory like auto_index module and get the value in lua so we can filter and combine it.

regard

Anton Heryanto

.

anton.heryanto

Hi Lance,

Thank for suggestion, yup because its for development, performance not really matter as long as not noticeable. therefore i try your idea by retry the popen request using while loop and break when result not nil and return cache after 100 times retry

its working perfectly, without any performance impact.

following updated code:
dev.lua
local popen = io.popen local say = ngx.say local capture = ngx.location.capture local cache = ngx.shared.templates local path = 'app' local cmd = 'find '.. path ..' -name "*.js"' local i = 0 local result while true do local handler,err = popen(cmd) if not handler then break end result = handler:read('*a') handler:close() if result or i == 100 then break end ngx.log(ngx.INFO, "result empty at ", i) i = i + 1 end if not result then ngx.log(ngx.WARN, "files not found") say(cache:get("js")) return end local files = split(result,'\n') -- using custom string split function local o = "" for i=1, #files in do local file = files[i] local res = capture('/'.. file) if res.status == 200 then o = o .. res.body end end cache:set("js", o) say(o)

Thank all for response

Anton Heryanto

On Friday, July 18, 2014 5:28:52 PM UTC+8, Lance Li wrote:
When got nil, just retry. For development I think it is enough. Maybe a short time cache, such as 10 secs, can do better.

Lance

On Fri, Jul 18, 2014 at 12:53 PM, Anton Heryanto <anton.h...@gmail.com> wrote:

Hi Vladislav,

thanks, its handy if we manually specified the file in query string, but how to dynamic list the files we uses in development, which currently i solve by using linux command find (possible js file is in subfolder)

Regard

Anton

On Friday, July 18, 2014 12:08:34 PM UTC+8, Vladislav Manchev wrote:

Hi Anton,

You should really use the echo module: http://wiki.nginx.org/HttpEchoModule#echo_foreach_split

Best,
Vladislav

On Fri, Jul 18, 2014 at 6:48 AM, Anton Heryanto <anton.h...@gmail.com> wrote:

Hello,

i have try to uses openresty for api and client side _javascript_ development, for code to be more readable js code is separated into multiple files, but in order to live test the code the js need to combine into single dev.js file in order to simplify development. i know it can be handle using other tool such nodejs gaze, but if will be more simpler if only openresty is required.

in index.html
<script src="dev.js"></script>

in nginx.conf

location /dev.js { default_type "application/_javascript_; charset=UTF-8"; content_by_lua_file "api/dev.lua"; }

dev.lua
local popen = io.popen local open = io.open local say = ngx.say local path = 'app' local cmd = 'find '.. path ..' -name "*.js"' local handler,err = popen(cmd) if not handler then return end for file in handler:lines() do local h = open(file,'r') say(h:read("*all")) h:close() end handler:close()

this code often get interupt system call error, i have read few about blocking nature of io.popen in openresty which cause error interrupt system call. so i have reduce io access by changes multiple file open handler into subrequest instead

dev.lua
local popen = io.popen local open = io.open local say = ngx.say local capture = ngx.location.capture local path = 'app' local cmd = 'find '.. path ..' -name "*.js"' local handler,err = popen(cmd) if not handler then return end local result = handler:read('*a') handler:close() if not result then ngx.log(ngx.WARN, "files not found") return end local files = split(result,'\n') -- using custom string split function for i=1, #files in do local file = files[i] local res = capture('/'.. file) if res.status == 200 then say(res.body) end end

the interrupt system call error gone, but the popen call often return nil result. the core function need to achieve actually how to get certain files in folder which later can be combined.

is there any other way to achieve this. trying using lua ffi to uses posix opendir, readdir to reduce io issue with openresty without luck aside of its cross platform issue, which we have to implement for every possible platform. or is there any way we can get list of files in directory like auto_index module and get the value in lua so we can filter and combine it.

regard

Anton Heryanto

.