lua-resty-sass design question about integration with OpenResty

aapo.talvensaari · 2014-10-02T22:23:26+00:00

Hello! On Wed, Oct 1, 2014 at 9:38 PM, Robert Paprocki wrote: > While writing some example code, I discovered that my openresty > installation seems t...

lua-resty-sass design question about integration with OpenResty

aapo.talvensaari

I have build a LuaJIT FFI bindings to LibSass here:

https://github.com/bungle/lua-resty-sass

It is currently undocumented as I'm looking ways to integrate it with OpenResty. Here is my first try:

nginx.conf:

location ~ ^(/css/.*)\.css$ {
    default_type text/css;
    try_files $uri @scss;
}

location @scss {
    content_by_lua '
        local res = ngx.location.capture(ngx.var[1] .. ".scss")
        if res.status == 200 then
            local sass = require "resty.sass"
            ngx.print((sass.compile(res.body)))
        else
            ngx.exit(res.status)
        end
    ';
}

Eventually I want this to work like this:

location ~ ^(/css/.*)\.css$ {
    default_type text/css;
    try_files $uri @scss;
}

location @scss {
    content_by_lua_file resty/sass/compile.lua;
}

So, when developing you have .scss files inside your /css directory. But what the browser wants (without js sass compilation or file watchers) is plain .css files. You create html file and reference .css file from there and lua-resty-sass will compile it on the fly (and cache).

How do you like this design? Obviously I will add caching to this as well (which can be disable for development). How should I cache these?

1. Write a compiled .css file to same dir where the .scss file was

2. Use shm (ngx.shared.DICT)

3. Use lua-resty-lrucache

4. Store compiled to datastore like memcached, redis etc.

5. Support all of these and ability to set configuration parameters via nginx.conf

Also, is the content_by_lua a way to go with this compilation phase? There is also a body_filter_by_lua and other phases where this might be attached. I'm using ngx.location.capture here to retrieve the .scss content, but should I use something else?

Thank you for your comments!

Aapo

agentzh

Hello!

On Thu, Oct 2, 2014 at 7:23 AM, Aapo Talvensaari wrote:
> How do you like this design? Obviously I will add caching to this as well
> (which can be disable for development). How should I cache these?
>

It depends on the actual caching requirements. Like: do we need to
re-compile when the .scss file changes?

> 1. Write a compiled .css file to same dir where the .scss file was
> 2. Use shm (ngx.shared.DICT)
> 3. Use lua-resty-lrucache
> 4. Store compiled to datastore like memcached, redis etc.
> 5. Support all of these and ability to set configuration parameters via
> nginx.conf
>

Personally I like 1) most. But we might need a way to invalidate the
.css file when the .scss file changes (maybe a timer or something to
save the stat() system call upon every request).

> Also, is the content_by_lua a way to go with this compilation phase?

Looking fine to me.

> There
> is also a body_filter_by_lua and other phases where this might be attached.

The whole point of body_filter_by_lua is that you cannot control the
content handler, like proxy_pass. You can still do streaming
processing, for example, easily in your content_by_lua.

> I'm using ngx.location.capture here to retrieve the .scss content, but
> should I use something else?
>

ngx.location.capture should be avoided here because I don't see why we
should pay the price of subrequests in this context. We can manipulate
the files directly from within Lua. The only thing we need to be
serious about is how to reduce the number of (file) system calls (for
the best performance).

Regards,
-agentzh

tor

Maybe nginx lua could be extended to re-use file stat and open
functions from Nginx?
There's the nginx file cache interface too that maybe could be
leveraged from Lua?

As far as I know there's no straightforward way to stat from Lua to
check file time stamp ?
Maybe one could use FFI to use the syscalls directly.

Did you already create something to solve these problems, Aapo?

Regards,
Tor



2014-10-03 21:14 GMT+02:00 Yichun Zhang (agentzh) <age...@gmail.com>:
> Hello!
>
> On Thu, Oct 2, 2014 at 7:23 AM, Aapo Talvensaari wrote:
>> How do you like this design? Obviously I will add caching to this as well
>> (which can be disable for development). How should I cache these?
>>
>
> It depends on the actual caching requirements. Like: do we need to
> re-compile when the .scss file changes?
>
>> 1. Write a compiled .css file to same dir where the .scss file was
>> 2. Use shm (ngx.shared.DICT)
>> 3. Use lua-resty-lrucache
>> 4. Store compiled to datastore like memcached, redis etc.
>> 5. Support all of these and ability to set configuration parameters via
>> nginx.conf
>>
>
> Personally I like 1) most. But we might need a way to invalidate the
> .css file when the .scss file changes (maybe a timer or something to
> save the stat() system call upon every request).
>
>> Also, is the content_by_lua a way to go with this compilation phase?
>
> Looking fine to me.
>
>> There
>> is also a body_filter_by_lua and other phases where this might be attached.
>
> The whole point of body_filter_by_lua is that you cannot control the
> content handler, like proxy_pass. You can still do streaming
> processing, for example, easily in your content_by_lua.
>
>> I'm using ngx.location.capture here to retrieve the .scss content, but
>> should I use something else?
>>
>
> ngx.location.capture should be avoided here because I don't see why we
> should pay the price of subrequests in this context. We can manipulate
> the files directly from within Lua. The only thing we need to be
> serious about is how to reduce the number of (file) system calls (for
> the best performance).
>
> Regards,
> -agentzh.

agentzh

Hello!

On Tue, Dec 30, 2014 at 5:47 AM, Tor Hveem wrote:
> Maybe nginx lua could be extended to re-use file stat and open
> functions from Nginx?

Yeah. We could create an nginx C module to expose that feature to the
Lua land, just as with the ngx_lua_upstream module:

    https://github.com/openresty/lua-upstream-nginx-module

> There's the nginx file cache interface too that maybe could be
> leveraged from Lua?
>

Yeah, maybe we can reuse that. Not sure without looking into the
bloody details :)

> As far as I know there's no straightforward way to stat from Lua to
> check file time stamp ?

I believe there're 3rd-party Lua libraries exposing such features.

> Maybe one could use FFI to use the syscalls directly.
>

Sure.

Sorry for the delay. Long holidays :)

Regards,
-agentzh