Replacing PHP

rezclick · 2014-12-27T01:57:31+00:00

Hello! On Tue, Dec 23, 2014 at 12:41 PM, Sophia Elanor wrote: > Here it is: http://pastebin.centos.org/14746/67300141/ > Okay, this is what I need....

Replacing PHP

rezclick

Hello all,

I saw on YouTube agentzh's NGinx conf 2014 talk about OpenResty and was smitten. Since then I've been thinking through how to replace our PHP app w/ OpenResty. I've already mapped out how most (80%) of our PHP will translate to OpenResty but I'm unsure about how to approach the HTML. Specifically, I would like some guidance on what is the best (i.e., efficient and maintainable) method to compose and/or manage HTML responses (20%).

I'm not new to programming. I've got history and there are couple things from old that I would like not to repeat:

1. Composition via string concatenation of massive chunks of markup. The problem is the HTML and the Lua code become too tightly coupled so suddenly the markup is outside the reach and understanding of our web designers. This is a maintainability nightmare.

2. Templating languages. I'm not refusing to use a templating language but I am concerned about their efficiency. Templating languages carry overhead because the markup gets parsed twice. Once by the runtime (e.g., constructing string objects) and the second time by the templating engine as it consumes the string objects to scan the markup to find/replace the template variables. Now you may be thinking, "but PHP is a templating langue", and you are right. But they solved the double parsing problem by making the templating engine and the runtime one and the same. Granted, they did a horrific job in terms of memory utilization and performance, but conceptually, they nailed it.

Please note, if a templating language is the only reasonable solution that I would appreciate a reference to a low overhead templating solution.

Thanks,

-rez

leafot

Template languages can compile the template into Lua code that is effectively a series of string concatenations when executed, so there is no parsing overhead on each request. Here's one that I've wrote for OpenResty: https://github.com/leafo/etlua

Here's an example of the generated code: https://github.com/leafo/etlua#lua_code-err--parsercompile_to_luaetlua_code

(PS I'm aware of the new LuaJIT string buffer support, but I haven't had a chance to implement, test, and benchmark yet. The above code should still be plenty fast)

On Fri, Dec 26, 2014 at 9:57 AM, <rez...@gmail.com> wrote:

Hello all,

I saw on YouTube agentzh's NGinx conf 2014 talk about OpenResty and was smitten. Since then I've been thinking through how to replace our PHP app w/ OpenResty. I've already mapped out how most (80%) of our PHP will translate to OpenResty but I'm unsure about how to approach the HTML. Specifically, I would like some guidance on what is the best (i.e., efficient and maintainable) method to compose and/or manage HTML responses (20%).

I'm not new to programming. I've got history and there are couple things from old that I would like not to repeat:
1. Composition via string concatenation of massive chunks of markup. The problem is the HTML and the Lua code become too tightly coupled so suddenly the markup is outside the reach and understanding of our web designers. This is a maintainability nightmare.

2. Templating languages. I'm not refusing to use a templating language but I am concerned about their efficiency. Templating languages carry overhead because the markup gets parsed twice. Once by the runtime (e.g., constructing string objects) and the second time by the templating engine as it consumes the string objects to scan the markup to find/replace the template variables. Now you may be thinking, "but PHP is a templating langue", and you are right. But they solved the double parsing problem by making the templating engine and the runtime one and the same. Granted, they did a horrific job in terms of memory utilization and performance, but conceptually, they nailed it.

Please note, if a templating language is the only reasonable solution that I would appreciate a reference to a low overhead templating solution.

Thanks,

-rez

.

rezclick

Looks great! I'll spend some time with it. Thank you.

One question, but before I ask, a brief disclaimer. I'm not yet intimately familiar w/ the inner workings of Lua so my question may be moot because the assumption I'm making is that the Lua runtime does not internally store strings as UTF-8 encoded bytes.

Now the question. Would you consider a mode where the static strings are stored/cached as UTF-8 encoded bytes? A long time ago I used to hack on the opensource version of a Java application server called Resin. Resin was and remains one of the fastest application servers of any language. One of Resins many optimizations was that when it compiled JSPs and servlets it converted all strings to bytes and threw away the strings. So not only was the generated code memory efficient (i.e, Java strings are internally UTF-16 so a byte array of UTF-8 encoded bytes are 1/2 the size for most Latin text) it avoided the constant string to byte conversion.

Regards,

-rez

On Fri, Dec 26, 2014 at 1:11 PM, leaf corcoran <le...@gmail.com> wrote:

Template languages can compile the template into Lua code that is effectively a series of string concatenations when executed, so there is no parsing overhead on each request. Here's one that I've wrote for OpenResty: https://github.com/leafo/etlua

Here's an example of the generated code: https://github.com/leafo/etlua#lua_code-err--parsercompile_to_luaetlua_code

(PS I'm aware of the new LuaJIT string buffer support, but I haven't had a chance to implement, test, and benchmark yet. The above code should still be plenty fast)

On Fri, Dec 26, 2014 at 9:57 AM, <rez...@gmail.com> wrote:
Hello all,

I saw on YouTube agentzh's NGinx conf 2014 talk about OpenResty and was smitten. Since then I've been thinking through how to replace our PHP app w/ OpenResty. I've already mapped out how most (80%) of our PHP will translate to OpenResty but I'm unsure about how to approach the HTML. Specifically, I would like some guidance on what is the best (i.e., efficient and maintainable) method to compose and/or manage HTML responses (20%).

I'm not new to programming. I've got history and there are couple things from old that I would like not to repeat:
1. Composition via string concatenation of massive chunks of markup. The problem is the HTML and the Lua code become too tightly coupled so suddenly the markup is outside the reach and understanding of our web designers. This is a maintainability nightmare.

2. Templating languages. I'm not refusing to use a templating language but I am concerned about their efficiency. Templating languages carry overhead because the markup gets parsed twice. Once by the runtime (e.g., constructing string objects) and the second time by the templating engine as it consumes the string objects to scan the markup to find/replace the template variables. Now you may be thinking, "but PHP is a templating langue", and you are right. But they solved the double parsing problem by making the templating engine and the runtime one and the same. Granted, they did a horrific job in terms of memory utilization and performance, but conceptually, they nailed it.

Please note, if a templating language is the only reasonable solution that I would appreciate a reference to a low overhead templating solution.

Thanks,

-rez

.

leafot

There's a guide on the Lua wiki about unicode support that you'll find useful: http://lua-users.org/wiki/LuaUnicode

On Fri, Dec 26, 2014 at 11:07 AM, RezDev <rez...@gmail.com> wrote:

Looks great! I'll spend some time with it. Thank you.

One question, but before I ask, a brief disclaimer. I'm not yet intimately familiar w/ the inner workings of Lua so my question may be moot because the assumption I'm making is that the Lua runtime does not internally store strings as UTF-8 encoded bytes.

Now the question. Would you consider a mode where the static strings are stored/cached as UTF-8 encoded bytes? A long time ago I used to hack on the opensource version of a Java application server called Resin. Resin was and remains one of the fastest application servers of any language. One of Resins many optimizations was that when it compiled JSPs and servlets it converted all strings to bytes and threw away the strings. So not only was the generated code memory efficient (i.e, Java strings are internally UTF-16 so a byte array of UTF-8 encoded bytes are 1/2 the size for most Latin text) it avoided the constant string to byte conversion.

Regards,
-rez

On Fri, Dec 26, 2014 at 1:11 PM, leaf corcoran <le...@gmail.com> wrote:
Template languages can compile the template into Lua code that is effectively a series of string concatenations when executed, so there is no parsing overhead on each request. Here's one that I've wrote for OpenResty: https://github.com/leafo/etlua

Here's an example of the generated code: https://github.com/leafo/etlua#lua_code-err--parsercompile_to_luaetlua_code

(PS I'm aware of the new LuaJIT string buffer support, but I haven't had a chance to implement, test, and benchmark yet. The above code should still be plenty fast)

On Fri, Dec 26, 2014 at 9:57 AM, <rez...@gmail.com> wrote:
Hello all,

I saw on YouTube agentzh's NGinx conf 2014 talk about OpenResty and was smitten. Since then I've been thinking through how to replace our PHP app w/ OpenResty. I've already mapped out how most (80%) of our PHP will translate to OpenResty but I'm unsure about how to approach the HTML. Specifically, I would like some guidance on what is the best (i.e., efficient and maintainable) method to compose and/or manage HTML responses (20%).

I'm not new to programming. I've got history and there are couple things from old that I would like not to repeat:
1. Composition via string concatenation of massive chunks of markup. The problem is the HTML and the Lua code become too tightly coupled so suddenly the markup is outside the reach and understanding of our web designers. This is a maintainability nightmare.

2. Templating languages. I'm not refusing to use a templating language but I am concerned about their efficiency. Templating languages carry overhead because the markup gets parsed twice. Once by the runtime (e.g., constructing string objects) and the second time by the templating engine as it consumes the string objects to scan the markup to find/replace the template variables. Now you may be thinking, "but PHP is a templating langue", and you are right. But they solved the double parsing problem by making the templating engine and the runtime one and the same. Granted, they did a horrific job in terms of memory utilization and performance, but conceptually, they nailed it.

Please note, if a templating language is the only reasonable solution that I would appreciate a reference to a low overhead templating solution.

Thanks,

-rez

.

quae

On 26 December 2014 at 14:07, RezDev <rez...@gmail.com> wrote:
> One question, but before I ask, a brief disclaimer. I'm not yet intimately
> familiar w/ the inner workings of Lua so my question may be moot because the
> assumption I'm making is that the Lua runtime does not internally store
> strings as UTF-8 encoded bytes.

Lua DOES store strings as a byte array.
Infact, lua has no concept of encodings: strings are just bytes.
UTF-8 is only a convention for lua code.

The optimisation you go on to describe doesn't help :)

aapo.talvensaari

On Friday, 26 December 2014 19:57:31 UTC+2, rezc...@gmail.com wrote:

Please note, if a templating language is the only reasonable solution that I would appreciate a reference to a low overhead templating solution.

I have compiled a list of templating etc. engines here:

https://github.com/bungle/lua-resty-template#alternatives

I can promote my own:

https://github.com/bungle/lua-resty-template

It is compiling and caching templating engine. By compiling I mean that it compiles your templates to Lua functions, and by caching I mean that it caches those functions. Compiling also means that you can use all Lua contructs in templates (somewhat how you mix and match strings with PHP). You may also precompile the templates if you want to (binary Lua blobs) (I may provide a shell script to precompilation in a future). (One thing PHP does better is the error reporting, that's one thing I will consider doing in a future for lua-resty-template).

Here is a mod_php style config for lua-resty-template using OpenResty:

https://github.com/bungle/lua-resty-template#lua-server-pages-lsp-with-openresty

You may also embed say etlua inside lua-resty-template if you wish, here is an example of embedding Markdown (or Hoedown):

https://github.com/bungle/lua-resty-template#embedding-markdown-inside-the-templates

etlua is also a great choice, and check Leaf's Lapis as well:

http://leafo.net/lapis/

Regards

Aapo

aapo.talvensaari

(PS I'm aware of the new LuaJIT string buffer support, but I haven't had a chance to implement, test, and benchmark yet. The above code should still be plenty fast)

I'm aware of this as well, but there doesn't seem to be any Lua side APIs for that or documentation. There is table.new and table.clear, but I have not seen string buffer API (not even in the latest LuaJIT v2.1 sources).

Regards

Aapo

agentzh

Hello!

On Sat, Dec 27, 2014 at 7:04 AM, Aapo Talvensaari wrote:
>> (PS I'm aware of the new LuaJIT string buffer support, but I haven't had a
>> chance to implement, test, and benchmark yet. The above code should still be
>> plenty fast)
>
> I'm aware of this as well, but there doesn't seem to be any Lua side APIs
> for that or documentation. There is table.new and table.clear, but I have
> not seen string buffer API (not even in the latest LuaJIT v2.1 sources).
>

The new string.buffer API for LuaJIT v2.1 is still a TODO. It's a part
of the phase #4 work sponsored by CloudFlare but Mike Pall has been
busy with other unrelated projects.

So the state of art right now is still to use a Lua table and table.concat().

Regards,
-agentzh