Asking for some basic nginx lua programming tips...

highclass99 · 2015-10-03T12:53:08+00:00

Hello! On Fri, Oct 2, 2015 at 9:26 PM, Aapo Talvensaari wrote: > I should know better, but it would help me a bit if someone could clarify > the diffe...

Asking for some basic nginx lua programming tips...

highclass99

Hello,

I not a heavy programmer, but have enough experience to make basic programs and scripts, including fully functioning socket/websocket servers.

(mainly I don't have real c and dtrace/strace experience)

I've finished through agentzh's tutorial, and if agentzh's reading this... thanks, it's really good and helpful for beginners.

I'm not sure if it's finished but I hope you keep up writing such good tutorials!

Thank you agentzh!

I'm starting testing nginx lua.

As I work usually on windows and am not using vim/vi, I'm curious on the following

I noticed if I write garbage in nginx lua like

location /test2 {

content_by_lua '

this will not run but nginx.conf will pass!!

}

nginx will still start as lua isn't parsing it until test2 is requested.

Do nginx lua developers have to test and run lua script block through the browser each time to see if they made mistakes?

Is there any way to do at least simple syntax checks beforehand?

Is the nginx lua development process for checking errors:

Create nginx config and lua scripts

Run through browser and check nginx error log?

Is there a better way?

I usually use a simple text editor called editplus on windows.

Is there some good lua ide or editor for windows that people recommend for nginx lua development?

I know some of these questions are pretty basic, but I am not a hardcore developer so please understand.

Thanks.

info

It will display an error once you run Lua, nginx is not suppose to check syntax here.
Testing is best done with config reloads with nginx and Curl to check what you want, we often add debug lines which writes to a separate logfile or adds stuff to a Curl output, mostly used to check what we assume is happening is really happening. Next to that we use Lua/delay with ab and openload to test for blocking issues, highload concurrency testing.

agentzh

Hello!

On Sat, Oct 3, 2015 at 12:53 PM, Jason Kim wrote:
> I've finished through agentzh's tutorial, and if agentzh's reading this... thanks, it's really good and helpful for beginners.
> I'm not sure if it's finished but I hope you keep up writing such good tutorials!
> Thank you agentzh!

Oh glad you like them. Yeah, I want to continue writing those
tutorials with more advanced topic :)

> 1.
> I noticed if I write garbage in nginx lua like
> location /test2  {
> content_by_lua '
> this will not run but nginx.conf will pass!!
> ';
> }
>
> nginx will still start as lua isn't parsing it until test2 is requested.
> Do nginx lua developers have to test and run lua script block through the
> browser each time to see if they made mistakes?

Yeah, this is something I want to improve in the near future, that is,
loading the Lua source at nginx configuration loading time, instead of
the first request time.

> Is there any way to do at least simple syntax checks beforehand?
>

This will land into a new ngx_lua module (and openresty) release soon.
I've been relying on the test toolchain to catch such things quickly
during development (see below).

> 2.
> Is the nginx lua development process for checking errors:
> Create nginx config and lua scripts
> Run through browser and check nginx error log?
> Is there a better way?
>

I use a test driven manner for Lua (and even C!) development in the
OpenResty and Nginx context. Basically I've been using (and
maintaining) the Test::Nginx::Socket test scaffold, which is highly
declarative and quite featureful:

https://metacpan.org/pod/Test::Nginx::Socket

The test toolchain can automatically scan the error log file and print
out any errors (well, you can specify a pattern to fit your own
varying needs) to the console while running the test.

Oh yeah, I should add a chapter on this to my nginx tutorials :)

> 3.
> I usually use a simple text editor called editplus on windows.
> Is there some good lua ide or editor for windows that people recommend for
> nginx lua development?
>

Vim? Oh, that's my favourite "IDE". Many years ago, while I was still
working on Windows occasionally, I use EditPlus for coding (not for
OpenResty development, for course, since OpenResty did not exist that
time). IMHO, just pick up a good editor or IDE that has moderate Lua
syntax highlighting support and etc.

> I know some of these questions are pretty basic, but I am not a hardcore
> developer so please understand.
>

Totally understood :)

Regards,
-agentzh

bellasys

Hello Jason,

I am a fellow programmer new to Openresty and Lua, although I'm more familiar to NGINX.

I began 6 weeks ago intending to create a specialized application server that would serve as a backend proxy end-point, as well handle significant load from http requests.

I have been extremely pleased with Openresty, although for the scale of my application, building a custom module in a "foreign" language has been daunting, but I think incredible considering my timeline.

Here are some notes on how I survived. Hope they help

1. From the page below you can get 2 things. I highly recommend Notepad++ for Windows. I also use Aptana Studio 3, which can organize my files but not achieve syntax checking and support for Lua. I do this because I develop primarily on Mac, but reference builds on Windows and Linux frequently within my network. Aptana glides between Mac/PC. You may not need it, although I consider Notepad++ indispensable.

NP++ has Lua support plus a whole lot more. If it isn't enabled by default you enable it in preferences as I have done with a multitude of (even more obscure) languages...

http://lua-users.org/wiki/LuaEditorSupport

I have found the Man pages for Lua 5.1 lacking in example. I get some more out of Lua 5.0 docs, but I have continuously found fundamental answers on the lua-users.org site in general.

2. When developing my Module I was struggling to find a single resource that would list the "phases" of Nginx with correlations to Lua. I finally found this which helped bring several points together- hopefully you find it useful as well:

http://www.nginxguts.com/2011/01/phases/

plus ngx_lua reference

https://github.com/openresty/lua-nginx-module#directives

I have more work to do to compile a single flowchart for my own purposes, but I may do this after my initial deadline and share it. Anyone else have a single resource like a flowchart to understand the phases?

3. Finally, there are a few Gotchas when coming from other programming environments. I am used to wiping my brain of syntax assumptions because I take on a lot of new languages, but I found this resource very helpful along with following best practices as outlined by the Lua man pages & other resources:

http://www.luafaq.org/gotchas.html

Hopefully these items help you get started.

Comments:

I have setup Openresty (nginx v.1.9.3) on Debian 8 (Jessie) with PostgreSQL 9.4. Without Openresty, I find it hard to get beyond Nginx 1.7 on Jessie. I have now built so quickly that I am now a raving fan of Lua and Openresty both.

In my development I had to try a few things before I got it right. I compiled lua-zlib module, and included libpng, and used some other modules for various aspects of my project thinking that these would work for me. For my custom app, I ended up using zlib and libpng as modules using the FFI bindings of LuaJIT so that I could use C-like code for my custom functions without requiring gd or imagick which would normally be used. Since I have a highly specialized app using custom created PNG's that do not require full featuresI felt like I needed a stripped-down interface to maximize speed where I could, particularly since using Postgres.

I feel as though I have developed a very robust app and the environment to support it in a very, very short period of time based on how it performs. I also wrangled with ngx_postgres and pgmoon - I prefer the latter to interface with postgres. Otherwise I have the custom Nginx conf that I've developed during the past few years to handle my infrastructure and make the dual backend/frontend of the app run smoothly, handling 10-20r/s.

regards,

bellasys

bogdan

Great job, Max

The resources you shared with us are great.
As a side note, you could consider trying Microsoft VS Code as an IDE, which is now open source. It is in a beta state, but the development goes pretty fast and the result is on my like. It has GitHub integration, which is great for me. And works on Windows, Linux and Mac too. It has built-in support for Lua (syntax highlighting) and there is a module that adds Lua snippets.
Please keep in mind that I am in no way related with Microsoft or VS Code, but I just like it and I thought I should share this experience.

Regarding yor database, if it is possible, take a look at non-relational databases. Redis seems to perform very well with OpenResty and is a nice piece of software too.

Just my two cents ideas :)

Kind regards,

Bogdan

Max Bellasys wrote:

Hello Jason,

I am a fellow programmer new to Openresty and Lua, although I'm more
familiar to NGINX.

I began 6 weeks ago intending to create a specialized application
server that would serve as a backend proxy end-point, as well handle
significant load from http requests.

I have been extremely pleased with Openresty, although for the scale
of my application, building a custom module in a "foreign" language
has been daunting, but I think incredible considering my timeline.

Here are some notes on how I survived. Hope they help

1. From the page below you can get 2 things. I highly recommend
Notepad++ for Windows. I also use Aptana Studio 3, which can organize
my files but not achieve syntax checking and support for Lua. I do
this because I develop primarily on Mac, but reference builds on
Windows and Linux frequently within my network. Aptana glides between
Mac/PC. You may not need it, although I consider Notepad++ indispensable.

NP++ has Lua support plus a whole lot more. If it isn't enabled by
default you enable it in preferences as I have done with a multitude
of (even more obscure) languages...

http://lua-users.org/wiki/LuaEditorSupport
<http://lua-users.org/wiki/LuaEditorSupport>

I have found the Man pages for Lua 5.1 lacking in example. I get some
more out of Lua 5.0 docs, but I have continuously found fundamental
answers on the lua-users.org <http://lua-users.org/> site in general.

2. When developing my Module I was struggling to find a single
resource that would list the "phases" of Nginx with correlations to
Lua. I finally found this which helped bring several points together-
hopefully you find it useful as well:

http://www.nginxguts.com/2011/01/phases/
<http://www.nginxguts.com/2011/01/phases/>

plus ngx_lua referen ce

https://github.com/openresty/lua-nginx-module#directives
<https://github.com/openresty/lua-nginx-module#directives>

I have more work to do to compile a single flowchart for my own
purposes, but I may do this after my initial deadline and share it.
Anyone else have a single resource like a flowchart to understand the
phases?

3. Finally, there are a few Gotchas when coming from other programming
environments. I am used to wiping my brain of syntax assumptions
because I take on a lot of new languages, but I found this resource
very helpful along with following best practices as outlined by the
Lua man pages & other resources:

http://www.luafaq.org/gotchas.html <http://www.luafaq.org/gotchas.html>

Hopefully these items help you get started.

Comments:

I have setup Openresty (nginx v.1.9.3) on Debian 8 (Jessie) with
PostgreSQL 9.4. Without Openresty, I find it hard to get beyond Nginx
1.7 on Jessie. I have now built so quickly that I am now a raving fan
of Lua and Openresty both.

In my development I had to try a few things before I got it right. I
compiled lua-zlib module, and included libpng, and used some other
modules for various aspects of my project thinking that these would
work for me. For my custom app, I ended up using zlib and libpng as
modules using the FFI bindings of LuaJIT so that I could use C-like
code for my custom functions without requiring gd or imagick which
would normally be used. Since I have a highly specialized app using
custom created PNG's that do not require full featuresI felt like I
needed a stripped-down interface to maximize speed where I could,
particularly since using Postgres.

I feel as though I have developed a very robust app and the
environment to support it in a very, very short period of time based
on how it performs. I also wrangled with ngx_postgres and pgmoon - I
prefer the latter to interface with postgres. Otherwise I have the
custom Nginx conf that I've developed during the past few years to
handle my infrastructure and make the dual backend/frontend of the app
run smoothly, handling 10-20r/s.

--
regards,
bellasys.

bellasys

Hello Bogdan,

thanks for your reply.

I feel the need for ACID compliance on this particular app since it is an SSO server, although I have rarely such need.

I will definitely look at Redis which I noticed from the OpenResty build notes. That's a great suggestion and other servers in my group will undoubtedly have use for it. Some of these servers were slated for Node.js apps and MongoDB which I am both fond of and familiar with- however, since discovering Lua/Nginx together in the OpenResty environment and realizing that I can both serve static sites and connect properly with my site-hubs without using other software, I am interested in evaluating Redis in production. It's just a matter of whether it will work efficiently in my (already developed and tested) architecture, or if small changes to that architecture will be productive in themselves.

Thanks again for your comments & reflections. Very helpful.

--bellasys