Relocatable Openresty

ian.biggs.48 · 2015-03-19T23:08:37+00:00

Hello! On Wed, Mar 18, 2015 at 6:52 AM, rvsw wrote: > There is a video > recording of the lightning talk on Monday June 3 2013 that you would have &gt...

Relocatable Openresty

ian.biggs.48

Normally when relocating Nginx, you'd use the -p flag to change the prefix path so that

everything runs neatly relative to that path.

Openresty does not appear to follow that prefix change so you end up running code from the location

specified during the build.

Not too much of a problem switching the package paths around (especially as all environment variables are available in init_by_lua via os.getenv() - guess this runs before Nginx strips them out).,

but the underlying paths (to LuaJIT etc) don't seem so easy to override.

Are there any thoughts about including a relocatable build option as seen in Perl (for instance)?

Alternatively, have everthing follow the -p prefix?

This is using 1.7.10.1.

Looks like I'm going to have to build for each path set I need to deploy to.

Use case is supporting multiple instances on the same server with the possibility

of running different patch levels for each instance, but a common build for any

given patch level.

Any best practive recommendations most welcome.

Thanks.

agentzh

Hello!

On Thu, Mar 19, 2015 at 8:08 AM, Ian Biggs wrote:
> Normally when relocating Nginx, you'd use the -p flag to change the prefix
> path so that
> everything runs neatly relative to that path.
>
> Openresty does not appear to follow that prefix change so you end up running
> code from the location
> specified during the build.
>

This should work in OpenResty because we do not change anything
related to -p handling. Will you provide as many details as possible
so that we can debug the problems you're seeing?

For example, the Getting Started page on openresty.org explicitly
makes use of this feature:

    http://openresty.org/#GettingStarted

Actually use of -p is the recommended way to deploy OpenResty-based
sites and applications :)

Regards,
-agentzh

ian.biggs.48

Thakns for the quick reply ...

This shows the problem:

./configure --prefix=/or1

gmake

gmake install

cd /or1/lualib

create fred.lua, just a hello world

function hifred()

ngx.say("hi from fred")

end

cd /or1/nginx/conf

add following to location /

content_by_lua '

local fred = require("fred")

hifred()

cd ..

./sbin/nginx -p `pwd` -c conf/nginx.conf

This works fine, ./sbin/nginx -s stop

cd /

cp -r /or1 /or2

cd /or2/nginx

./sbin/nginx -p `pwd` -c conf/nginx.conf

This works fine, ./sbin/nginx -s stop

Now move the original instance

mv /or1 /or3

./sbin/nginx -p `pwd` -c conf/nginx.conf

This gives ...

./sbin/nginx: error while loading shared libraries: libluajit-5.1.so.2: cannot open shared object file: No such file or directory

Which suggests to me that despite the fact that i'm in /or2/nginx and using -p that no everything is taking note.

If we put /or3 back as /or1 then it will start ok.

moving /or1/lualib/fred.lua to /or2/lualib/fred.lua results in it not being found. Thats not too bad although not ideal as

I can get around that with lua_package_path.

Hope that explains what I'm seeing.

On Thursday, March 19, 2015 at 3:08:36 PM UTC, Ian Biggs wrote:

Normally when relocating Nginx, you'd use the -p flag to change the prefix path so that
everything runs neatly relative to that path.

Openresty does not appear to follow that prefix change so you end up running code from the location
specified during the build.

Not too much of a problem switching the package paths around (especially as all environment variables are available in init_by_lua via os.getenv() - guess this runs before Nginx strips them out).,
but the underlying paths (to LuaJIT etc) don't seem so easy to override.

Are there any thoughts about including a relocatable build option as seen in Perl (for instance)?
Alternatively, have everthing follow the -p prefix?

This is using 1.7.10.1.

Looks like I'm going to have to build for each path set I need to deploy to.
Use case is supporting multiple instances on the same server with the possibility
of running different patch levels for each instance, but a common build for any
given patch level.

Any best practive recommendations most welcome.

Thanks.

agentzh

Hello!

On Thu, Mar 19, 2015 at 2:12 PM, Ian Biggs wrote:
> This works fine, ./sbin/nginx -s stop
>
> Now move the original instance
>
> mv /or1 /or3
>

No, you should never never move the OpenResty installation tree
because it breaks RPATH links leading to .so breakage (as the runtime
error you already saw). It's never supposed to work.

Instead you should put your application related configurations and
source files in a different separate directory, as demonstrated in the
Getting Started page demo.

Regards,
-agentzh

ian.biggs.48

So the -p flag is pretty much redundant as only the nginx core is affected by it?

I can see a way round it. Given what I need to set up it would certainly be very convenient

to be able to just tar a working installation and dump it where I need it. With the environments

that we run and the way that we run them we'll end up with different paths for each environment.

I'll have to generate path substitutions to move between each environment. Thats not new :)

Seem to spend half my time doing that and not getting on with the actual code.

Many thanks for your time. Much appreciated.

agentzh

Hello!

On Thu, Mar 19, 2015 at 2:44 PM, Ian Biggs wrote:
>
> I can see a way round it. Given what I need to set up it would certainly be
> very convenient
> to be able to just tar a working installation and dump it where I need it.

You can move OpenResty's installation tree across different boxes as long as:

1. The operating system and related system libraries are exactly the
same (or ABI compatible);
2. The new location is exactly the same as the old location.

This is how binary packages in popular Linux distributions work (like
Deb packages or PRM packages).

All the software (not just OpenResty) playing with RPATH or something
like that won't allow moving the installation tree to somewhere
different than the original --prefix=PATH setting.

> With the environments
> that we run and the way that we run them we'll end up with different paths
> for each environment.
>

Copying binary software around is not something considered "good
practice". Try packaging binary software as proper binary packages
like Deb packages or RPM packages instead.

You *could* still move the installation trees yourself, but you must
specify the LD_LIBRARY_PATH environment or configure ldconfig properly
to accommodate your change. Magic won't happen automatically when
OpenResty's DSO libraries are not in your system-wise default library
search paths and you move them to somewhere invisible to the RPATH
settings.

Regards,
-agentzh

joeblow68

Another way is to package Openresty in a Docker container. That works well for me.

Jon

On Thursday, March 19, 2015 at 10:44:53 PM UTC+1, Ian Biggs wrote:

So the -p flag is pretty much redundant as only the nginx core is affected by it?

I can see a way round it. Given what I need to set up it would certainly be very convenient
to be able to just tar a working installation and dump it where I need it. With the environments
that we run and the way that we run them we'll end up with different paths for each environment.

I'll have to generate path substitutions to move between each environment. Thats not new :)
Seem to spend half my time doing that and not getting on with the actual code.

Many thanks for your time. Much appreciated.