Hi OpenResty folks,
I'm working on incrementally replacing PHP with Lua/OpenResty in various components of our site. Right now I'm looking into using ImageMagick via leafo/magick to process uploaded images and serve various versions for a responsive site. It'd be very much along the lines of "Creating an image server", which seems to fit our needs really well. We'd want to do more than only resizing images, but all the processing would use ImageMagick functions.
I have all the pieces in place to start prototyping, but I'm thinking ahead about overall performance, as all the info I can find seems to confirm that doing file I/O will block nginx. I've found a few mailing list threads here which talk about blocking I/O, others recommending magick for image processing without mentioning it, and I'd love to get clear on the best approach.
In the magick example, it does this:
-- make sure the file exists
local file = io.open(source_fname)
if not file then
return_not_found()
end
file:close()
[...]
-- resize the image
local magick = require("magick")
magick.thumb(source_fname, size, dest_fname)
This seems like it checks the source file with the 'io' module and leaves all processing to the ImageMagick function, including writing out the resized image to a file. There's no mention of I/O blocking, except in the comments. So I'm wondering:
- Is this just a calculated trade-off, based on comparatively small io:open/close operations, that we should just try out and measure for our use case? Or am I missing something?
- I guess ImageMagick blocks under the hood on file I/O (I've traced through to WriteImage which talks about ThreadSupport but I'm getting a bit out of my depth there). If so, and if the "magick" FFI calls are in-process to nginx, won't ImageMagick file operations like img:write() block nginx too?
- This mailing list thread suggests "caching and sharing the file descriptor in Lua land, to save open and close syscalls"; how might that work here? Would that mostly help out if we were trying to write to the same file from multiple places in our code? Are there some examples I could look at, if this would be appropriate?
- Would piping out to ImageMagick's CLI tool via lua-resty-shell be an alternative? Looks like that relies on forking new sockproc processes for each command though, I guess this is a bit like returning to a process-per-request model in some sense?
- Failing that, would it make any sense to read image data into memory via ngx.location.capture, process them in-memory using magick:load_image_from_blob, and write back the result (of img:get_blob()) via subrequest POSTs to an internal 'upload' URL? Will the processing block in other places anyway? Or is that just plain crazy? :-D
I'd love to get some overall advice on how (/whether) to move this over. Is this sort of thing going to work well with OpenResty, is the tradeoff fine as it is in the examples, is it best done elsewhere, is the tradeoff going to be the same wherever we do it? Or am I worrying unnecessarily about something which works fine? Obviously it depends on how frequently the processing would have to happen, and we'd be caching & serving processed files as much as possible, but there's a lot of image content that would only get processed on request because of nature of the site and the responsive environment.
Thanks very much for any tips or help you can give!
Cheers,
Igor
