Hello!
On Mon, Jan 4, 2016 at 7:53 AM, Clodius, Brandon wrote:
> What’s the process of validating compatibility with the latest version of
> nginx (1.9.9) for example? Is it more than running the test suite?
>
The process is usually like this:
1. Add a no-pool patch for the new NGINX core in the no-pool-nginx
repos (for testing purposes):
https://github.com/openresty/no-pool-nginx
2. In the ngx_openresty repos, run the ./util/upgrade-patches.pl
script to update OpenResty's patches for the NGINX core, update the
NGINX version number in util/ver, and then run "make" to generate a
new OpenResty source tarball in the current working directory. Resolve
conflicts (if any) while running "make". That is,
cd ~/git/ngx_openresty
vim util/ver # update the "main_ver" value
./util/upgrade-patches.pl 1.9.9 # 1.9.9 is just an example
make
3. Run all the components' test suites in all the various testing
modes on Linux x86_64 and Linux i386 on Amazon EC2 as demonstrated
here:
https://qa.openresty.org/
The detailed rules for setting up the testing environment and running
tests in different modes are here:
https://github.com/openresty/opsboy/blob/master/samples/ortest-ec2.ob.tt
The script is written in a DSL defined by opsboy but should be
self-explanatory. On my side, it's usually as simple as a few
commands:
cd ~/git/opsboy/misc/logs
./dispatcher -r -t 270 -a 'linux x86_64' -f t tr tw tv trv twv th thv tl to
./dispatcher -r -t 270 -a 'linux i386' -f t tr tw trv tv twv th thv tl to
The letters after "t" are testing mode names. For example, "r" means
mockeagain reading mode, "w" means mockeagain testing mode, "h" means
HUP reload mode, "l" means brute force leak check mode, "o" means
special C compiler optimizing flags, and "v" means valgrind memcheck
mode. Different testing modes can be combined, as in "trv", meaning
mockeagain reading mode combined with valgrind memcheck mode.
It usually takes a dozen moderate EC2 instances (c3.large for x86_64
boxes and c1.medium for i386) to run 3 ~ 5 hours in parallel.
The EC2 instances quit automatically and upload the test data to a
central place (my agentzh.org server). And then from my laptop, I
generate the final HTML report from all the test data:
cd ~/git/opsboy/misc/logs
make pull
make gen
The report's entry is html/index.html, which can be opened locally by
a web browser. This is exactly what you are seeing on the
qa.openresty.org site.
4. Run CloudFlare Lua CDN's applications' test suites. These are not
open, for obvious reasons :)
5. Run simple sanity tests on all the officially supported operating
systems (Mac OS X, FreeBSD, Solaris, Linux, and Windows). The simple
tests are performed in the following script:
https://github.com/openresty/ngx_openresty/blob/master/util/dist-check
For most users, the step 1 should usually be sufficient.
Please let me know if you have any further questions.
Best regards,
-agentzh
