Caching with Moodle

adam.r.vest · 2015-10-05T01:41:11+00:00

Hello Bogdan,thanks for your reply.I feel the need for ACID compliance on this particular app since it is an SSO server, although I have rarely such need.&#x...

Caching with Moodle

adam.r.vest

Afternoon all,

I've used this module before with other CMS's with great success, but for the life of me I cannot get Nginx to cache anything from a Moodle installation. I am fairly certain this is due to the Slash Arguments (https://docs.moodle.org/dev/Install_Moodle_On_Ubuntu_with_Nginx/PHP-fpm#Moodle ) function it uses. I believe this since, if I go to a file directly that doesn't come from that (say, README.txt), I see in the log that the request is cached, and I see subsequent requests in memcached return as get_hit's. For the life of me, however, I am unable to get Moodle and this module to play nicely and cache requests for things I direct it to, no matter what hodgepodge of rewrite rules and configurations I use. For reference, I've included my configuration lines below:

server {
    error_log /var/log/nginx/domain.com.err info;
    access_log /var/log/nginx/domain.com.log combined;
    listen       80;
    server_name  domain.com www.domain.com;
    root   /var/www/moodle;
    index index.php;
    rewrite ^/(.*\.php)(/)(.*)$ /$1?file=/$3 last;
    location / {
        try_files $uri $uri/ $uri/index.php /index.php?$args;
    }
    location ~* \.(?:ico|html?|txt|js|css|ttf|svg|woff2?|eot|xml|docx?|xlsx?|pdf)$ {
       set $key $uri$args;
       srcache_fetch GET /memc $key;
       srcache_store PUT /memc $key;
       srcache_store_statuses 200 301 302;
       try_files $uri =404;
    }
    location /memc {
        internal;
        memc_connect_timeout 100ms;
        memc_send_timeout 100ms;
        memc_read_timeout 100ms;
        memc_ignore_client_abort on;
        set $memc_key $query_string;
        set $memc_exptime 300;
        memc_pass unix:/var/run/memcached/memcached.sock;
    }
    location ~ \.php$ {
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        include fastcgi.conf;
        fastcgi_pass unix:/var/run/fpm.sock;
        fastcgi_index index.php;
    }
    location ~ /\.ht {
        deny all;
    }
}

Example of a request that is not cached, but I believe should be:

[04/Oct/2015:13:20:17 -0400] "GET /theme/yui_combo.php?rollup/3.17.2/yui-moodlesimple-min.css HTTP/1.1" 200 1043

Example of a request that is cached:

[04/Oct/2015:13:27:34 -0400] "GET /README.txt HTTP/1.1" 200 760

Versions of software:Nginx: 1.9.5

PHP: 5.6.13
memc-nginx-module: 0.16
srcache-nginx-module: 0.30
OS: CentOS 7.1.1503

Any assistance on getting this all to work correctly (if at all possible) would be appreciated.

Thanks.

agentzh

Hello!

On Mon, Oct 5, 2015 at 1:41 AM, Adam Vest wrote:
> Example of a request that is not cached, but I believe should be:
>
> [04/Oct/2015:13:20:17 -0400] "GET
> /theme/yui_combo.php?rollup/3.17.2/yui-moodlesimple-min.css HTTP/1.1" 200
> 1043
>
> Example of a request that is cached:
>
> [04/Oct/2015:13:27:34 -0400] "GET /README.txt HTTP/1.1" 200 760

Given that you are using try_files heavily in your configuration, my
hunch is that, it's just because that your srcache configurations are
not in the location(s) that actually serve the content. (Note that
try_files essentially initiate "internal redirects" to other locations
per se.)

To make sure, you need to (temporarily) enable the nginx debugging
logs and analyze (or share for us to analyze) all the debugging logs
for the guilty request. Please see

    http://nginx.org/en/docs/debugging_log.html

and

    https://github.com/openresty/srcache-nginx-module#trouble-shooting

Regards,
-agentzh

adam.r.vest

Hey, how about that - a debug log that isn't just a bunch of unintelligible nonsense! Thanks for the tip!

So after turning on the log, I discovered that my rewrite wasn't just rewriting the URI, it was actually splitting off the part I was interested in to $args - hence, my location would never match since it was no longer a part of the URI:

2015/10/06 15:28:42 [notice] 16529#16529: *294 "^/(.*\.php)(/)(.*)$" matches "/lib/_javascript_.php/1443339958/lib/jquery/jquery-1.11.2.min.js"

2015/10/06 15:28:42 [notice] 16529#16529: *294 rewritten data: "/lib/_javascript_.php", args: "file=/1443339958/lib/jquery/jquery-1.11.2.min.js"

Definitely not what I was expecting, but it explained a lot. So after discovering that, I rewrote my configs with that in mind. After a little experimentation to get the order right (so the matching would happen at the proper time), I finally started seeing the results stored and retrieved from within memcached (as evidenced by my increasing "get_hits" metric!). My final working configs for those interested:

server {
    error_log /var/log/nginx/domain.com.err info;
    access_log /var/log/nginx/domain.com.log combined;
    listen       80;
    server_name  domain.com www.domain.com;
    root   /var/www/moodle;
    location / {
        index index.php;
        rewrite ^/(.*\.php)(/)(.*)$ /$1?file=/$3 last;
        location ~ \.php$ {
            fastcgi_split_path_info ^(.+\.php)(/.+)$;
            include fastcgi.conf;
            fastcgi_pass unix:/var/run/fpm.sock;
            fastcgi_index index.php;
            srcache_ignore_content_encoding on;

            # Used to match requests caught by rewrite for slash arguments
            if ($args ~ .*\.(html|htm|txt|js|css|ttf|svg|woff|eot|xml|doc|docx|xls|xlsx|pdf)$) {
                set $key $args;
                srcache_fetch GET /memc $key;
                srcache_store PUT /memc $key;
                srcache_store_statuses 200 301 302;
            }
        }

        # Leaving below in place to account for requests NOT rewritten for slash arguments
        if ($uri ~ .*\.(html|htm|txt|js|css|ttf|svg|woff|eot|xml|doc|docx|xls|xlsx|pdf)$) {
            set $key $uri;
            srcache_fetch GET /memc $key;
            srcache_store PUT /memc $key;
            srcache_store_statuses 200 301 302;
        }
    }

    location /memc {
        internal;
        memc_connect_timeout 100ms;
        memc_send_timeout 100ms;
        memc_read_timeout 100ms;
        memc_ignore_client_abort on;
        set $memc_key $query_string;
        set $memc_exptime 300;
        memc_pass unix:/var/run/memcached/memcached.sock;
    }
    location ~ /\.ht {
        deny all;
    }
}

It's also worth mentioning that I removed the "try_files" stuff. After just adding the "index" directive, I've not seen where "try_files" has been needed - everything seems to work fine. I'm open to further improvements on the above if you see room for it.

Thanks again!

On Sunday, October 4, 2015 at 11:25:47 PM UTC-4, agentzh wrote:

Hello!

On Mon, Oct 5, 2015 at 1:41 AM, Adam Vest wrote:
> Example of a request that is not cached, but I believe should be:
>
> [04/Oct/2015:13:20:17 -0400] "GET
> /theme/yui_combo.php?rollup/3.17.2/yui-moodlesimple-min.css HTTP/1.1" 200
> 1043
>
> Example of a request that is cached:
>
> [04/Oct/2015:13:27:34 -0400] "GET /README.txt HTTP/1.1" 200 760

Given that you are using try_files heavily in your configuration, my
hunch is that, it's just because that your srcache configurations are
not in the location(s) that actually serve the content. (Note that
try_files essentially initiate "internal redirects" to other locations
per se.)

To make sure, you need to (temporarily) enable the nginx debugging
logs and analyze (or share for us to analyze) all the debugging logs
for the guilty request. Please see

http://nginx.org/en/docs/debugging_log.html

and

https://github.com/openresty/srcache-nginx-module#trouble-shooting

Regards,
-agentzh