Monday, August 30, 2010

XtraBackup 1.3b

I have installed the newer version of XtraBackup, a tool for online backup of MySQL databases. The first tests show that a lot of errors were fixed in this, not yet released, version. Version 1.2 failed to make incremental backups on one of my servers, could not restore full backup on another, and segfaulted during full backup on the third one. 1.3b makes copies of all three and is able to restore them flawlessly.

A 233GB database was copied in 2:15'. I highly recommend you upgrade your installation of Xtrabackup to v.1.3b and to test it on production servers.

Download page at Percona.com is here: XtraBackup-1.3-beta. I usually download x86_64 binary files for Linux here.

Monday, August 23, 2010

Gaps in graphs generated by rrdtool

I sometimes see questions like: "Why are there gaps in my Munin graphs?" or "How do I get rid of holes in Nagiosgraph charts?". Some classical answers may be found in Munin FAQ or in 'man munin.conf'.

There is one more option, though. The problem may be caused by buggy plugins. By rounding errors in the plugins, to be precise.

From my experience, the gaps are usually found in the graphs where the data is a sum of some indicators. For example, the CPU load graph. The maximum value is calculated as number-of-cpus*100%. The actual value is calculated as the sum of a number of values: the time spent by the CPU in user mode, in system mode, waiting for I/O, in idle state, etc. Under certain conditions, this sum may exceed 100%. In this case, Rrdtool will omit the measurement from the graph and there will be a hole.

To get rid of the gaps, for example, in Munin's 'cpu' plugin, I replaced the lines

        PERCENT=$(($NCPU * 100))
        MAX=$(($NCPU * 100))

with

        PERCENT=$(($NCPU * 100 + 200))
        MAX=$(($NCPU * 100 + 200))

200 might be an overkill, but it looked a bit better in my Munin :). Of course, another option is to check the actual data for being not greater than the max value.

Monday, August 9, 2010

Using variables in complex configurations of Nginx

There are some situations for which Nginx config file does not provide adequate solutions. Very often, using variables, both built-in and user-defined, might help.

For example, Nginx does not support complex conditions, neither as logical operations nor as nested if's. The solution is to define a new variable. If you want a certain URL rewrite to take place only if the Referer header contains "somedomain.com" OR is empty, you can write:

$dorewrite = 'no';
if ($http_referer ~* somedomain.com) {
       set $dorewrite 'yes';
}
if ($http_referer = '') {
       set $dorewrite 'yes';
}
if ($dorewrite = 'yes') {
       rewrite ^ new-request;
}

If you want to rewrite the URL if both conditions are true, for example, the Referer header contains "somedomain.com" AND there's a GET parameter 'id' equal to '12345', modify the snippet above using the De Morgan's law:

$dorewrite = 'yes';
if ($http_referer !~* somedomain.com) {
       set $dorewrite 'no';
}
if ($arg_ID != '12345') {
       set $dorewrite 'no';
}
if ($dorewrite = 'yes') {
       rewrite ^ new-request;
}

Note the variable $arg_ID. This is a built-in variable. There is one such variable for every GET parameter in the request. Its name is composed of $arg_ and the name of the parameter. If you know all parameters you need to serve the request, these variables will help you to extract and pass them around.

If you use complicated regular expressions which you'd like to simplify, define another variable that would contain a part of the regexp:

if ($arg_ID ~* ^doc-number-([-a-z0-9]*)$) {
        set $newarg $1;
}
if ($dorewrite = 'yes') {
        rewrite ^ /doc/view/$newarg? break;
}