I recently updated to the newest version, 9.10 AKA Karmic Koala and I love it. It’s fast and everything works perfectly. Or nearly everything…

The one thing that really annoyed me in this version was the MySQL terminal. All of a sudden after the upgrade ctrl+r for reverse search didn’t work anymore. And there was more…delete didn’t work either and esc+backspace was behaving weirdly too. Unfortunately I did not have time to look for a solution because of my work load. But then xmas came and here I am, stuck at home with the family, giving me a lot of time to research about those things that were very low on my priorities list. And I found a solution!

It seems that Ubuntu replaced libreadline with libedit. And readline is the library that provides all those cool command line capabilities. So let’s enable it again, but instead of compiling the whole thing again, let’s just use a wrapper, the readline wrapper!

First, let’s install it:

sudo apt-get install rlwrap

That’s it! Now all we need to do is to start mysql command line interface through rlwrap:

rlwrap -a mysql -uUSER -pPASSWORD DBNAME

To make things easier, let’s create an alias in our bashrc file.

vi ~/.bashrc

And add the alias:

alias mysql='rlwrap -a mysql'

Credit: http://ubuntuforums.org/showthread.php?t=1147183

One of the things that stumbled me the most about bazaar is that it does not have a own authentication method. Users can access repositories through ssh, sftp, etc. While this is very usefull if everyone working on the repo has shell access to the server, it does not allow “virtual users” the way svn does. Well, not out of the box anyway… There’s always a workaround. This post tries to explain how to configure Bazaar in the centralized model and set user level permissions to the working directory.

Setup initial repository on server

Init central repository with no working trees, since no-one is going to be committing in the central server. I’m calling it bzr_test.

user@server$ cd /path/to/repo/base
user@server$ bzr init-repo --no-trees bzr_test

Init first branch (trunk) on newly created repo:

user@server$ bzr init bzr_test/trunk

Securing with bzr_access on server

(http://bazaar-vcs.org/BzrAccess)

Firstly, get the bzr sources from launchpad, unpack it and copy the bzr_access script located in the contrib directory to some path, I usually use /usr/local/sbin.

user@server$ cd /tmp
user@server$ wget http://launchpad.net/bzr/2.0/2.0.1/+download/bzr-2.0.1.tar.gz
user@server$ tar zxfv bzr-2.0.1.tar.gz
user@server$ cd bzr-2.0.1
user@server$ cp contrib/bzr_access /path/to/bzr_access

or

user@server$ wget http://bazaar-vcs.org/bzr/bzr.dev.knits/contrib/bzr_access -O /path/to/bzr_access

And make sure the script is executable:

user@server$ chmod +x /path/to/bzr_access

Then, add a system user to be used to access the repository:

user@server$ sudo adduser \
    --system \
    --shell /bin/sh \
    --gecos 'bzr version control' \
    --group \
    --disabled-password \
    --home /path/to/repo \
    bzr_username

Once you created the system user, create a .ssh dir in the user’s home directory (I use the repository’s root)

user@server$ mkdir ~bzr_username/.ssh
user@server$ vi ~bzr_username/.ssh/authorized_keys

And add the following line (one per user) to the user’s authorized_keys file:

# ~bzr_username/.ssh/authorized_keys
command="/path/to/bzr_access /path/to/bzr /path/to/repo username",no-port-forwarding,no-X11-forwarding,no-agent-forwarding CLIENT_PUBLIC_KEY

Parameters:
/path/to/bzr_access = /usr/local/sbin/bzr_access
/path/to/bzr: Can be found by executing whereis bzr. Usually in /usr/bin/bzr
/path/to/repo: /home/development/bzr/bzr_test bzr
username: bzr_username
CLIENT_PUBLIC_KEY: is the client’s public key found in ~/.ssh/id_rsa.pub on the client’s machine

Now, you need to add a bzr_access.conf file to your repo’s root:

user@server$ vi /path/to/repo/bzr_access.conf

# /path/to/repo/bzr_access.conf
[groups]
admins = admin1
devels = user1, user2, user3
 
[/]
@admins = rw
@devels = r

OK! Your repo is initted and secured! User admin1 is an admin and cat read & write to the repo while user1, user2 and user3 are devels and have only read access.

Checkout or branch the repository on the client

user@client$ mkdir bzr_test
user@client$ cd bzr_test
user@client$ bzr co bzr+ssh://bzr_username@melimato.com/trunk my_checkedout_trunk
user@client$ bzr branch bzr+ssh://bzr_username@melimato.com/trunk my_branch

Limitations

Taken from http://bazaar-vcs.org/BzrAccess

Currently, each bzr_access.conf configuration file is limited to specifying the access control for the directory that it is in. As such, to cater for a setup with multiple projects that should be segregated, one must either

Create a user account for each project, thus duplicating the configuration for each user account but allowing each user to maintain just one SSH key pair.

Create a single user account, but require each user to have an SSH key pair for each project. They will then have to select the appropriate private key to access the desired directory. Each of these directories will have its own bzr_access.conf, but there need be only one authorized_keys file.

Create an SSH key pair for each project and have users share private keys. However, removing the access of a user from a given directory will mean revoking a key pair and issuing a new private key to the other users with access to that directory.

Today I was had to move a few web sites from one server to another. As I did not have ssh or ftp access to it, the only way to do it was through Plesk. As I don’t have experience using Plesk, I did not know that Plesk’s built in File Explorer can only be used to manage your web site’s files within the server, but it does not let you download those files. So, looking around Plesk’s admin interface I found the backup feature. That looked promissing, so I clicked on it and it brought me to another page called “Backup Manager”. This page allows you to set up periodic or one time backups. Backups can be saved in Plesk’s local repository or in an external FTP server. So I set up all the FTP details and start the backup. Once the backup finished I tried to open it on a linux box and file-roller gave an error when it tried to extract it. A `file` showed that the file was indeed in gzip format.

$ file mydomain.com_2009.11.02_13.07
mydomain.com_2009.11.02_13.07: gzip compressed data, from Unix, last modified: Mon Nov  2 13:08:11 2009

Being a gzip file I obviously tried to gunzip it, but again no luck

$ gunzip mydomain.com_2009.11.02_13.07
gunzip mydomain.com_2009.11.02_13.07
gzip: mydomain.com_2009.11.02_13.07: unknown suffix -- ignored

After googling around for a quite a while I did not find out what this “unknown suffix” error meant, but instead I found on the Parallels Knowledge Base how to do it. I’m copying/pasting it below for future reference:

II. SECOND WAY:

It can be done using mpack tools to work with MIME files. This packet is included into Debian:

# apt-get install mpack

For other Linux systems you can try to use RPM from ALT Linux:
ftp://ftp.pbone.net/mirror/ftp.altlinux.ru/pub/distributions/ALTLinux/Sisyphus/files/i586/RPMS/mpack-1.6-alt1.i586.rpm
or compile mpack from the sources: http://ftp.andrew.cmu.edu/pub/mpack/.
– Create an empty directory to extract the back up file:
# mkdir recover
# cd recover

and copy backup into it. By default Plesk backup is gzipped (if not, use cat), so run zcat to pass data to munpack to extract content of directories from the backup file:

# zcat DUMP_FILE.gz > DUMP_FILE 
# cat DUMP_FILE | munpack

In result you get the set of tar and sql files that contain domains’ directories and databases. Untar the needed directory. For example if you need to restore the httpdocs folder for the DOMAIN.TLD domain:

# tar xvf DOMAIN.TLD.htdocs

That did it, after untaring the htdocs file I finnaly got what I wanted. Hope that saves someone some time.

Recently, I’ve been doing a few projects using Symfony PHP framework and Bazaar for version control. One of the great features of Symfony is that it generates base classes automatically to interact with the database from a pre-defined schema file. While this is a great feature, every time the schema gets changed, Symfony regenerates all the files thus flagging them as modified in Bazaar. To avoid committing those files all the time, I created a .bzrignore file to ignore them and leave my repository nice and clean.

Firstly, init bazaar for your new project:

bzr init

Next edit your .bzrignore file located in the root dir of your project

vi .bzrignore

And add the following lines:

trunk/cache/*
trunk/log/*
trunk/config/*schema-transformed.xml
trunk/plugins/.*

The above lines are pretty much self-explanatory, the only important thing here to keep in mind is that if you want to ignore all the files inside a directory you firstly have to add the directory to bazaar (bzr add dir) and then ignore everything inside them.
The magic really happens in the following line.
For Doctrine users:

trunk/lib/**/base/*

And Propel

 **/om/*. 
 **/map/*.

This tells bzr to ignore anything inside any directory named base inside the lib directory, and om and map if you’re using propel, which is where all base classes are stored.

This way, your repository will only store the files that you edit. All files generated automatically by Symfony or plugins are ignored, so when checking out a Symfony project you will have to execute:

./symfony doctrine-build-all

or

./symfony propel-build-all

To see which files are currently being ignored use the following command:

bzr ignored

And the output should be something like this:

The first column tells which file is ignored and the second one informs the rule (pattern) used for the match.

trunk/cache/admin                                  trunk/cache/*
trunk/cache/frontend                               trunk/cache/*
trunk/cache/project_autoload.cache                 trunk/cache/*
trunk/lib/filter/doctrine/base/BaseMyClassFormFilter.class.php trunk/lib/**/base/*
trunk/lib/form/doctrine/base/BaseMyClassForm.class.php trunk/lib/**/base/*
trunk/lib/model/doctrine/base/BaseMyClass.class.php trunk/lib/**/base/*
trunk/log/admin_dev.log                            trunk/log/*
trunk/log/frontend_dev.log                         trunk/log/*
trunk/plugins/.channels                            trunk/plugins/.*
trunk/plugins/.depdb                               trunk/plugins/.*
trunk/plugins/.depdblock                           trunk/plugins/.*
trunk/plugins/.filemap                             trunk/plugins/.*
trunk/plugins/.lock                                trunk/plugins/.*
trunk/plugins/.registry                            trunk/plugins/.*
trunk/web/uploads/                     trunk/web/uploads/*

Comments are very much appreciated!

First lets install pureftpd using apt-get:

Install pureftpd with the MySQL backend:

apt-get install pure-ftpd-mysql

Since I like having different log files for different services I edit syslog.conf to separate ftp log files to specific one:

vi /etc/syslog.conf

Add these line at the end of the file:

ftp.* /var/log/ftp

Create the required file and restart sysklogd:

touch /var/log/ftp
/etc/init.d/sysklogd restart

Edit MySQL connection information for pureftpd user:

vi /etc/pure-ftpd/db/mysql.conf

Add or uncomment the following lines in the file. Don’t forget to change the user and password details.

MYSQLSocket      /var/run/mysqld/mysqld.sock
MYSQLUser       pure-ftpd
MYSQLPassword   pure-ftpd_pass
MYSQLDatabase   pureftpd
MYSQLCrypt      cleartext
MYSQLGetPW      SELECT Password FROM users WHERE User="\L"
MYSQLGetUID     SELECT Uid FROM users WHERE User="\L"
MYSQLGetGID     SELECT Gid FROM users WHERE User="\L"
MYSQLGetDir     SELECT Dir FROM users WHERE User="\L"
MySQLTransactions On

Some more configuration settings and pure-uploadscript setup:

echo no > /etc/pure-ftpd/conf/PAMAuthentication
echo yes > /etc/pure-ftpd/conf/CreateHomeDir
echo yes > /etc/pure-ftpd/conf/CallUploadScript
echo yes > /etc/pure-ftpd/conf/ChrootEveryone

Edit PurFtpd’s common settings:

vi /etc/default/pure-ftpd-common

Change / uncomment the following lines:

STANDALONE_OR_INETD=standalone
VIRTUALCHROOT=false
UPLOADSCRIPT=/path/to/pure-ftpd/uploadscript
UPLOADUID=0
UPLOADGID=0

Create MySQL users, apply permissions for the details set on /etc/pure-ftpd/db/mysql.conf . Also create users table for PureFtpd web admin:

CREATE DATABASE pureftpd;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP ON pureftpd.* TO 'pure-ftpd'@'localhost' IDENTIFIED BY 'pure-ftpd_pass';
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP ON pureftpd.* TO 'pure-ftpd'@'localhost.localdomain' IDENTIFIED BY 'pure-ftpd_pass';
FLUSH PRIVILEGES;
CREATE TABLE `users` (
  `User` VARCHAR(32) NOT NULL,
  `Password` VARCHAR(32) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL DEFAULT '',
  `Uid` INT(11) NOT NULL DEFAULT '14',
  `Gid` INT(11) NOT NULL DEFAULT '5',
  `Dir` VARCHAR(128) NOT NULL DEFAULT '',
  `QuotaFiles` INT(10) NOT NULL DEFAULT '500',
  `QuotaSize` INT(10) NOT NULL DEFAULT '30',
  `ULBandwidth` INT(10) NOT NULL DEFAULT '80',
  `DLBandwidth` INT(10) NOT NULL DEFAULT '80',
  `Ipaddress` VARCHAR(15) NOT NULL DEFAULT '*',
  `Comment` tinytext,
  `Status` enum('0','1') NOT NULL DEFAULT '1',
  `ULRatio` SMALLINT(5) NOT NULL DEFAULT '1',
  `DLRatio` SMALLINT(5) NOT NULL DEFAULT '1',
  PRIMARY KEY  (`User`),
  UNIQUE KEY `User` (`User`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1

Create uploadscript.sh referred on previous step:

#!/bin/bash
 
    # UPLOAD_SIZE  : the size of the file, in bytes.
    # UPLOAD_PERMS : the permissions, as an octal value.
    # UPLOAD_UID   : the uid of the owner.
    # UPLOAD_GID   : the group the file belongs to.
    # UPLOAD_USER  : the name of the owner.
    # UPLOAD_GROUP : the group name the file belongs to.
    # UPLOAD_VUSER : the full user name, or the virtual user name. (127 chars max)
 
# Do something useful

Install the PureFTPd user manager:

1. Download the user manager for PureFtpd from here
2. Uncompress the file to a web accessible directory:
3. Point your browser to the chosen directory and open the file install.php
4. Follow the installation steps

And you’re done!

Most web sites I do have many domains but want all those domains to point to the “main domain” www.domain.com. My first solution was to put Apaches’s directive ServerAlias in the site’s virtual host configuration. This lead to content duplication since domain.com and domain.net and www.domain.com are two different web sites for the rest of the world. To avoid this problem the only solution I’ve found was to create two different virtual hosts and set Apache’s Redirect directive. The first VirtualHost will hold all the domains aliases and redirect permanently (301) to the “main domain”

<VirtualHost ip_address:80>
    ServerName domain.com
    ServerAlias domain.net
    ServerAlias domain.biz
    RedirectMatch permanent /(.*) http://www.domain.com/$1
</VirtualHost>
 
<VirtualHost ip_address:80>
    ServerName www.domain.com
    etc...
</VirtualHost>

I’m posting this since I always forget it…

date +%s

More info: http://www.cyberciti.biz/faq/linux-unix-formatting-dates-for-display/

As opposed to rsync, unison is a two way file synchronizer. Similar to MySQL’s master to master replication. To set it up in Ubuntu (Intrepid) keep on reading…

1. Install unison on both servers

apt-get install unison

2. Create a script to add to crontab to call unison periodically

unison -numericids -owner -silent -times -contactquietly -batch -auto -log -logfile /var/log/unison.log profile_file 2>&1

3. Create the above mentioned profile file on ~/.unison/profile_file.prf

root = ssh://server_ip//path/to/root/dir
root = /path/to/root/dir
ignore = Regex .*(cache|Cache|te?mp|logs?).* # ignore log and cache files

That’s it! Now add the following line to your crontab to execute it every 5min

*/5 * * * * /path/to/my_unison_script.sh

After a few days of running this I found that sometimes one unison execution would overlap another. To resolve this I found a little tool called lockrun

To get it to work I did:

1. Download the source code from the official lockrun website

wget http://unixwiz.net/tools/lockrun.c

2. Build/install as shown in http://unixwiz.net/tools/lockrun.html

gcc lockrun.c -o lockrun
sudo cp lockrun /usr/local/bin/

3. Change the previous crontab line to use lockrun

*/5 * * * * /usr/local/bin/lockrun --lockfile=/tmp/hosting.lockrun -- /path/to/my_unison_script.sh

All done!!!

Make svnserve load on startup:

First create a new script on init.d

vi /etc/init.d/svnserve

The script’s contents:

#!/bin/bash
### BEGIN INIT INFO
# Provides:          svnserve
# Required-Start:    $local_fs $remote_fs $network $syslog
# Required-Stop:     $local_fs $remote_fs $network $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start subversion server
### END INIT INFO
 
svnserve --listen-host=myhost.com -d -r /path/to/svn/repos

Then make the script is executable

chmod +x /etc/init.d/svnserve

Use update-rc.d to add script to init

update-rc.d svnserve defaults

That’s it!

I prefer compressing old logs to save space. Edit logrotate.conf

vi /etc/logrotate.conf

Uncomment the following line:

compress

Since I keep all my hosting’s logs in /home/hosting/log I need to add a new task to logrotate:

Create a new file under logrotate.d called hosting:

vi /etc/logrotate.d/hosting

Add the following contents to that file:

/home/hosting/*/log/*.log {
  weekly
    missingok
    rotate 52
    compress
    delaycompress
    notifempty
    create 640 root adm
    sharedscripts
    postrotate
        if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
            /etc/init.d/apache2 reload > /dev/null
        fi
    endscript
}

Explanation of script lines according to man page:

missingok
If the log file is missing, go on to the next one without issuing an error message.

rotate count
Log files are rotated times before being removed or mailed to the address specified in a mail directive. If count is 0, old versions are removed rather then rotated.
In our case we will remove logfiles older than 1 year (52 weeks)

compress
Duh!

delaycompress
Postpone compression of the previous log file to the next rotation cycle. This has only effect when used in combination with compress. It can be used when some program can not be told to close its logfile and thus might continue writing to the previous log file for some time.

notifempty
Do not rotate the log if it is empty (this overrides the ifempty option).

create mode owner group
Immediately after rotation (before the postrotate script is run) the log file is created (with the same name as the log file just rotated). mode specifies the mode for the log file in octal (the same as chmod(2)), owner specifies the user name who will own the log file, and group specifies the group the log file will belong to. Any of the log file attributes may be omitted, in which case those attributes for the new file will use the same values as the original log file for the omitted attributes. This option can be disabled using the nocreate option.

sharedscripts
Normally, prescript and postscript scripts are run for each log which is rotated, meaning that a single script may be run multiple times for log file entries which match multiple files (such as the /var/log/news/* example). If sharedscript is specified, the scripts are only run once, no matter how many logs match the wildcarded pattern. However, if none of the logs in the pattern require rotating, the scripts will not be run at all. This option overrides the nosharedscripts option and implies create option.
postrotate/endscript
The lines between postrotate and endscript (both of which must appear on lines by themselves) are executed after the log file is rotated. These directives may only appear inside of a log file definition.