HOWTO : MySQL and XCache performance tuning on Ubuntu 10.10

The following settings is for tuning the MySQL database performance on Ubuntu 10.10. It is well tested on Intel Xeon 4 core CPU x 2 and 8 GB RAM. It is also well tested on Drupal 6.2.

Step 1 :

sudo nano /etc/mysql/my.cnf

Change the values as the following :

[mysqld_safe]
nice = -5

[mysqld]
key_buffer = 384M
thread_cache_size = 384
max_connections = 500
table_cache = 1800
# If you have 8 cores CPU, the value should be 16 (no. of processor x 2)
thread_concurrency = 16
query_cache_limit = 4M
query_cache_size = 128M

[isamchk]
key_buffer = 64M

Step 2 :

sudo restart mysql

Step 3 (Optional) :

If you are using Hiawatha Web Server and PHP5, the following settings will further tune the performance also.

sudo apt-get install php5-xcache

Step 3a (Optional) :

sudo nano /etc/php5/conf.d/xcache.ini

Change the value of xcache.size as the following :

xcache.size = 64M

Step 3b (Optional) :

sudo /etc/init.d/php-fcgi restart

Step 3c (Optional) :

To test if it is working or not :

sudo php-cgi -v

If you can see the "with XCache v1.3.0, Copyright (c) 2005-2009, by mOo", your xcache is working.

That's all! See you.

HOWTO : Highest secured Hiawatha Web Server 7.4 on Ubuntu 10.10 Server

Hiawatha is a very secure and fast web server in the market. It supports PHP, Perl, Python and Ruby. It is also very lightweight, easy to configure and setup too. For the performance, please refer to the study of SaltwaterC at here.

This tutorial is writing for setting up the highest secured web server. Please also to apply the "Optional" steps mentioned below for making the highest secured web server.

Prerequisite

Select OpenSSH and Mail Server when installing Ubuntu Server 10.10.

Update the fresh install system to the latest status.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade

If the kernel or kernel modules have been updated, you are required to reboot your system before going further.

Step 1 - Installation of PHP5 and MySQL

sudo apt-get install mysql-server mysql-client php5-cgi php5 php5-cli php5-mysql php5-curl php5-gd php5-idn php-pear php5-imagick php5-imap php5-mcrypt php5-memcache php5-ming php5-ps php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl php5-xcache php5-suhosin php5-ffmpeg apache2-utils mini-httpd

Step 1a - Installation of PHP5 and MariaDB (Alternative)

Since MySQL is now owned by Oracle, the developers of previous MySQL reformed and developed MariaDB under GPL v2. It is compatible to MySQL and running much faster than MySQL too. You can use MariaDB as alternative. The commands and API are the same, such as "mysql -u root -p".

Prepare for installation of MariaDB
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 1BB943DB

sudo nano /etc/apt/sources.list.d/mariadb.list

Append the following lines.

deb http://mirrors.xmission.com/mariadb/repo/5.2/ubuntu maverick main
deb-src http://mirrors.xmission.com/mariadb/repo/5.2/ubuntu maverick main

Save the change and install the following.

sudo apt-get update

sudo apt-get install mariadb-server mariadb-client php5-cgi php5 php5-cli php5-mysql php5-curl php5-gd php5-idn php-pear php5-imagick php5-imap php5-mcrypt php5-memcache php5-ming php5-ps php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl php5-xcache php5-suhosin php5-ffmpeg apache2-utils mini-httpd

** You can also replace your MySQL to MariaDB in this way. The MySQL will be uninstall automatically. You can also use the my.cnf of MySQL without any problem or replace by the new one.

Step 2 - Installation of Hiawatha

Install required dependenices for Hiawatha.

sudo apt-get install libc6-dev libssl-dev dpkg-dev debhelper fakeroot libxml2-dev libxslt1-dev

Download the latest version of Hiawatha (the current version at this writing is 7.4).

wget http://www.hiawatha-webserver.org/files/hiawatha-7.4.tar.gz
tar -xzvf hiawatha-7.4.tar.gz
cd hiawatha-7.4

./configure
make deb

cd ..

sudo dpkg -i hiawatha_7.4_amd64.deb

or

sudo dpkg -i hiawatha_7.4_i386.deb

Step 3 - Configure PHP5 (Optional for security only)

The following settings are for making PHP5 more secure.

sudo nano /etc/php5/cgi/php.ini

Make changes as is.

cgi.rfc2616_headers = 1

zlib.output_compression = On
zlib.output_compression_level = 6

display_errors = Off
log_errors = On
allow_url_fopen = Off
safe_mode = On
expose_php = Off
enable_dl = Off
disable_functions = system, show_source, symlink, exec, dl, shell_exec, passthru, phpinfo, escapeshellarg, escapeshellcmd

*some PHP applications may require safe_mode = off

Step 4 - Configure php-fcgi (PHP's FastCGI)

sudo nano /etc/hiawatha/php-fcgi.conf

Uncomment the following line and change it as is.

Server = /usr/bin/php5-cgi ; 127.0.0.1:2005 ; www-data ; /etc/php5/cgi/php.ini

sudo php-fcgi -c /etc/hiawatha/php-fcgi.conf

If you make any change on php-fcgi.conf, make sure to restart it by the following commands.

sudo php-fcgi -k -c /etc/hiawatha/php-fcgi.conf
sudo php-fcgi -c /etc/hiawatha/php-fcgi.conf

Step 5 - Configure Hiawatha (Part 1)

sudo nano /etc/hiawatha/hiawatha.conf

Uncomment ServerId at GENERAL SETTINGS.

ServerId = www-data

Uncomment the following entries at BINDING SETTINGS.

Binding {
   Port = 80
#   Interface = 127.0.0.1
   MaxKeepAlive = 30
   TimeForRequest = 3,20
}

Step 5a (Optional for security purpose) :

Add the following line at the GENERAL SETTINGS.

ConnectionsTotal = 1000
ConnectionsPerIP = 30
SystemLogfile = /var/log/hiawatha/system.log
GarbageLogfile = /var/log/hiawatha/garbage.log
ExploitLogfile = /var/log/hiawatha/exploit.log

LogFormat = extended
ServerString = Apache
CGIwrapper = /usr/sbin/cgi-wrapper

Make changes for the following entries at BANNING SETTINGS.

BanOnGarbage = 300
BanOnMaxPerIP = 300
BanOnMaxReqSize = 300
BanOnTimeout = 300
KickOnBan = yes
RebanDuringBan = yes

BanOnDeniedBody = 300
BanOnSQLi = 300
BanOnFlooding = 30/1:300
BanlistMask = deny 192.168.0.0/24, deny 127.0.0.1

KickOnBan = yes
ReconnectDelay = 3

Step 5b :

The entries at COMMON GATEWAY INTERFACE (CGI) SETTINGS should be looking like this.

CGIhandler = /usr/bin/perl:pl
CGIhandler = /usr/bin/php5-cgi:php
CGIhandler = /usr/bin/python:py
CGIhandler = /usr/bin/ruby:rb
CGIhandler = /usr/bin/ssi-cgi:shtml
CGIextension = cgi

FastCGIserver {
   FastCGIid = PHP5
   ConnectTo = 127.0.0.1:2005
   Extension = php, php5
   SessionTimeout = 30
}

Step 5c :

Add the following line at VIRTUAL HOSTS.

Include /etc/hiawatha/enable-sites/

*Make sure the make a directory enable-sites under /etc/hiawatha.

sudo mkdir /etc/hiawatha/enable-sites

Step 6 - Configure Hiawatha (Part 2)

If your domain is mysite.com, you are required to create a file namely mysite.com and place it under /etc/hiawatha/enable-sites/mysite.com.

VirtualHost {
   Hostname = www.mysite.com, mysite.com
   WebsiteRoot = /var/www/mysite
   StartFile = index.php
   AccessLogfile = /var/log/hiawatha/access.log
   ErrorLogfile = /var/log/hiawatha/error.log
   TimeForCGI = 15
#   UseFastCGI = PHP5
#   UseToolkit = banshee
   DenyBody = ^.*%3Cscript.*%3C%2Fscript%3E.*$
   ExecuteCGI = yes
   PreventCSRF = yes
   PreventSQLi = yes
   PreventXSS = yes
   DenyBot = Googlebot:/
   DenyBot = twiceler:/
   DenyBot = MSNBot:/
   DenyBot = yahoo:/
   DenyBot = BaiDuSpider:/
   DenyBot = Ask:/
   DenyBot = Yahoo! Slurp:/
   DenyBot = Sogou web spider:/
   DenyBot = Sogou-Test-Spider:/
   DenyBot = Baiduspider+:/
   DenyBot = Yandex:/
   DenyBot = UniversalFeedParser:/
   DenyBot = Mediapartners-Google:/
   DenyBot = Sosospider+:/
   DenyBot = YoudaoBot:/
   DenyBot = ParchBot:/
   DenyBot = Curl:/
   DenyBot = msnbot:/
   DenyBot = NaverBot:/
   DenyBot = taptubot:/
   WrapCGI = jail_mysite
}

Step 7 - Configure Hiawatha (Part 3) (Optional for security purpose)

sudo nano /etc/hiawatha/cgi-wrapper.conf

CGIhandler = /usr/bin/perl
CGIhandler = /usr/bin/php5-cgi
CGIhandler = /usr/bin/python
CGIhandler = /usr/bin/ruby
CGIhandler = /usr/bin/ssi-cgi

Wrap = jail_mysite ; /var/www/mysite ; www-data:www-data

Step 8 - Configure Apparmor (Optional for security purpose)

Execute the following command and then let the web site running for a while, maybe a week or so.

sudo aa-genprof hiawatha

About one week later or the web page/site is misbehaving, issue the following command to update the profile. Remember to reload the profile after the command has been issued.

sudo aa-logprof

Or, if you are impatient, you can edit the following file instead.

sudo nano /etc/apparmor.d/usr.sbin.hiawatha

The content of usr.sbin.hiawatha should look like this or make it look like this.

# Last Modified: Thu Jun 3 01:52:13 2010
#include <tunables/global>

/usr/sbin/hiawatha {
   #include <abstractions/apache2-common>
   #include <abstractions/base>
   #include <abstractions/nameservice>
   #include <abstractions/php5>

   capability chown,
   capability dac_override,
   capability fowner,
   capability fsetid,
   capability setgid,
   capability setuid,

   /bin/dash rix,
   owner /etc/hiawatha/ r,
   /etc/hiawatha/** r,
   /etc/host.conf r,
   /etc/hosts r,
   /etc/mailname r,
   /etc/nsswitch.conf r,
   owner /etc/passwd r,
   /etc/php5/ r,
   /etc/php5/** r,
   /etc/postfix/** r,
   /etc/protocols r,
   /etc/resolv.conf r,
   /etc/services r,
   /etc/snmp/snmp.conf r,
   /sys/devices/system/cpu/ r,
   /tmp/** rwk,
   /usr/bin/php5-cgi rix,
   /usr/lib/postfix/cleanup rix,
   /usr/lib{,32,64}/** mr,
   /usr/sbin/cgi-wrapper rix,
   /usr/sbin/postdrop rix,
   /usr/sbin/sendmail rix,
   /usr/share/ r,
   /usr/share/** r,
   /var/www/ r,
   /var/www/** rwk,
   /var/lib/ r,
   /var/lib/** rw,
   /var/lib/hiawatha/** rw,
   owner /var/log/hiawatha/** w,
   /var/log/hiawatha/** r,
   owner /var/run/ r,
   owner /var/run/** w,
   /var/run/** r,
   /var/spool/postfix/** rw,
   /var/spool/postfix/pid/** wk,
}

Make the profile in enforce mode (activate the above settings).

sudo aa-enforce hiawatha

If you have change some settings, you should reload the profile.

sudo apparmor_parser -r < /etc/apparmor.d/usr.sbin.hiawatha

If you want to disable this profile.

sudo ln -s /etc/apparmor.d/usr.sbin.hiawatha /etc/apparmor.d/disable/
sudo apparmor_parser -R < /etc/apparmor.d/usr.sbin.hiawatha

If you want to re-enable this profile after it has been disabled.

sudo rm /etc/apparmor.d/disable/usr.sbin.hiawatha
sudo apparmor_parser -r < /etc/apparmor.d/usr.sbin.hiawatha

Step 9 - Enhance the security of cgi-wrapper (Optional for security purpose)

Now, your hiawatha is very secure but I would like to make it more secure.

sudo apt-get install libcap2-bin

Apply Capabilities on cgi-wrapper.

sudo chmod u-s /usr/sbin/cgi-wrapper
sudo setcap cap_setgid,cap_setuid+ep /usr/sbin/cgi-wrapper

The result of getcap :

sudo getcap /usr/sbin/cgi-wrapper

It will display :
/usr/sbin/cgi-wrapper = cap_setgid,cap_setuid+ep

Step 10 - Configure logwatch (Optional)

HOWTO : Logwatch for Hiawatha on Ubuntu

Step 11 - Change the ownership of the log files

cd /var/log/hiawatha
sudo chown www-data:www-data access.log
sudo chown www-data:www-data error.log
sudo chown www-data:www-data exploit.log

* "php-fcgi.log" and "system.log" leave them untouched (root:root).

Step 11a - Change ownership of all directories and files at the /var/www/mysite

Put the web application files to /var/www/mysite and then change the ownership of all directories and files under /var/www/mysite to root:root.

cd /var/www/mysite
sudo chown -R root:root *

Step 12 - Start, Stop and Restart Hiawatha

sudo /etc/init.d/hiawatha start
sudo /etc/init.d/hiawatha stop
suod /etc/init.d/hiawatha restart

Step 13 - Performance tuning for MySQL or MariaDB (Optional)

You can fine tune the MySQL or MariaDB as per this link.

Step 14 - Performance tuning for Ubuntu (Optional)

You can fine tune the Ubuntu Server as per this link.

Remarks :

If you encounter "500 Internal Server Error", you may consider to make the Apparmor to "Complain mode".

sudo aa-complain hiawatha

After several days browsing the website, you may consider to turn the Apparmor to "Enforce mode".

sudo aa-logprof

sudo aa-enforce hiawatha

It is because the captioned usr.sbin.hiawatha may not 100% work for you.

That's all! See you.

HOWTO : Most secure web server (Hiawatha 6.17.1) on Ubuntu 9.04 Server

What is Hiawatha?

Hiawatha is a web server that developed by Hugo Leisink since 2002. Hiawatha is not as well known as Apache; however, it has some unique features that Apache lacks of. Apache requires some modules to do the security purpose, such as modsecurity and mod_rewrite. Hiawatha is already built-in. She can ban some bad traffic and bad activities on your web server. Her footprint is also small, that is 130kb, surprise?! She is the default web server for Austrumi and Puppy Linux.

Although the user manual at her official site is not detail enough (at the time of this writing), it is quite easy to configure and runs on a production server. There may be a bug at cgi-wrapper in Hiawatha 6.17.1 and it cannot be configured to run PHP5 in cgi-wrapper mode at the moment. However, perl is no problem.

Hiawatha runs MySQL and PHP great in cgi mode. It can run in Windows environment too (but not yet tried). This tutorial is going to show you how to configure Hiawatha to work with MySQL and PHP.

Installation of Linux, Hiawatha, MySQL and PHP - LHMP

Step 0 - Install Ubuntu 9.04

Install Ubuntu 9.04 Server and OpenSSH. If your web application requires email function, you should also install Mail Server also.

Make sure you have perform the following commands at the terminal (or console).

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade

If the kernel or kernel modules have been updated, you should reboot your computer/server.

Step 1 - Install PHP5 and MySQL

sudo apt-get install mysql-server mysql-client php5-cgi php5 php5-cli php5-mysql php5-curl php5-gd php5-idn php-pear php5-imagick php5-imap php5-mcrypt php5-memcache php5-mhash php5-ming php5-ps php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl

*Note : some modules will not be required, such as php5-sqlite and php5-snmp. If your web application requires them, make sure to install them.

Step 2 - Install Hiawatha

Download the current Hiawatha, 6.17.1 at this time of writing.

sudo wget http://www.hiawatha-webserver.org/files/hiawatha-6.17.1.tar.gz
tar -xzvf hiawatha-6.17.1.tar.gz
cd hiawatha-6.17.1

Install requires dependenices.

sudo apt-get install libc6-dev libssl-dev dpkg-dev debhelper fakeroot libxml2-dev libxslt1-dev

At the hiawatha-6.17.1 directory, build the Hiawatha deb package.

./configure
make deb

The deb package will be created at your home directory, such as /home/samiux. You can install it now.

cd ..

For 64-bit system :
sudo dpkg -i hiawatha_6.17.1_amd64.deb

For 32-bit system :
sudo dpkg -i hiawatha_6.17.1_i386.deb

Step 3 - Configure PHP5

Edit the php.ini.

sudo nano /etc/php5/cgi/php.ini

Make change as is.

display_errors = Off
log_errors = On
allow_url_fopen = Off
safe_mode = On
expose_php = Off
enable_dl = Off
disable_functions = system, show_source, symlink, exec, dl, shell_exec, passthru, phpinfo, escapeshellarg, escapeshellcmd

*Note : some PHP application may requires safe_mode = Off.

Edit Hiawatha's php-fcgi.conf.

sudo nano /etc/hiawatha/php-fcgi.conf

Uncomment the following line.
Server = /usr/bin/php5-cgi ; 127.0.0.1:2005 ; www-data

Activate php-fcgi.

sudo php-fcgi -c /etc/hiawatha/php-fcgi.conf

If you make any change on php-fcgi.conf, make sure to restart it by following commands.

sudo php-fcgi -k -c /etc/hiawatha/php-fcgi.conf
sudo php-fcgi -c /etc/hiawatha/php-fcgi.conf

Step 4 - Configure Hiawatha

Edit the file hiawatha.conf.

sudo nano /etc/hiawatha/hiawatha.conf

Uncomment ServerId at GENERAL SETTINGS.
ServerId = www-data

Add the following line at the GENERAL SETTINGS. Apache compatible log file format.
LogFormat = extended

Uncomment the following entries at BINDING SETTINGS.
Binding {
   Port = 80
   MaxKeepAlive = 30
   TimeForRequest = 3,20
}

Uncomment all the entries at BANNING SETTINGS.
BanOnGarbage = 300
BanOnMaxPerIP = 60
BanOnMaxReqSize = 300
KickOnBan = yes
RebanDuringBan = yes
BanOnSQLi = 0
BanOnFlooding = 10/1:15
BanlistMask = allow 192.168.0.0/24

*Note : Make change to the Banlistmask in order to meet your network requirement.

Uncomment php5-cgi and CGIextension lines.
#CGIhandler = /usr/bin/perl:pl
CGIhandler = /usr/bin/php5-cgi:php
#CGIhandler = /usr/bin/python:py
#CGIhandler = /usr/bin/ruby:rb
#CGIhandler = /usr/bin/ssi-cgi:shtml
CGIextension = cgi

Uncomment all the entries of FastCGIserver and rename ConnectTo to 127.0.0.1:2005.

FastCGIserver {
   FastCGIid = PHP5
   ConnectTo = 127.0.0.1:2005
   Extension = php, php5
   SessionTimeout = 30
}

Optional - Create the following lines under URL TOOLKIT.

UrlToolkit {
   ToolkitID = CMS_common
   RequestURI isfile Return
   RequestURI exists Return
   Match ^/(favicon.ico|robots.txt|sitemap.xml)$ Return
   Match .*\?(.*) Rewrite /index.php?$1
   Match .* Rewrite /index.php
}

*Note : UrlToolkit is similar to Apache's mod_rewrite.

Create a VirtualHost for your site.

VirtualHost {
   Hostname = samiux.blogspot.com
   Alias = /php_my_admin:/usr/share/phpmyadmin
   WebsiteRoot = /var/www/blog
   StartFile = index.php
   AccessLogfile = /var/log/hiawatha/blog/access.log
   ErrorLogfile = /var/log/hiawatha/blog/error.log
   TimeForCGI = 5
   UseFastCGI = PHP5
   UseToolkit = CMS_common
   ExecuteCGI = yes
   PreventCMDi = yes
   PreventCSRF = yes
   PreventSQLi = yes
   PreventXSS = yes
   DenyBot = Googlebot:/
   DenyBot = twiceler:/
   DenyBot = MSNBot:/
   DenyBot = yahoo:/
   DenyBot = BaiDuSpider:/
   DenyBot = Ask:/
   DenyBot = Yahoo! Slurp:/
   DenyBot = Sogou web spider:/
   DenyBot = Sogou-Test-Spider:/
   DenyBot = Baiduspider+:/
   DenyBot = Yandex:/
   DenyBot = UniversalFeedParser:/
   DenyBot = Mediapartners-Google:/
   DenyBot = Sosospider+:/
}

*Note : Some CMS will not well when PreventCMDi = yes. DenyBot entries are optional. If you do not want spiders and bots to crawl your site, you should enable it. Those entries are examples only. UseToolKit is also optional.

Make sure /var/log/hiawatha/blog exists (example) and its ownership is www-data.

If not, make it as is.
sudo chown -R www-data:www-data /var/log/hiawatha/blog

Restart Hiawatha.
sudo /etc/init.d/hiawatha restart

Now, make sure the ownership of access.log and error.log are www-data. If not, make them as is.

sudo chown www-data:www-data /var/log/hiawatha/blog/*

Step 5 - Configure Apparmor (to make Hiawatha more safety)

Create Apparmor profile for Hiawatha.
sudo aa-genprof hiawatha

Edit the profile usr.sbin.hiawatha.
sudo nano /etc/apparmor.d/usr.sbin.hiawatha

Make the entries look like this.

# Last Modified: Tue Sep 1 10:28:15 2009
#include <tunables/global>

/usr/sbin/hiawatha {
   #include <abstractions/base>

   capability chown,
   capability dac_override,
   capability net_bind_service,
   capability setgid,
   capability setuid,
   capability sys_chroot,

   network inet tcp,

   /etc/group r,
   /etc/hiawatha/** r,
   /etc/nsswitch.conf r,
   /etc/passwd r,
   /usr/bin/php5-cgi rix,
   /usr/sbin/cgi-wrapper mr,
   /usr/sbin/hiawatha mr,
   /usr/share/dbconfig-common/** r,
   /usr/share/phpmyadmin/ r,
   /usr/share/phpmyadmin/** r,
   /var/lib/** r,
   /var/lib/hiawatha/* rw,
   /var/log/hiawatha/* r,
   /var/log/hiawatha/** rw,
   /var/log/hiawatha/blog/* r,
   /var/log/hiawatha/blog/** a,
   /var/run/hiawatha.pid w,
   /var/www/ r,
   /var/www/** rw,
}

Make the profile in enforce mode (active).
sudo aa-enforce hiawatha

If you have change some settings, you should reload the profile.
sudo apparmor_parser -r < /etc/apparmor.d/usr.sbin.hiawatha

If you want to disable this profile.
sudo ln -s /etc/apparmor.d/usr.sbin.hiawatha /etc/apparmor.d/disable/
sudo apparmor_parser -R < /etc/apparmor.d/usr.sbin.hiawatha

If you want to re-enable this profile after it has been disabled.
sudo rm /etc/apparmor.d/disable/usr.sbin.hiawatha
sudo apparmor_parser -r < /etc/apparmor.d/usr.sbin.hiawatha

Step 6 - Configure CGI-Wrapper

To be continue ....

Reference :
Hiawatha Manual
Hiawatha Features
AppArmor

That's all. See you!