These are:

• Logging to DB instead of log files
• Runtime configuration of logging.

Logging to DB instead of log files

Coming from a web development background rather than a sysadmin background I'm far more comfortable manipulating and analysing data using SQL. So to be able to log all the queries or just the slow queries for an application to the db during application development or load testing is a huge benefit.

To enable logging to DB you can add the following to your my.cnf

The logs will be written to the 'slow_log' and 'general_log' tables in the mysql database.

Note - logging to tables has more overhead than logging to file, so would suggest using it primarily for development purposes.

Full details of the options are on the mysql manual on log tables

Runtime configuration of logging.

This allows you to turn on and off logging without restarting MySQL - which just saves a little bit of time and makes it much nicer for debugging problems.

To turn on the logging of all queries run:

And to turn them both off use:

If you also want to see queries not using indexes in the slow query log you can set the following variable:

Hope it helps, Mark

However, this time I finally managed to get past all the hurdles and jump through all the hoops to make this work.

Scenario is a follows. Audio file is comes in in alaw or ulaw format and we want to trim any silence before and after it.

First step is to convert it from alaw to pcm encoded wav file as SOX has issues filtering silence in alaw files

Then we do the trimming in one fell swoop.

A huge thanks to this article for giving such a thorough overview of the how silence detection works in sox.

Too easy, when you know how :-)

This sets turns it on for everything except gif,jpeg,png or mp3, as these are already well compressed.

Cheers, Mark

Running the command df was showing lots of free space.

However, a quick google on cp "no space left on device" turned up the suggestion to try:

This showed up the problem straight away - I had run out of inodes.

Once the problem is identified it is generally easy to resolve, so I figured out that there was a cron job that was running a wget task and not discarding the output, and so had saved 600,000+ files in their home directory.

However, when I tried to delete them I encountered another problem:

There were so many files that I couldn't use standard delete commands.

Another quick google turned up this gem:

I ran the following command to check that I was going to delete the correct files:

One final thing was to fix up the cron job that was causing the problem. Adding the parameter --delete-after to wget kept the directory nice and clean.

All fixed. One of the things this has reinforced for me is how important it is to have /home on a seperate partition. If this had not been the case then the problem would have taken longer to happen (due to larger filesystem) but would have been more destructive - as all systems and processes would not have been able to create new files.

All the DB storage and application was using UTF8 throughout, but the PDF exports were showing nothing when they should have been showing Chinese characters.

To fix this up you need to do the following:

1. Need fonts installed on system:

2. Need fonts installed in CFAdmin

• Go to CFAdmin and Font Management
• Select directory /usr/share/fonts/truetype/arphic
• Select "Add" and you should see 2 new fonts added.

3: CFdocument Hotfix for CF7.0.2

Configure cffont.properties file

The cffont.properties file is located in /opt/jrun4/servers/{instance name}/cfusion.ear/cfusion.war/WEB-INF/cfusion/lib on the jrun multiserver version.

5. Restart CF

Once you restart CF you documents should be coming out with their Chinese fonts intact.

I'm sure there are additional fonts to add to get it working natively with all the other asian languages, so I'll add to this as and when I need/discover them. Please feel free to post a comment if you find any more font's that should be added to the list.

• Automatically fixing file system errors
• Ignoring directories from updatedb

Automatically fixing file system errors

If you have a remote server, i.e. at a data center 10km or 1000km away this should prevent some panic when rebooting the server remotely.

Ext2/3 will do a filesystem check after a certain number of reboots or time. Most of the time any errors are fixed automatically, but certain errors by default require the root shell and the administrator to fix them. I've seen a few of these happen, but I have always replied yes to the prompts as I don't know enough about file systems to fix it I said no.

So, to prevent the need to rush to a data center and plug in a keyboard and mouse just to press the "Y" key there is and option to automatically assume yes.

On Ubuntu in the file /etc/default/rcS you need to change the following:

Ignoring directories from updatedb

To tell locate to ignore a directory you need to add it to the PRUNEPATHS line in the /etc/updatedb.conf file like so:

In the above code I have added the /srv entry to make it ignore all my backups which are held under the /srv directory.

Note: These tips were tested on Ubuntu linux, other distributions will have similar functionality but the file locations may vary.

Cheers, Mark

I found this article on Installing Dell OpenManage Server Administrator on Ubuntu x64 on Keith's Code and am posting here with modifications for 32 bit for my own reference

Dell's OpenManage Server Administrator is a powerful tool for keeping track of your server's health and making sure everything is running as it should. Unfortunately, Dell only released packages for RedHat and SuSE, so installing the software on Ubuntu can be tricky, at best.

There is some information on the Internet about getting the package to install on exotic distributions, but I never found anything definitive. This tutorial will guide you through installing the software and getting the web access up and running. 

My setup is: Dell 2950 with 1 Intel 2.2GHz Processor, 2GB RAM, PERC 5/i SCSI Raid Controller with 2 73GB 15K RPM SAS disks running Ubuntu 8.04.2 LTS 32-bit.

1. Install the kernel modules

The first step is installing the needed kernel modules to support IPMI:

Next,you will want to add those modules to the end of the /etc/modules file. This will allow them to be loaded at boot time.

2. Setup Apt to get the OMSA package

As stated before, Dell only released the OpenManage Server Administrator for RedHat and SuSE. However, someone was kind enough to create an installable Debian based package for us. In order to get access to that package, you will need to add the following line to /etc/apt/sources.list:

Then, you will need to install the Pulic Key for that repository. To do that, create a temporary directory in your home folder and run the following commands from that folder:

3. Install the needed packages

Now it's time to install all the packages that will be needed:

4. Start SNMP

The following commands will enable SNMP and get it going:

If you get and error like:

You may need to reconfigure the dynamic linker so that it knows the new libraries are.

5. Start the web services

Start the Dell OMSA web interface:

6. Add a user to access the OMSA site

If your system is setup properly, you shouldn't have a password for the root user. To access the OpenManage client, you need to have root access. There is a way to do this without giving the root user a password. I created a user called dellroot that will be able to access the OMSA client:
sudo useradd dellroot
sudo usermod -g root dellroot
sudo passwd dellroot

Enter a secure password for the user when prompted.

7. Check it out

Now you should be able to access the OpenManage web interface by going to https://<servername>:1311 (e.g., https://www.example.com:1311)

Make sure you are using https and you add the port number of 1311 to the end of the URL. You can login using the dellroot user you just created. 

If you are using firefox, you will have to add a security exception for the self-signed certificate the server is using.

Thanks Keith for the helpful reference.

Cheers, Mark

This simple find command shows the offending files:

A quickie but a goodie.

Cheers, Mark

There is a bit of setup to be done but when you get the whole picture is it's actually very easy. To do it you need at least 2 asterisk boxes - the one you are load testing (lets call it 'stressbox' for this example) and a another one to drive load to it which we'll call 'driver'.

We need to do the following steps:

• Create an sip account on stressbox for driver to connect to
• Register the account from the driver machine
• Create a dial plan which simulates a call
• Create a call file for the call
• Start you system monitoring
• Create lots of call files

So here is the step by step:

Create an sip account on stressbox for driver to connect to

In the sip.conf file on stressbox (the one we are testing) add the following:

Register the account from the driver machine

Your driver machine should now be able to make calls to the stressbox machine and you should see a line that looks like this if you were watching the console:

Create a dial plan which simulates a call

Create a call file for the call

A bit of explanation of these lines: Channel: SIP/6@xxx.xxx.xxx.xxx - the 6 is the extension to dial on the stressbox server - change as necessary. The ip adress should be the stressbox one. Context: dialplan-loadtest1 - this is the dial plan that will be run for this call.

Start you system monitoring

For this I use a vmstat on linux via a little script I wrote which makes it easy to collect stats from multiple machines

It would be used as follows:

Create lots of call files

You are not ready to test you system. To do this you need to create copies the test.call file and move them into /var/spool/asterisk/outgoing on the driver machine.

I use a little script to do this:

This is used as follows:

This will make 5 calls with a 2 second delay between the start of each call.

Note: It needs to be run as sudo so it can change the ownership correctly.

Credits: Thanks to Stuart Elvish for his help with getting this working so nicely.

Happy load testing. Cheers, Mark

multivmstat is a php command line script which makes it easy to collect the statistics. Download the script

To run it you specify a list of servers to check:

This will run vmstat on each of the servers with a 5 second interval between samples.

It will create the following files:

Note: You mush have ssh access to each machine - ideally with a certificate so that no interactive authentication is required.

Download the script