Learnosity Logo
Learnosity Banner Image

Remote collection of vmstat log files

We've been doing a lot of load testing recently and are planning on doing a lot more, so we've developed a simple little script to make the collection of the vmstat (performance statistics) a bit quicker.

multivmstat is a php command line script which makes it easy to collect the statistics. Download the script

To run it you specify a list of servers to check:

./multivmstat server1,server2,server3 5

This will run vmstat on each of the servers with a 5 second interval between samples.

It will create the following files:

server1-vmstat.log
server2-vmstat.log
server3-vmstat.log

Note: You mush have ssh access to each machine - ideally with a certificate so that no interactive authentication is required.

Download the script

Ubuntu Server upgrade howto

One of the nice things about Ubuntu is how easy it makes it to upgrade between versions - typically start up update-manager and it will prompt you if there is a new version available and handle all the details of the upgrade for you. I've done numerous upgrades this way and never had any issues.

However, I have a number of ubuntu servers which I manage and previously I have upgraded them by changing the /etc/apt/sources.list and then running "sudo apt-get dist-upgrade"

Today I discovered that there is an automated script to handle it - which I presume is what powers the gui version behind the scenes.

You need to make sure you have update-manager-core installed:

sudo apt-get install update-manager-core

Then you simply run do-release-upgrade

sudo do-release-upgrade

I've just upgraded two different servers from Feisty as it has recently stopped being supported. I upgraded them to Gutsy and then to Hardy which is a Long Term Supported version. The whole process took about an hour of which it took about 5 minutes of my attention to answer a few simple questions.

Who said Linux is hard :-) Cheers, Mark

Choppy sound on Twinkle Softphone on Ubuntu Linux - Solved

I just reinstalled my system with the latest Ubuntu 8.10 x64 and one of the apps I use regularly for testing and developing Asterisk applications is Twinkle, which is a great VOIP softphone.

It works great, but when I tried it in my new install the sound was all choppy, i.e. I would get every half of each second of the audio.

A bit of fiddling about with the settings and I got it solved. Posting it here so when I forget about it and run into the same problem again I'll know how to fix it.

The solutions is as simple as:

  • From the menu select Edit -> System Settings.
  • Select Audio on the left
  • Change the audio devices to use the most approriate device that is NOT "ALSA: default: Default Device" - in my case it was "ALSA: plughw:0,0: HDA Intel (AD193x Analog)"

I do like the very friendly naming scheme for devices :-) Hope it helps, Mark

Change timezone on Ubuntu server (CLI only)

It should be simple to do this and it is. The hardest part was finding the correct way to do it, so for future reference:

Using the command line, you can use dpkg-reconfigure tzdata.

sudo dpkg-reconfigure tzdata

Follow the instructions to reconfigure the timezone. This was taken from the ubuntu wiki article

Cheers, Mark

Custom Language files for Asterisk on Ubuntu

There are a number of steps needed to get custom language files working in a nice fashion for asterisk on ubuntu.

For this example I'm going to replace the standard digits of 0 to 10 with files that I've had recorded.

Create files in the correct format:

Starting with files in the format: 0.wav, 1.wav .... 10.wav which are 44.1kHz 16 bit PCM mono wav files.

We can convert them to alaw, ulaw and gsm formats which reduces asterisk transcoding with the following script: (requires sox to be installed)

#!/bin/sh
#
# Script to batch convert wav files to gsm,alaw and ulaw formats
# to help reduce overhead for asterisk server
# Run it in a directory of wav files.

for a in *.wav; do
   echo "Processing $a";
   baseName=`echo $a|sed -e s/.wav//`;
   #Convert to 8bit wav first at 8000Hz - this can take a while
   sox "$a" -r 8000 -c1 "$baseName_temp.wav" resample -ql;

   echo "...additional formats";
   #Convert to gsm
   sox "$baseName_temp.wav" "$baseName.gsm";

   #convert to alaw - note using .alaw extension as asterisk likes that
   sox "$baseName_temp.wav" -t al "$baseName.alaw";
   #convert to ulaw - note using .ulaw extension as asterisk likes that
   sox "$baseName_temp.wav" -t ul "$baseName.ulaw";

   #clean up the .wav files
   rm "$baseName_temp.wav"
done

Move the files to the correct place on the server:

I'm going to use the name 'learnosity' for the language as these are the files that we've recorded. So we need to create the following directory on the asterisk machine:

/usr/share/asterisk/sounds/digits/learnosity

If you wanted to do more audio files, you can create learnosity directories for the other audio files.

Once this is done, copy the alaw, ulaw and gsm files in but leave the original wav files out.

Tell asterisk to use these files:

The easiest way to use these language files is to set the channel language for the sections you want to use it:

Here's an example from extensions.conf:

exten => 4,1,Answer()
exten => 4,n,SayDigits(123456789)
exten => 4,n,Set(CHANNEL(language)=learnosity)
exten => 4,n,SayDigits(123456789)
exten => 4,n,Hangup()

If you dial extension 4 you get the digits 1 to 9 in the default language and if you've done everything correctly you will get your newly recorded digits the second time.

Cheers, Mark

Howto refresh /dev/disk/by-uuid on Ubuntu

I was recently setting up a Ubuntu server and was partitioning it after it had been installed.

There was lots of free space on the drives as the root partition was only using a small portion of the disk. After running fdisk to partition it and mkfs.ext3 on the partitions to format them I couldn't see them in /dev/disk/by-uuid.

A quick google presented the solution:

sudo udevtrigger

According to the man page this makes udev request the kernel device uevents, which in essence makes it read the disk info again and show it all up so you can mount it happily.

Cheers, Mark

Extracting from sound from Flash (aka NellyMoser)

A current project that we are working on requires us to be able to record sound via a flash plugin. Initially I thought it would be nice and easy as I've seen demos of both video and audio broadcasting - however, the one big problem is that the current flash client only allows you to record video to a netstream, you can't get any access to it in the flash player.

So you need something like Flash Media Server or Red5 to record it.

However, once you record it to the server it is an flv and the audio codec is stored as a NellyMoser encoded audio portion. This codec is not supported by many applications and after a quick google found the nelly2pcm project on google code.

This will convert a flv sound file to a raw pcm file - which you can then do useful stuff with. So here's how to do it on a Ubuntu machine.

$ tar -xjvf nelly2pcm.tar.bz2
$ cd nelly2pcm
$ make
cc -Wall -c -o nelly2pcm.o nelly2pcm.c
cc -Wall -c -o nelly.o nelly.c
cc -Wall -c -o nelly_tables.o nelly_tables.c
cc -Wall nelly2pcm.o nelly.o nelly_tables.o -lm -o nelly2pcm
You should now have a nelly2pcm executable file in this directory.

You can run it as follows:

$ ./nelly2pcm test.flv > test.raw
mono Nellymoser stream with 16-bit samples at 44kHz

This will create a raw sound file with no headers and output the line above which you'll need for the next part.

To play this file back you can use sox (apt-get install sox)

$ play -r 44100 -c 1 -2 -s test.raw

If you've got all the options correct then this will play the file back. Once you got it correct you can then use sox to create a wav file which is essentially the same except that it has a header which contains all the settings (eg bitrate etc)

$ sox -r 44100 -c 1 -2 -s test.raw test.wav

From a wav you can convert to whatever you like. I'm looking forward to the Flash Player 10 release as this messing will no longer be necessary as it will support encoding with the free Speex codec.

Cheers, Mark

Update:

In the 12 hours since I wrote this post it appears that a DMCA takedown notice has been served on the nelly2pcm site and google have taken it down. I've no idea if it's related to this post or not and no details are available yet on the chillingeffects site but hopefully it will be updated shortly. As I mentioned earlier, I can't wait for flash 10 with the speex codec so we don't have to use Nellymoser, well done Adobe for including it.

Convert Mac Line endings to Linux

I was cleaning up some source code yesterday and wanted to print it all out, however the Mac line endings (we use Mac's and Ubuntu mostly) were messing up the printout on my Ubuntu machine.

A quick google later and I found the solution, a program called 'tr', posting here for next time:

tr "\r" "\n" < filewithmaclineendings.txt > filewithunixlineendings.txt

Obviously you could convert from Unix to Mac by reversing it.

tr "\n" "\r" < filewithunixlineendings.txt > filewithmaclineendings.txt

Cheers, Mark

Monitoring multiple log files in a cluster

While chatting with my friendly local sysadmin the other day about monitoring multiple log files across our cluster of server he pointed me towards multitail.

With a little bit messing about I managed to get it doing exactly what I needed.

I created the following little script which connects to 4 remote servers and monitors the log files (file names and server names have been changed to protect the innocent).

#!/bin/bash
multitail -l "ssh server1 tail -f /var/log/myinteresting.log" \
-l "ssh server2 tail -f /var/log/myinteresting.log" \
-l "ssh server3 tail -f /var/log/myinteresting.log" \
-l "ssh server4 tail -f /var/log/myinteresting.log"

Now with a single command I can monitor the log files across the entire cluster.

Happy days. Cheers, Mark

DVD to iPod conversion - Linux, Mac & Windows

I've been looking for a while for a way to convert DVD's and other movies that I have into iPod format and have been failing miserably. That is until today, when I came across the unusually named HandBrake.

It does exactly what is says on the tin and makes it very easy to copy dvd's. Initially I used it from my Mac and it worked flawlessly.

I also tried the Linux version on Ubuntu Hardy (8.04) which has no GUI but it is pretty simple when you get the hang of it.

Here's a typical command line:

HandBrakeCLI --preset="iPod Low-Rez" -i /media/cdrom -o myvideo.mp4 -t 2

This reads from /dev/cdrom and uses the handy "iPod Low-Rez" preset which fixes it all up nicely for the iPod.

And here's one that takes the second title:

HandBrakeCLI --preset="iPod Low-Rez" -i /media/cdrom -o myvideo.mp4 -t 2

The wiki has more information on command line usage.

Happy iPodding.

Cheers, Mark

More Entries