Tech Blog

Normalising audio with sox

Posted At : November 25, 2009 11:48 AM

Update - as of sox 14.3.0 it is much simpler than described below - simply do: sox --norm before.wav after.wav

Seem to be on a roll with Sox tonight. Also figured out how to normalise audio files nicely.

The first thing to do with sox is to get it to calculate the max volume adjustment possible:

Audio waveform before and after Normalisation

sox before.wav -n stat -v

This will return a number like: 4.234

You can then call sox again using this number:

sox -v 4.234 before.wav after.wav

And if you're feeling particularly good you can put the whole command on a single line as follows:

sox -v `sox before.wav -n stat -v 2>&1` before.wav after.wav

Easy when you know how.

Removing silence from audio using sox

Posted At : November 25, 2009 11:11 AM

I've had a look at this a few times before and each time I've gotten a little bit further and then gotten frustrated and failed.

However, this time I finally managed to get past all the hurdles and jump through all the hoops to make this work.

Audio before and after

Scenario is a follows. Audio file is comes in in alaw or ulaw format and we want to trim any silence before and after it.

First step is to convert it from alaw to pcm encoded wav file as SOX has issues filtering silence in alaw files

sox -t alaw in.alaw -c1 -2 -r8000 -e signed-integer temp.wav

Then we do the trimming in one fell swoop.

sox temp.wav out.wav silence 1 0.1 0.1% reverse silence 1 0.1 0.1% reverse

A huge thanks to this article for giving such a thorough overview of the how silence detection works in sox.

Too easy, when you know how :-)

Load Testing Asterisk Applications - HOWTO

Posted At : January 6, 2009 3:00 AM 2 Comments

I've been working on Load Testing some of our asterisk applications and thought I'd document the process here so I can find it when I need to do it again.

There is a bit of setup to be done but when you get the whole picture is it's actually very easy. To do it you need at least 2 asterisk boxes - the one you are load testing (lets call it 'stressbox' for this example) and a another one to drive load to it which we'll call 'driver'.

We need to do the following steps:

  • Create an sip account on stressbox for driver to connect to
  • Register the account from the driver machine
  • Create a dial plan which simulates a call
  • Create a call file for the call
  • Start you system monitoring
  • Create lots of call files

So here is the step by step:

Create an sip account on stressbox for driver to connect to

In the sip.conf file on stressbox (the one we are testing) add the following:

[loadtest]
type=friend
username=loadtest
secret=1111
context=test
host=dynamic
Do a 'sip reload' in the asterisk console to register this change.

Register the account from the driver machine

Then to register the account from the driver machine add the following to sip.conf on driver:
register => loadtest:1111@xxx.xxx.xxx.xxx
Where xxx.xxx.xxx.xxx is the ip address of the stressbox server.

Your driver machine should now be able to make calls to the stressbox machine and you should see a line that looks like this if you were watching the console:

-- Registered SIP 'loadtest' at yyy.yyy.yyy.yyy port 9350 expires 120

Create a dial plan which simulates a call

Then on the driver machine we need to create a dial plan which simulates the call we want to do and place it in extensions.conf. Here's an example dial plan:

[dialplan-loadtest1]
exten => s,1,Answer
;wait for welcome to play
exten => s,n,Wait(10)
;enter username
exten => s,n,SendDTMF(1234)
exten => s,n,Wait(4)
;enter password
exten => s,n,SendDTMF(0000)
;listen to menu
exten => s,n,Wait(10)
;select menu option 1
exten => s,n,SendDTMF(1)
exten => s,n,Wait(3)
;select menu option 2
exten => s,n,SendDTMF(2)
exten => s,n,Wait(4)

;Play some audio
exten => s,n,Playback(sampleaudio1)
;Hangup
exten => s,n,Hangup
This example does a login to a phone system and then selects some options and plays some audio and hangs up. The dial plan can be as simple or as complex as you like.

Create a call file for the call

Last config step is to create a call file which will tell asterisk on driver to call asterisk on stressbox. Create a file test.call with the following:
Channel: SIP/6@xxx.xxx.xxx.xxx
MaxRetries: 1
RetryTime: 5
WaitTime: 10
Context: dialplan-loadtest1
Extension: s
Priority: 1

A bit of explanation of these lines: Channel: SIP/6@xxx.xxx.xxx.xxx - the 6 is the extension to dial on the stressbox server - change as necessary. The ip adress should be the stressbox one. Context: dialplan-loadtest1 - this is the dial plan that will be run for this call.

Start you system monitoring

If you are going to do some load testing then you probably want to monitor the load on the machine - it is recommended to record the load of all machines involved - i.e. stressbox and driver to ensure that driver is not the bottleneck in the load test.

For this I use a vmstat on linux via a little script I wrote which makes it easy to collect stats from multiple machines

It would be used as follows:

./multivmstat stressbox,driver 5
This will connect via ssh to each box and run vmstat on them every five seconds. This can be graphed later - I'll post the scripts for this shortly.

Create lots of call files

You are not ready to test you system. To do this you need to create copies the test.call file and move them into /var/spool/asterisk/outgoing on the driver machine.

I use a little script to do this:

#!/bin/sh
#usage: makecalls callfile count delaybetween

callfile=$1
count=$2
delaybetween=$3

callnumber=0
asteriskcalldir=/var/spool/asterisk/outgoing/
asteriskuser=asterisk

echo "Using $callfile for $count iterations with a $delaybetween second delay"

while [ $callnumber -lt $count ];
do
   callnumber=`expr $callnumber + 1`
   echo "Call Number: $callnumber"
   cp $callfile $callfile.tmp
   chown $asteriskuser:$asteriskuser $callfile.tmp
   mv $callfile.tmp $asteriskcalldir
   sleep $delaybetween
done

This is used as follows:

sudo ./makecalls test.call 5 2

This will make 5 calls with a 2 second delay between the start of each call.

Note: It needs to be run as sudo so it can change the ownership correctly.

Credits: Thanks to Stuart Elvish for his help with getting this working so nicely.

Happy load testing. Cheers, Mark

Choppy sound on Twinkle Softphone on Ubuntu Linux - Solved

Posted At : November 6, 2008 9:53 PM 1 Comments

I just reinstalled my system with the latest Ubuntu 8.10 x64 and one of the apps I use regularly for testing and developing Asterisk applications is Twinkle, which is a great VOIP softphone.

It works great, but when I tried it in my new install the sound was all choppy, i.e. I would get every half of each second of the audio.

A bit of fiddling about with the settings and I got it solved. Posting it here so when I forget about it and run into the same problem again I'll know how to fix it.

The solutions is as simple as:

  • From the menu select Edit -> System Settings.
  • Select Audio on the left
  • Change the audio devices to use the most approriate device that is NOT "ALSA: default: Default Device" - in my case it was "ALSA: plughw:0,0: HDA Intel (AD193x Analog)"

I do like the very friendly naming scheme for devices :-) Hope it helps, Mark

Custom Language files for Asterisk on Ubuntu

Posted At : September 4, 2008 4:27 AM

There are a number of steps needed to get custom language files working in a nice fashion for asterisk on ubuntu.

For this example I'm going to replace the standard digits of 0 to 10 with files that I've had recorded.

Create files in the correct format:

Starting with files in the format: 0.wav, 1.wav .... 10.wav which are 44.1kHz 16 bit PCM mono wav files.

We can convert them to alaw, ulaw and gsm formats which reduces asterisk transcoding with the following script: (requires sox to be installed)

#!/bin/sh
#
# Script to batch convert wav files to gsm,alaw and ulaw formats
# to help reduce overhead for asterisk server
# Run it in a directory of wav files.

for a in *.wav; do
   echo "Processing $a";
   baseName=`echo $a|sed -e s/.wav//`;
   #Convert to 8bit wav first at 8000Hz - this can take a while
   sox "$a" -r 8000 -c1 "$baseName_temp.wav" resample -ql;

   echo "...additional formats";
   #Convert to gsm
   sox "$baseName_temp.wav" "$baseName.gsm";

   #convert to alaw - note using .alaw extension as asterisk likes that
   sox "$baseName_temp.wav" -t al "$baseName.alaw";
   #convert to ulaw - note using .ulaw extension as asterisk likes that
   sox "$baseName_temp.wav" -t ul "$baseName.ulaw";

   #clean up the .wav files
   rm "$baseName_temp.wav"
done

Move the files to the correct place on the server:

I'm going to use the name 'learnosity' for the language as these are the files that we've recorded. So we need to create the following directory on the asterisk machine:

/usr/share/asterisk/sounds/digits/learnosity

If you wanted to do more audio files, you can create learnosity directories for the other audio files.

Once this is done, copy the alaw, ulaw and gsm files in but leave the original wav files out.

Tell asterisk to use these files:

The easiest way to use these language files is to set the channel language for the sections you want to use it:

Here's an example from extensions.conf:

exten => 4,1,Answer()
exten => 4,n,SayDigits(123456789)
exten => 4,n,Set(CHANNEL(language)=learnosity)
exten => 4,n,SayDigits(123456789)
exten => 4,n,Hangup()

If you dial extension 4 you get the digits 1 to 9 in the default language and if you've done everything correctly you will get your newly recorded digits the second time.

Cheers, Mark

Job Vacancy Sydney AU - Graduate or Junior OO Programmer

Posted At : May 8, 2008 2:17 PM

We're looking for a Computer Science Graduate or similar with strong Object Oriented programming skills and the ability to grasp cutting edge technologies quickly.

If you are looking to work with the latest technogies and are a motivated self starter with a positive "can do" attitude this is the job for you. The ability to take problems and deliver solutions is a must.

There will be significant on the job learning as we are always pushing the boundaries and using the latest and greatest technologies. We are currently working on projects using Actionscript/Flex, ColdFusion, PHP, and Javascript, and are leveraging technologies such as VOIP, SMS, Instant Messenger (Jabber XMPP) to deliver cutting edge solutions.

Suitable applicants will have a Bachelors Degree in Computer Science or Engineering, or similar tertiary qualification. Remuneration commensurate with experience.

Responsibilities would include:

  • Developing new functionality for Asterisk and VOIP applications
  • Creating Rich Internet Applications with Adobe Flex
  • Creation of Desktop applications with Adobe AIR
  • Design and development of new Web applications

Must haves:

  • A robust foundation in Object Oriented programming
  • Demonstrated experience in at least one OO language
  • Motivation to learn and push the boundaries
  • Understanding of XHTML and CSS

Preferable:

  • Experience with PHP, ColdFusion, Farcry CMS, XML, Web Standards etc
  • Competency using linux-based tools (SSH, bash, etc)
  • Familiar with using source control tools (Subversion, etc)
  • Familiarity with Actionscript or Flash

This role may be full time, part-time or on a contract basis depending on the candidates skills and experience. You will be working in a casual workplace with flexible hours in the centre of Sydney.

If this sounds like the job for you email your resume to mark@lynchconsulting.com.au if interested - no agencies please.

Setting up Asterisk with Faktortel on Ubuntu - HOWTO

Posted At : January 30, 2008 10:03 PM 1 Comments

Having just set up an asterisk box to connect to Faktortel I thought I'd post the settings I used as the ones on the Faktortel support website do not work quite right:

For a default asterisk install on Ubuntu do the following:

$ sudo apt-get install asterisk

You then need to edit the following files three files:

sip.conf

[general]
port = 5060 ; Port to bind to (SIP is 5060)
bindaddr = 0.0.0.0 ; Address to bind to (all addresses on machine)

; sets the codecs you can use. First disallow all defaults, then set our own ind ividually.
disallow = all
allow=gsm

; Setup a user with the phone number 1000 on our network.
[1000]
tos=reliability
type=friend
username=1000
host=dynamic
context=default
reinvite=no
canreinvite=no
secret=abc123
nat=yes
qualify=yes

iax.conf

; Setup the general options, including our default port for use with IAX
; the amount of bandwidth we have on our connection to the server (best left as low)
; jitterbuffer, this isn't enabled at the moment in the latest CVS but we don't want it
; to suddenly come on when it is enabled.

[general]
bandwidth=low
jitterbuffer=no
tos=lowdelay


; This line registers your server with the FaktorTel main server, what this means is that
; it tells us where you are on the internet so you can receive incoming calls from our network
; don't forget to replace "YOUR-USERNAME" and "YOUR-PASSWORD" with the ones supplied to you.

register => YOUR-USERNAME:YOUR-PASSWORD@iax.faktortel.com.au

; The following section sets up an IAX user called "faktortel", this is the user which will place
; the call from your server to our network. First we tell it not to use any other codec by setting
; our "Disallow" and "allow" settings. Then we set the type to "friend", then set our password and
; username for the faktortel network (don't forget to change these to the ones supplied to you).
; using the "host" command we tell the user which server to connect to, then using "auth" we tell
; it the type of password to look for, and finally using "qualify" we tell it to keep in touch
; with the server ever 3 seconds. (This is fast enough).

[faktortel]
disallow=all
allow=gsm
type=friend
username=YOUR-USERNAME
secret=YOUR-PASSWORD
host=iax.faktortel.com.au
auth=md5
context=incoming
qualify=yes

extensions.conf

; First we setup a context called "Default" this is our "Encapsulation" context. The only point of this
; context is to point to all our other groups in order. We only have one other outgoing context at the
; moment so we are just including one "faktortel-outgoing" however this will allow you to make a much
; more sturdy network later if you use it as a full PBX. This is also the context we use to place calls out of.

[default]
include => faktortel-outgoing


; FaktorTel-Outgoing is another context, this tells anyone who is trying to make a call out from the PBX
; which starts with a "0" to put that call out to the PSTN via FaktorTel. The format here is best
; described at "www.voip-info.org" under "configuration files" "extensions.conf".

[faktortel-outgoing]
exten => _0.,1,DIAL(IAX2/YOUR-USERNAME@faktortel/${EXTEN:1})
exten => _0.,2,Congestion

; Adding an extension for the number 1234 which will give you the Time.
exten => 1234,1,SayUnixTime
exten => 1234,2,Hangup


; This is our "incoming" context, when a call comes in from the faktortel network it is sent here
; because we set "context=incoming" in our IAX.CONF file under [faktortel]. This then tells it
; that ANY call that comes in should ring the SIP based phone at extension "1000" which we setup
; earlier.

[incoming]
; Add extension for you DID number
exten => YOUR_DID_NUMBER,1,Answer
exten => YOUR_DID_NUMBER,n,SayUnixTime
exten => YOUR_DID_NUMBER,n,Hangup

Hope this helps you get going with Asterisk. Cheers, Mark