Tech Blog

Normalising audio with sox

Posted At : November 25, 2009 11:48 AM

Update - as of sox 14.3.0 it is much simpler than described below - simply do: sox --norm before.wav after.wav

Seem to be on a roll with Sox tonight. Also figured out how to normalise audio files nicely.

The first thing to do with sox is to get it to calculate the max volume adjustment possible:

Audio waveform before and after Normalisation

sox before.wav -n stat -v

This will return a number like: 4.234

You can then call sox again using this number:

sox -v 4.234 before.wav after.wav

And if you're feeling particularly good you can put the whole command on a single line as follows:

sox -v `sox before.wav -n stat -v 2>&1` before.wav after.wav

Easy when you know how.

Removing silence from audio using sox

Posted At : November 25, 2009 11:11 AM

I've had a look at this a few times before and each time I've gotten a little bit further and then gotten frustrated and failed.

However, this time I finally managed to get past all the hurdles and jump through all the hoops to make this work.

Audio before and after

Scenario is a follows. Audio file is comes in in alaw or ulaw format and we want to trim any silence before and after it.

First step is to convert it from alaw to pcm encoded wav file as SOX has issues filtering silence in alaw files

sox -t alaw in.alaw -c1 -2 -r8000 -e signed-integer temp.wav

Then we do the trimming in one fell swoop.

sox temp.wav out.wav silence 1 0.1 0.1% reverse silence 1 0.1 0.1% reverse

A huge thanks to this article for giving such a thorough overview of the how silence detection works in sox.

Too easy, when you know how :-)

Apache Deflate Howto

Posted At : November 15, 2009 8:15 AM

For my own reference, settings to turn on apache mod_deflate.

This sets turns it on for everything except gif,jpeg,png or mp3, as these are already well compressed.

SetOutputFilter DEFLATE
   SetEnvIfNoCase Request_URI \
      \.(?:gif|jpe?g|png|mp3)$ no-gzip dont-vary

Cheers, Mark

Windows File Sharing (SMB/CIFS/Samba) over SSH

Posted At : November 14, 2009 2:35 AM

While working with a client recently setting up a Netgear VPN so he could securely access his internal file server. The VPN setup was straightforward but every time the VPN client connected to the VPN server the VPN server/firewall would crash - leaving no connectivity.

In order to come up with a reliable solution to this we decided to use the SSH server we had available and tunnel the windows sharing across the local port forwards, much simpler and more reliable.

Thanks to this article it was a breeze to set up.

Steps are as follows:

  • Create loopback adapter on windows
  • Configure loopback adapter on windows
  • Reboot
  • Configure SSH connection
  • Test it all out

Create loopback adapter on windows

We'll give your computer an additional (fake) IP address, and we'll port forward to that address instead of the computer's real IP. Windows XP will continue to do file sharing on the real IP address. We'll assign it an IP of 10.0.0.1 (that's what we configured putty to use above.)

  1. System->Control Panel->Add Hardware
  2. Yes, Hardware is already connected
  3. Add a new hardware device (at bottom of list)
  4. Install the hardware that I manually select
  5. Network adapters
  6. Microsoft , Microsoft Loopback Adapter
  7. (Go through the installation procedure.)

Configure loopback adapter on windows

  1. Open your new fake ethernet adapter (Network Connections) , enter a made-up IP address (I suggest 10.0.0.1, which is a privately routable address that most folk don't use.)
  2. Enable Client for Microsoft Networks.
  3. Disable File and Printer Sharing for Microsoft Networks
  4. Enable Interent Protocol (TCP/IP)
  5. Click on properties for TCP/IP.
  6. Enter your chosen IP address (10.0.0.1), subnet mask (255.255.255.0). You can leave gateway blank.
  7. Under advanced->WINS, Enable LMHosts Lookup and Disable NetBIOS over TCP/IP

Reboot

In order to make it all work now it he appropriate time to reboot so windows initialises everything correctly.

Configure SSH connection

  • Download Putty
  • Enter IP address
  • Enter Auth Key (if using SSH keys)
  • Enter Port forwards for: (these connect the ports on you local machine to
    • 10.0.0.1:137 to 127.0.0.1:137
    • 10.0.0.1:138 to 127.0.0.1:138
    • 10.0.0.1:139 to 127.0.0.1:139
    • 10.0.0.1:445 to 127.0.0.1:445
  • Save the config.

Test it all out

Now to connect you need to do the following steps:

  • Open putty, load the settings and connect.
  • Open Exporer and type in: \\10.0.0.1\

You should now be connected to your remote windows system over a secure encrypted tunnel.

Cheers, Mark

cp "No space left on device" problem - Solved

Posted At : November 14, 2009 2:12 AM

One of my backup scripts started reporting errors recently about running out of space:

cp: cannot create regular file `filename in here': No space left on device

Running the command df was showing lots of free space.

However, a quick google on cp "no space left on device" turned up the suggestion to try:

df -i

This showed up the problem straight away - I had run out of inodes.

Once the problem is identified it is generally easy to resolve, so I figured out that there was a cron job that was running a wget task and not discarding the output, and so had saved 600,000+ files in their home directory.

However, when I tried to delete them I encountered another problem:

# rm filepattern*
bash: /bin/rm: Argument list too long

There were so many files that I couldn't use standard delete commands.

Another quick google turned up this gem:

I ran the following command to check that I was going to delete the correct files:

find . | grep filepattern
And then added the command to actually do the delete:
find . | grep filepattern | xargs rm

One final thing was to fix up the cron job that was causing the problem. Adding the parameter --delete-after to wget kept the directory nice and clean.

All fixed. One of the things this has reinforced for me is how important it is to have /home on a seperate partition. If this had not been the case then the problem would have taken longer to happen (due to larger filesystem) but would have been more destructive - as all systems and processes would not have been able to create new files.