These are:

• Logging to DB instead of log files
• Runtime configuration of logging.

Logging to DB instead of log files

Coming from a web development background rather than a sysadmin background I'm far more comfortable manipulating and analysing data using SQL. So to be able to log all the queries or just the slow queries for an application to the db during application development or load testing is a huge benefit.

To enable logging to DB you can add the following to your my.cnf

The logs will be written to the 'slow_log' and 'general_log' tables in the mysql database.

Note - logging to tables has more overhead than logging to file, so would suggest using it primarily for development purposes.

Full details of the options are on the mysql manual on log tables

Runtime configuration of logging.

This allows you to turn on and off logging without restarting MySQL - which just saves a little bit of time and makes it much nicer for debugging problems.

To turn on the logging of all queries run:

And to turn them both off use:

If you also want to see queries not using indexes in the slow query log you can set the following variable:

Hope it helps, Mark

To do this I used a SSH sock proxy like I have previously blogged about.

So I fired this up so that I could review the site I wanted to look at. It worked a charm through firefox but there is no where to set up the proxy in jmeter.

To make it work you need to let the JVM know what proxy to use like so:

No jmeter will use the socks proxy on port 8080 on my local machine. Nice.

About Learnosity

Our service:

• Makes it practical for students to practice Oral and Aural skills
• Is efficient and effective for teachers, as they can listen to each student individually at a time to suit them
• Can be used for homework assignments or “High Stakes Assessments”

We are continuing to grow our core development team and we need another great developer to help us keep up with demand.

We need someone who can:

• Use Javascript or Actionscript to create great user interfaces
• Develop highly scalable web applications
• Cut code with the best in the world

You will also need to be:

• energetic with a butt kicking attitude
• ready to create cutting edge web 2.0 apps
• keen to continue learning new technologies
• able to have a conversation with non technical people

You'll need:

• 3 or more years of programming experience
• Expert in at least on Client side language (Actionscript or AJAX)
• Expert in at least one Server side language (eg PHP, Java, ColdFusion, etc)
• Understanding of Object Oriented design
• Understanding of XHTML and CSS

It would be good if you have:

• A degree in Computer Science, Engineering or similar.
• been working with open source tools
• been playing around with iPhone/Android applications
• experience with some of Linux/VOIP/SIP/Asterisk/Jabber/XMPP

This is a full time role and you will be working in a casual workplace with flexible hours in the Sydney CBD.

If this sounds like the job for you, email a covering letter explaining why you'll be great and your resume to mark@learnosity.com - no agencies please.

Seem to be on a roll with Sox tonight. Also figured out how to normalise audio files nicely.

The first thing to do with sox is to get it to calculate the max volume adjustment possible:

This will return a number like: 4.234

You can then call sox again using this number:

And if you're feeling particularly good you can put the whole command on a single line as follows:

Easy when you know how.

However, this time I finally managed to get past all the hurdles and jump through all the hoops to make this work.

Scenario is a follows. Audio file is comes in in alaw or ulaw format and we want to trim any silence before and after it.

First step is to convert it from alaw to pcm encoded wav file as SOX has issues filtering silence in alaw files

Then we do the trimming in one fell swoop.

A huge thanks to this article for giving such a thorough overview of the how silence detection works in sox.

Too easy, when you know how :-)

This sets turns it on for everything except gif,jpeg,png or mp3, as these are already well compressed.

Cheers, Mark

In order to come up with a reliable solution to this we decided to use the SSH server we had available and tunnel the windows sharing across the local port forwards, much simpler and more reliable.

Thanks to this article it was a breeze to set up.

Steps are as follows:

• Create loopback adapter on windows
• Configure loopback adapter on windows
• Reboot
• Configure SSH connection
• Test it all out

Create loopback adapter on windows

We'll give your computer an additional (fake) IP address, and we'll port forward to that address instead of the computer's real IP. Windows XP will continue to do file sharing on the real IP address. We'll assign it an IP of 10.0.0.1 (that's what we configured putty to use above.)

1. System->Control Panel->Add Hardware
2. Yes, Hardware is already connected
3. Add a new hardware device (at bottom of list)
4. Install the hardware that I manually select
5. Network adapters
6. Microsoft , Microsoft Loopback Adapter
7. (Go through the installation procedure.)

Configure loopback adapter on windows

1. Open your new fake ethernet adapter (Network Connections) , enter a made-up IP address (I suggest 10.0.0.1, which is a privately routable address that most folk don't use.)
2. Enable Client for Microsoft Networks.
3. Disable File and Printer Sharing for Microsoft Networks
4. Enable Interent Protocol (TCP/IP)
5. Click on properties for TCP/IP.
6. Enter your chosen IP address (10.0.0.1), subnet mask (255.255.255.0). You can leave gateway blank.
7. Under advanced->WINS, Enable LMHosts Lookup and Disable NetBIOS over TCP/IP

Reboot

Configure SSH connection

• Download Putty
• Enter IP address
• Enter Auth Key (if using SSH keys)
• Enter Port forwards for: (these connect the ports on you local machine to
  • 10.0.0.1:137 to 127.0.0.1:137
  • 10.0.0.1:138 to 127.0.0.1:138
  • 10.0.0.1:139 to 127.0.0.1:139
  • 10.0.0.1:445 to 127.0.0.1:445
• Save the config.

Test it all out

• Open putty, load the settings and connect.
• Open Exporer and type in: \\10.0.0.1\

You should now be connected to your remote windows system over a secure encrypted tunnel.

Cheers, Mark

Running the command df was showing lots of free space.

However, a quick google on cp "no space left on device" turned up the suggestion to try:

This showed up the problem straight away - I had run out of inodes.

Once the problem is identified it is generally easy to resolve, so I figured out that there was a cron job that was running a wget task and not discarding the output, and so had saved 600,000+ files in their home directory.

However, when I tried to delete them I encountered another problem:

There were so many files that I couldn't use standard delete commands.

Another quick google turned up this gem:

I ran the following command to check that I was going to delete the correct files:

One final thing was to fix up the cron job that was causing the problem. Adding the parameter --delete-after to wget kept the directory nice and clean.

All fixed. One of the things this has reinforced for me is how important it is to have /home on a seperate partition. If this had not been the case then the problem would have taken longer to happen (due to larger filesystem) but would have been more destructive - as all systems and processes would not have been able to create new files.

All the DB storage and application was using UTF8 throughout, but the PDF exports were showing nothing when they should have been showing Chinese characters.

To fix this up you need to do the following:

1. Need fonts installed on system:

2. Need fonts installed in CFAdmin

• Go to CFAdmin and Font Management
• Select directory /usr/share/fonts/truetype/arphic
• Select "Add" and you should see 2 new fonts added.

3: CFdocument Hotfix for CF7.0.2

Configure cffont.properties file

The cffont.properties file is located in /opt/jrun4/servers/{instance name}/cfusion.ear/cfusion.war/WEB-INF/cfusion/lib on the jrun multiserver version.

5. Restart CF

Once you restart CF you documents should be coming out with their Chinese fonts intact.

I'm sure there are additional fonts to add to get it working natively with all the other asian languages, so I'll add to this as and when I need/discover them. Please feel free to post a comment if you find any more font's that should be added to the list.

• Automatically fixing file system errors
• Ignoring directories from updatedb

Automatically fixing file system errors

If you have a remote server, i.e. at a data center 10km or 1000km away this should prevent some panic when rebooting the server remotely.

Ext2/3 will do a filesystem check after a certain number of reboots or time. Most of the time any errors are fixed automatically, but certain errors by default require the root shell and the administrator to fix them. I've seen a few of these happen, but I have always replied yes to the prompts as I don't know enough about file systems to fix it I said no.

So, to prevent the need to rush to a data center and plug in a keyboard and mouse just to press the "Y" key there is and option to automatically assume yes.

On Ubuntu in the file /etc/default/rcS you need to change the following:

Ignoring directories from updatedb

To tell locate to ignore a directory you need to add it to the PRUNEPATHS line in the /etc/updatedb.conf file like so:

In the above code I have added the /srv entry to make it ignore all my backups which are held under the /srv directory.

Note: These tips were tested on Ubuntu linux, other distributions will have similar functionality but the file locations may vary.

Cheers, Mark