Learnosity Logo
Learnosity Banner Image

"Show Full Columns" problem with CFMX and MySQL solved

We've been performing some load testing on a new website we've developed and our helpful sysadmin noticed lots of queries happening on the DB that looked like this:

SHOW FULL COLUMNS FROM `dbname`.`tablename`

We weren't intentionally doing these queries but they were coming from somewhere. A bit of detective work via google found a very enlightening article about this problem.

A quick read of this explained:

It turns out ColdFusion was asking Connector/J for the metadata on every field, which in turn triggered a SHOW FULL COLUMNS query for every varchar and text column returned.

It also went on to say that is had been fixed as of Connector/J 5.07.

On reading the release notes it mentions that the

Driver detects when it is running in a ColdFusion MX server (tested with version 7), and uses the configuration bundle coldFusion, which sets useDynamicCharsetInfo to false (see previous entry), and sets useLocalSessionState and autoReconnect to true.

However, from my testing it wasn't doing it as we were still seeing the problem. However, we are running the MultiServer version of ColdFusion which sits atop Jrun4.

So I added the parameter directly to the querystring:

&useDynamicCharsetInfo=false
So it now looks like:

This has stopped the problem, and given a significant performance gain. On one of the sites I was load testing it gave a 12% throughput increase and on another one which had less queries it gave a 4% increase.

Hope it helps.

Cheers, Mark

Job Vacancy Sydney AU - Graduate or Junior OO Programmer

We're looking for a Computer Science Graduate or similar with strong Object Oriented programming skills and the ability to grasp cutting edge technologies quickly.

If you are looking to work with the latest technogies and are a motivated self starter with a positive "can do" attitude this is the job for you. The ability to take problems and deliver solutions is a must.

There will be significant on the job learning as we are always pushing the boundaries and using the latest and greatest technologies. We are currently working on projects using Actionscript/Flex, ColdFusion, PHP, and Javascript, and are leveraging technologies such as VOIP, SMS, Instant Messenger (Jabber XMPP) to deliver cutting edge solutions.

Suitable applicants will have a Bachelors Degree in Computer Science or Engineering, or similar tertiary qualification. Remuneration commensurate with experience.

Responsibilities would include:

  • Developing new functionality for Asterisk and VOIP applications
  • Creating Rich Internet Applications with Adobe Flex
  • Creation of Desktop applications with Adobe AIR
  • Design and development of new Web applications

Must haves:

  • A robust foundation in Object Oriented programming
  • Demonstrated experience in at least one OO language
  • Motivation to learn and push the boundaries
  • Understanding of XHTML and CSS

Preferable:

  • Experience with PHP, ColdFusion, Farcry CMS, XML, Web Standards etc
  • Competency using linux-based tools (SSH, bash, etc)
  • Familiar with using source control tools (Subversion, etc)
  • Familiarity with Actionscript or Flash

This role may be full time, part-time or on a contract basis depending on the candidates skills and experience. You will be working in a casual workplace with flexible hours in the centre of Sydney.

If this sounds like the job for you email your resume to mark@lynchconsulting.com.au if interested - no agencies please.

Getting Flex 4.0 up and running on Ubuntu

I came across a blog entry today from Mike Morearty about the flex 4 source tree.

It also mentioned the swfdump tool which looked interesting

I downloaded it and went to the bin directory to try swddump but it didn't work straight away complaining about a missing jar file.

I thought this would mean a lot of pain to get it all compiling but I was very surprised. Here were the steps to get it working on Ubuntu 8.04 Hardy Heron.

Step 1

Get the code from the repository

This will check out the code to a directory called flex4 under your current directory. I'm assuming you have Subversion installed but if you don't you'll need to run this first.
sudo apt-get install subversion

Get the tools to compile flex

sudo apt-get install ant ant-optional sun-java6-jdk
That wasn't hard now was it?

Compile

Go into the correct directory and call "ant" which will use the build.xml to build it all.
cd flex4
ant

Some couple of minutes later you should get the following message

BUILD SUCCESSFUL
Total time: 2 minutes 41 seconds

Now you have a build of Flex4. Next step is to figure out how to configure Flex Builder to use the new compiler.

NOTE: This is not a finished version of Flex4 yet. Just the work in progress.

Cheers, Mark

MD5 Checksum for Images (Binaries) in CF

I've been working on a piece of an application which is needs to verify that the images received have not been modified or tampered by using MD5 hashes.

Note: If you want to skip my ramble and get to the solution scroll to the bottom for the example code and download.

I thought it would be nice and easy to do it with the CF Hash function and on first attempt it appeared to be:

<cffile action="read" file="#getDirectoryFromPath(getCurrentTemplatePath())#firework.jpg" variable="myTextFile">
Hash: <cfoutput>#hash(myTextFile)#</cfoutput>
Gives:
Hash: 2A3643821420D5349665D2231842FD89

However, the hashes get created on by CF but get verified by an Flex application so I was using the as3corelib's MD5 function and couldn't get the hashes to match.

So I decided to try another MD5 implementation to see which of my functions was wrong - unfortunately it turned out that both of them were wrong but for two different reasons. I'll cover the AS3 problem in a different post.

The output from md5sum was as follows:

$ md5sum firework.jpg
2346b6ab017de31688ee35949612db07 firework.jpg

This didn't match what I had generated. A bit more head scratching and I realised that the problem was that I was trying to do an md5 hash on a binary file but the CF Hash() only takes a "string".

After loading the file with "readbinary" the hash function didn't work (no surprise) and a bit of digging around led me to use some Java functionality to handle the generation of the Hash.

<cffile action="readbinary" file="#getDirectoryFromPath(getCurrentTemplatePath())#firework.jpg" variable="myBinaryFile">
<cfset var md5 = createObject("java","java.security.MessageDigest").getInstance("MD5")>
<cfset md5.update(myBinaryFile,0,len(myBinaryFile))>
<cfset checksumByteArray = md5.digest()>

This leaves us with a ByteArray digest of the file - which we need to convert to a familiar hex encoded MD5 Hash.

<cfloop from="1" to="#len(checksumByteArray)#" index="i">
   <cfset hexCouplet = formatBaseN(bitAND(checksumByteArray[i],255),16)>
   <!--- Pad with 0's --->
   <cfif len(hexCouplet) EQ 1>
      <cfset hexCouplet = "0#hexCouplet#">
   </cfif>
   <cfset checkSumHex = "#checkSumHex##hexCouplet#">
</cfloop>
Binary Hash: <cfoutput>#checkSumHex#</cfoutput>
This gives the following checksum:
Binary Hash: 2346b6ab017de31688ee35949612db07

Bingo - all working nicely now. I've packaged it all up to make my life easier I've packaged it up into a nice CFC - Crypto.cfc.

Solution and Download

Here's the example code and a running demo of it.

The Crypto CFC is available from the subversion repository.

Here is the code to use it:

<cffile action="readbinary" file="#getDirectoryFromPath(getCurrentTemplatePath())#firework.jpg" variable="myBinaryFile">
<cfset md5 = createObject("component","au.com.lynchconsulting.cfc.utility.Crypto").hashBinary(myBinaryFile)>

<h3>MD5 Sum calculated from hashBinary()</h3>
Hash Binary: <cfoutput>#md5#</cfoutput>

Hope it helps. Cheers, Mark

Complex CF Webservices problem - solved

Couple of days ago I ran into ColdFusion Webservices problem using cfinvoke and complex arguments. The Java Webservice I was talking to expected 3 arguments - 2 strings and 1 array of structures. So far, I only had to pass String arguments to webservices, which worked fine. For some reason, and correct me if I'm wrong, ColdFusion doesn't seem to translate complex variables into correct soap-equivalent format when sending the soap packet out. I got the following error - "Web service operation 'myMethod()' with parameters {args} could not be found". I was definitely calling the right method, but CF Array I was sending was not recognised by the Webservice.

I was pretty determined to get this working and decided to build the SOAP envelope manually and send it to the webservice using a CFHTTP POST. This actually worked very nicely and this is how I did it:

[More]

JMeter load testing - Regex problem solved

I've been having fun playing with JMeter to do some load testing on some of our applications recently and I came across this problem which had me stumped. Eventually my friendly sysadmins put me out of my misery. So to prevent other people from the same issues - here is the problem and the solution.

Jmeter screenshot - not working

JMeter is a great load testing application but it can be a bit tricky to when you are learning it.

For the particular testing I needed to get it to do the following:

  • Login in as a user
  • Retrieve the valid userid
  • Create a new test for the user
  • Get the testid back
  • Load the test details
  • Submit the test

I pretty quickly got everything working with some regex's to extract the userid and testid as required and my test case looked like the picture above.

Everything was working except the final submission of the test, which was failing as the regex variable was always using the default value, despite the fact that is was working for the previous step.

Jmeter screenshot - working!!

It turns out that the regex needed to be nested inside the request it was referring to or it would be called after every request, which in most cases would return the default value as the regex wouldn't match.

Once I put the regex's inside the requests the were not constantly being reset and everything worked a treat.

Hope it helps.

Mark

Cross platform Database IDE

Ever since I've moved to using Linux full time I've been looking for a solid Database IDE. I've tried and tested quite a few.

Aqua Data Studio
This is my current DB IDE of choice and has some significant features in the new version 6.5 that means I'll be staying with it for a while, particularly as it runs on Linux, Macs and Windows identically.
  • Schema Comparison - very handy for keeping development and production db's schemas in sync.
  • Charts & Pivot Tables - makes it easy to create adhoc graphs withouth having to resort to Excel/OpenOffice Calc
  • Morph to Delimited List - A simple but powerful function that saves a lot of time. Allows you to take the results of a sql query and convert to comma seperated list. I've done dozens of adhoc CF scripts to handle this before :-)
  • Subversion Support - As we use subversion for our code it's handy to have it built in.

MySQL GUI tools (Query Browser & Admin)
These have some very nice functionality but have always found them to be pretty unstable - they just crash a lot with no explanation.
Navicat
This is pretty good option, but the Linux version is the poor cousin to the Windows & Mac versions.

Love to hear any comments feedback or suggestions.

howto document your code with ASDoc

Last night I was trying to document our AS3 code, using ASDoc, for a project that we have been working on. For those of you who don't know, ASDoc is a command line utility which allows you to generate code documentation in HTML format. A good example of what ASDoc output looks like would be, Flex 2 Language Reference.

Documenting code wasn't as easy as I thought it would be, so I thought I'd share this with you and hopefully it will save someone some time. I use an Intel Mac and this is what you need to do to get your basic code documentation generated successfully:

1. In your terminal window go to your Flex SDK bin directory. Typically this is located under /Applications/Adobe Flex Builder 2/Flex SDK 2/bin

2. From "bin" directory run this command:

./asdoc -doc-sources=/Users/marko/Documents/workspace/yourFlexProject/ -source-path=/Users/marko/Documents/workspace/yourFlexProject/

If you have any external Libraries in your Flex project you will need to include them in your command:

-external-library-path=/path_to_your_swc/myLib.swc

The key thing to remember for Mac users is that you need to run the command from Flex SDK bin directory and use -source-path option to tell the compiler where the source is located. -doc-sources will tell the compiler to generate output for the current directory and all subdirectories recursively. If you run this utility from any other directory other than "bin", you will get NoClassDefFoundError.

I haven't tried running ASDoc in Windows yet, but it should be very similar to running it on a Mac. And since ASDoc is a command line java utility, you would think it would run on Linux too. Well, for all you Linux users out there, this is the message I got from the compiler:

Error: Could not create toplevel.xml: Unknown OS for ASDoc. ASDoc will only run on Windows and Mac OS X.
That last sentence is pretty self explanatory :)

ASDoc is nicely documented here.

Cheers
Marko

Install SSL certificate in ColdFusion

If you are using self signed ssl certs for web-services you may run across the following error:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated

If you browse to the URL using Firefox you will likely get a certificate warning and a prompt which allows you to continue anyway despite the warning. With CF you can't do this so you have to add the certificate to the list of "known" certs using the java keytool.

Instructions:

First - save the certificate to your desktop using Internet Explorer or the Cert Viewer Plus plugin for firefox.

Then open a command prompt and go to cfusion jre/bin directory

$ cd /opt/jrun/jre/bin

From here you should have see access to the keytool application which is how you modify the security settings.

You can show all certs:

./keytool -list -storepass changeit -noprompt -keystore ../lib/security/cacerts

or Import a new cert:

./keytool -import -storepass changeit -noprompt -alias mynewcert
-keystore ../lib/security/cacerts -trustcacerts -file mynewcert.cer

Explanation of the options:

  • alias - this is a friendly name for the cert that you can specify
  • storepass is the password for the security store. Unless you have already changed it it will be 'changeit'
  • file - this is the certificate file you have downloaded in your browser.

If it reports the following error:

keytool error: java.io.FileNotFoundException: ../lib/security/cacerts
(Permission denied)
Check that you have write permissions to the cacerts file.

Note: If you are a Mac user you will find that ColdFusion may not have JRE directory installed under JRun. This is because CF uses OS X Java install, so you will need to run your keytool command from the following directory:

/Library/Java/Home/bin

To find out the location of your Java Home, log in CF Administrator and click on Settings Summary.

Hope it helps and thanks to the other people who have posted about this.

Sources:

Useful package for Ubuntu - ubuntu-restricted-extras

I just happened across this package today - and while I had pretty much everything already installed it is a handy way to add some of the essential packages which cannot be distributed in the main Ubuntu distribution due to licensing issues.

It installs the following:

  • Flash Player 9
  • Sun Java 6.0 runtime and browser plugin
  • Microsoft Core Fonts - Arial, Verdana etc
  • GStreamer Video Codecs

Installing is as simple as:

sudo apt-get install ubuntu-restricted-extras
or search for it in Synaptic.

Cheers, Mark

More Entries