Search A Tail Of Mathematics Part 1

Writen by Ken Metcalf

As an Engineer I have always been fascinated by the simplest answers to complex problems. The poster boy for this of course was Albert Einstein when he coined his famous e=mc². Recently I spotted a reference to an article which stated that search does not follow the 80/20 rule, which I do agree with. In terms of search, the long tail is a key factor and not the bulk of search terms upfront. This set me thinking; is there a relationship between the long tail of search and some fundamentally simple mathematical equation?

In a previous article, I alluded to the similarity between some common mathematical equations which describe certain social behavior and the search theory of the "long tail". My thinking evolved when I noticed a similarity between the curve used to represent the long tail and another curve I had seen during the course of my studies years ago.

The mathematical theory which I am referring to is that of exponential decay, represented by the formula exp(-ax). If one were to plot the curve of exponential decay for various values, the curve looks remarkably like the curve for search representing the long tail. There are many examples of things around us at a social level which obey the laws of exponential decay. Also in the sciences we see many phenomenons like air pressure vs. height above sea level that obey the law of exponential decay.

So to try to put some weight behind my assumptions, I decided to do an actual experiment. I used the Overture Keyword Suggestion tool around the keywords dog food. I took the list of keywords it suggested and these keywords I then ranked from most frequently searched to least frequently searched.

The list contained 97 Keywords, with the most frequently searched 2983 times and then tailing down to number 97 with a lowly 272 searches. To see the graph representing my findings, please click here.

Plotting these keyword searches out on a graph vs. number of keywords confirmed my initial suspicion that the curve of the tail of search resembled that of an exponential decay curve. Next, I created a second curve using exp(-ax). For those of you not familiar with exponential decay x represents the number of terms and a is a constant. I set this constant to 1/25 or 0.04 and plotted it out. The result was very encouraging as one can see a clear correlation between the data from Overture and the calculated data.

So what does this all mean? Well, firstly it shows that although Search Engine Marketing does not necessarily obey the 80/20 principle, in terms of keyword research it does obey another simple mathematical principle, that of exponential decay. Secondly, the fact that we can mathematically work with equations that define the long tail opens up several possibilities especially in terms of investigating the quantity of keywords in the tail vs. the bulk high search keywords.

In Part 2, I will investigate some of the possible applications of the exponential decay predictions in terms of the search tail. A key factor here will be the investigation of quantitative research rather than financial research. Ultimately we will have to tie these two together to optimize a keyword strategy.

Ken Metcalf is a professional engineer, also holding a business commerce degree. He writes articles in his spare time on technical topics ranging from low level programming to The SEO Blog which deals with search engine optimization. For more information on using Search Engine Technology visit The SEO pages here