Click Through Rates and Search Classification

An important number in search engine optimization is the Click through Rate (CTR) for each position on a Search Engine Results Page (SERP).

In a previous article, Click Through Rates. The AOL Data Revisited, we looked at one of the historically important sources of “click through” statistics: the 2006 AOL Data Logs. SEOers have long been interested in the large 71% drop in click-throughs, between the #1 and #2 positions and the impact this has on potential Web sales. This large drop has motivated companies to strive to obtain the #1 position.

But how accurate are the AOL numbers? Does the ROI justify the costs involved in reaching #1? One of the problems with the AOL numbers is that they assume that all searchers search the same way and have the same click through patterns. Analyzing the AOL data in greater detail shows that this assumption is incorrect, the search patterns can be quite different, and this throws all the numbers off.

To get a better handle on click through rates, this article revisits the AOL Data Logs and breaks web searches into different classifications: Navigational, Informational, Goods and Services Searches, Comparison Shoppers and Buyers, and examines the difference in click through patterns between the groups.

The AOL Data in Detail

An earlier 2002 study, “A taxonomy of web search” by Andrei Broder (formerly of IBM Research), classifies web searches into three major types based on “query intent”:

• Navigational. The immediate intent is to reach a particular site.
• Informational. The intent is to acquire some information assumed to be present on one or more web pages.
• Transactional. The intent is to perform some web-mediated activity such as Web shopping or downloading.

The Broder study estimates the search percentages of the 3 groups as: Navigational: 25%, Transactional: 36% and Informational: 39%.

Building on this study, the AOL data logs were first broken into 2 major groups: Navigational and Informational, and click through values were determined for the 2 groups. A search was assumed to be Navigational if it contained any of the following: “http”, “.com”, “.org”, “.mil”, “.edu”, “www”, or “.gov”.

Out of a total 20,524,584 searches, 5,507,044 were deemed to be Navigational, this is 27% of all searches, agreeing fairly well with the 25% from the Broder study.

Table 1.0 (below) shows a breakdown of click through data for informational and navigational searches. The navigational numbers confirms that navigational searchers are not comparison shoppers. Over 80 percent of all click throughs are for the first two results, implying that navigational searchers have pretty much made up their minds about what they are looking for and if they don’t find it immediately they give up and try another search. Informational searchers, on the other hand, tend to look at a broader range of results.

Table 1.0. Click Through Data for Informational and Navigational Searches

Comparison Shoppers and Buyers

Of greater interest, from an SEO standpoint, is determining the click through patterns of active shoppers and potential future customers. After looking again at search demographics in the AOL data logs, three new categories were created. The first, named “Goods and Services Searcher”, selects searches for goods and services, focusing mostly on “brick-and-mortar” businesses. Some search query examples are:

• real estate in phoenix
• best new york restaurants
• richmond dentist
• best california ophthalmologists
• and so on…

As noted in the Broder Study it is often difficult to establish the “query intent” of a searcher by looking at the search query alone. There are times when someone doing a navigational search may use a search query that looks like an informational search. For example, someone looking for the webpage “shephardchiro.com” in Portland might use the search string “chiropractic portland” as a “lazy mans” shortcut. If the large drop-off between the #1 and #2 positions is due to navigational searches, it might be worthwhile to separate them out.

In an attempt to “weed out” navigational searchers from the “Goods and Services Searchers” a second new category, called “Comparison Shoppers”, was created. It selects “Goods and Services” searchers who visit (ie click-through) more than one site per search, which should hopefully remove the navigational searchers, or searchers who are not actively shopping.

Finally, the last category, optimistically named “Buyers”, selects “Comparison Shoppers” who after visiting several sites, return to a previously visited site, where in theory they make a purchase/conversion. Table 2.0 (below) shows the search and click through totals for the different categories. Note that the CTR of the Buyers is approximately 2 percent of the CTR of the Goods and Services. As a general rule of thumb the conversion rate for a website is assumed to be around 1-3% so the assumption that the Buyers category are making conversions may be a reasonable one.

Table 2.0. Total Click Through and Searches

Figure 1.0 (below) shows the percentage of Total Click-Throughs (CT/TotCT) for the Goods and Services, Comparison Shoppers and Buyers search categories. The “All” category is the original AOL data from Table 1.0 (All Searches). Table 3.0 (below) shows the data in greater detail.

In general, the percentage of click throughs decreases from the #1 position to the #10 position, but the large drop between #1 and #2 in the “All” category is less pronounced in the “Comparison Shoppers” category and appears to disappear altogether in the “Buyers” category. This could indicate that it is less important to obtain the #1 position when targeting buyers and active shoppers.

Figure 1.0. Click Through Percentages for Different Search Categories

Table 3.0. Click Through Data for Different Search Categories

Click Through Rates for New Customers

When attempting to calculate the return on investment for an SEO campaign it would be helpful to know which click through rate pattern best represents new customers. Although it’s probable that none of the patterns exactly match new customers it seems reasonable that some combination of Comparison Shoppers and Buyers would be the closest approximation.

To that end, a new hypothetical category labeled “New Customers” was created from a weighted composite of Goods and Services, Comparison Shoppers and Buyers. The resulting CTR Pattern is shown in Figure 2.0 and Table 4.0 (below) along with the original AOL data (ALL), Goods and Services, Comparison Shoppers and Buyers CTR patterns.

Figure 2.0. Normalized Click Through Rates for different Search Categories (in %)

Table 4.0. Normalized Click Through Rates for different Search Categories (in %)

Note: for comparison purposes, the click through rate data for comparison shoppers and buyers was normalized by dividing by their respective “conversion rates”, 2% for buyers and 60% for comparison shoppers.

Conclusions and Observations

The way people search differs dramatically depending on what they are searching for. Navigational searchers rarely look beyond the first two positions on a search results page. Active shoppers look deeper. The large drop-off we see in the original AOL Data between the #1 and #2 positions appears to be much less pronounced for active shoppers. From the standpoint of SEO this may mean that obtaining the #1 position is less critical than previously thought.

For SEO purposes, the hypothetical pattern “New Customers” may be useful for calculating click through rates and ROI. To be continued…

Notes and Limitations

The AOL data, while useful, has its limitations. The demographics are going to be different from those of Google. For example, there are probably very few Business-To-Business or .EDU searches.

Also, the data is from 2006. Search algorithms have improved since then. Google is much better at handling spelling errors. New features such as Google Tips, Instant Search and Universal Search have been added as well as changes in Local Search.

New Tools

Recently Google added the ability to its Webmaster tools to analyze Click Through Rates. Several studies have come out this year analyzing this new source of data. (see resources below)

Click-through Resources

• AOL Search Data Scandal (Wikipedia)
• AOL Data, techcrunch.com 8/6/2006
• Collection of sources of the raw AOL Data
• AOL Click-Through Analysis by Richard Hearne, 12/8/2006 (redcardinal) *
• Who uses AOL? New York Times, 9/10/2008
• Overview of Google Rankings, SeoBook, 2008?
• Google SERP CTR Data by Search Rank, SeoBook, 4/15/2010
• Review of the Broder Study, SeoBook, 5/28/2009
• A taxonomy of web search, The Broder Study, 2002
• University of Hamburg, Eye-Tracking Study, 7/26/2006
• Cornell University, Eye-Tracking Analysis, 7/25/2004
• Distribution of Clicks on Googles SERPs, Eye-tracking study, 10/26/2006
• AOL Data and Eye-Tracking, 7/12/2009
• CTR and Google Webmaster Tools, by Neil Walker, 5/11/2010, (seomad.com)*
• The Value of Google Result Positioning, Chitika Research, 5/25/2010, *
• Review of the Walker and Chitika studies, Patrick Altoft, 5/27/2010, *
• Webmaster Tools and CTR, googlewebmastercentral, 4/2010
• Accuracy of Webmaster Tools CTR, Andy-Heaps, 5/20/2010
• Organic Click Through Rate Analysis, Mark Edmondson, 6/1/2010, *
• Is Click Through Rate only 42% for the first ranking position?, RiseDigital, Aug 24, 2010, *