A challenge to Google has been brewing for a few years, as business and professional users have experienced long-term relevancy problems.
The idea of folding verticals into general search engines is a concept that has surfaced before. Aaron Wall discussed the topic in the SEO Book blog last year, and Danny Sullivan covered it indepth in 2003 on Search Engine Watch in "Searching With Invisible Tabs."
I got to thinking about the idea recently when a friend mentioned frustration over not finding critical information that could have been found quickly in a vertical search engine. The vertical was nowhere to be found in Google. Maybe it was buried somewhere deep in the abyss, or relegated to the supplemental index.
Google Hell
A recent Forbes article by Andy Greenberg, "Condemned to Google Hell," highlights a condition faced by a number of unfortunate websites. Google Hell refers to a situation where a highly ranked site or a brand new site is put into Google's supplemental index for long periods of time without explanation or recourse.
The supplemental index is a holding place for low-quality pages or pages designed specifically to achieve high rankings without providing quality content to produce a good user experience. It was created to take a load off the GoogleBot, which crawls an increasingly vast web, to index and rank web pages.
Supplemental index woes
Supposedly, pages in the supplemental index are crawled less frequently than pages in the main index. When a site goes from the main to supplemental index, it can remain there six months to a year before reappraisal. This cuts off the bulk of the income for most sites, and there is nothing they can do about it. As the web grows exponentially, Google seems to be putting more and more sites in the supplemental index.
Many times when this happens, the affected website made changes to improve its rankings but instead was penalized without explanation. A suspected reason for this is questionable linking activities: buying links or accepting links from low-quality sites in order to improve PageRank. Other times, a site can be victim of black-hat search consultants.
At any rate, if a site suspects certain changes caused the penalty and then takes steps to remove the changes, this doesn't make any difference because once in the supplemental index, the site usually remains there indefinitely.
That's why it's called Google Hell.
Criteria for supplemental index inclusion
Search technicians and web designers have tried to divine the criteria used for inclusion in the supplemental index, but Google does not provide any explanations. Some of the possible reasons include:
Duplicate content, which can sometimes occur inadvertently
- Low-quality sites with minimal content and graphics
- Sites with few quality inbound links
- Sites that buy links
- Sites with links from bad neighborhoods
- New sites of low quality
Google relevancy problems
Google has encountered a challenge in the past few years as business and professional users have experienced relevancy problems. First, Jupiter Research identified an opportunity for vertical search engines in 2005, stating that general search engines were good at cataloging vast amounts of information but not very good at helping users make purchase decisions. Jupiter concluded that vertical search engines could prosper because they could provide more relevant results in their niches, and they have the ability to offer better keyword pricing for paid search. Jupiter predicted that vertical search would drive industry growth. That was two years ago, and verticals are still striving to become universally known.
Secondly, Outsell conducted a study last year titled, "Vertical Search Delivers What Big Search Engines Miss," reinforcing the claim that vertical search engines have opportunity for growth due in part to the dissatisfaction with the three major search engines (Google Yahoo! and MSN).
This report found the average internet search failure rate was 31.9 percent. Outsell identified market trends fueling the growth of vertical search engines, including failed general searches and rising keyword prices in paid search.
Lastly, Convera surveyed more than 1,100 online business professionals to examine their search practices, successes and failures. The findings echoed those of Outsell in that only 21 percent of business professionals felt their search queries on general search engines were understood, and only 10 percent found "needed" information on their first attempt in major search engines. On the other hand, 90 percent of them felt vertical search engines could provide more relevant content, and 85 percent felt they could find content not indexed by general search engines in vertical search engine databases.
The paid search conundrum
Google gets 97 percent of its revenue from paid search. However, keyword prices are so high that many advertisers are priced out of the market. This, too, opens an opportunity for vertical search.
In 2005, Jupiter reported that paid search spending in the U.S. was highly concentrated into four categories: retail, financial services, media and entertainment and travel. This group of advertisers accounted for 79 percent of the billions spent on paid search in 2004. The verticals in these industries could challenge Google and Yahoo for paid search revenue.
Because paid search has more or less stabilized on general search engines, there's an opportunity for the verticals to take up the slack. This could cause a shift of ad dollars to multiple niche search engines, especially if they can provide lower keyword prices and more relevant clickthroughs resulting in higher conversion rates. Perhaps the volume of traffic will not be as high on the verticals as on the majors, but there would be fewer irrelevant clicks and better advertiser ROI as the click-to-sale ratio increases with a targeted audience.
The Google dilemma
Google's dilemma is that it is so successful, it has priced itself out of the market, especially with mid-sized and smaller advertisers. These advertisers represent a big chunk of potential revenue. Google is losing that business because keyword prices have become unaffordable in certain categories. The verticals will be courting all the advertisers that can't afford Google.
What can Google do to keep from throwing the baby out with the bath water? One idea is to put vertical tabs in the OneBox.
Vertical OneBox tabs
Google.com uses the OneBox mechanism to provide users with access to information from various different databases such as Images, Video News, Maps, Mail, et cetera, in the upper left-hand corner of the Google home page. There is also a link to "More" for access to additional databases, all searchable from a single text entry box where users can type a query.
If Google were to add vertical databases to the OneBox, this could benefit users while expanding Google's inventory for paid search ads. The multitude of vertical databases could be added without removing them from the main index. However, there are so many vertical search engines and directories that it might not be feasible to make them all searchable from the OneBox.
Sullivan covered the pros and cons of this in the Search Engine Watch article mentioned above. In this article, he jokingly theorized Google could look like a laundry list of vertical links taking up half the homepage. Then he went on to explain why this is unfeasible.
Two-click approach with invisible tabs
What Google could do instead is point vertical queries to the relevant vertical. There are hundreds of verticals with names that are hardly mnemonic; therefore, most users don't remember names and URLs when they need vertical search. If they were able to immediately find the right vertical on Google, they wouldn't need the name. This two-click approach could take users through the search refinement process without the hassle usually faced on major search engines. While Google recently announced that verticals will be folded into its web results with Universal Search, Google's verticals do not include the multitude of topic- and industry-specific verticals on the web.
General search engines could be the bridge to more focused vertical searches, but this hasn't happened yet. In his article, Sullivan suggested implementation of invisible tabs that would work behind the scenes, taking vertical queries to the appropriate engine or directory. In some cases, this happens already. For instance, a search for "pictures of DNA" on either Google or Ask will yield DNA images on top that click to verticals. But this doesn't work for all verticals. While some search engines showed an interest in invisible tabs when queried by Sullivan in 2003, none have fully implemented this to date.
In the meantime, the dissatisfaction with general search is growing, and multiple new search technologies are bent on beating Google. Perhaps the idea of invisible tabs can be revitalized today, folding all the verticals into general search.
Jason Prescott is CEO of JP Communications. Read full bio.
No comments:
Post a Comment