There’s been a host of news this week as the deal between Microsoft and Yahoo was finally inked. The net result: Yahoo drops its own search engine and adopts Microsoft’s Bing, increasing the market share for Bing to 28% against Google’s 65% and leaving 7% for everyone else (stats in the US)
Other chatter recently about search has been how real-time snippets of information, like Twitter updates, change the dynamics of search results. If you search for information on Google it’s unlikely you will see any Twitter results there. How can 140 characters rival an entire web page for relevance? But can it change?
Travel back in time to around 2003, and Bill Gates made a comment that, at the time, I thought was wrong. From memory it went along the lines “Search is done, there isn’t much more you can do to improve on what’s out there”. Of course, this was before Google IPO’d and people realised just how much money was being made on the back of those little text ads. All of sudden, improving search/winning market share became a much more interesting prospect.
But whilst I disagreed at the time, recently I’ve changed my mind. I think the comment was spot on for search as we know it today.
Relevance on Google is far from perfect. If you do a search involving words that have any kind of commercial value, chances are the result you really want is buried somewhere on page 5 or beyond. Hilariously, sometimes the ads help. If you are looking for a particular hotel chain to book a room, if they’ve paid enough for the ad space, they’ve got a better chance of appearing on your first page of results than with just the web site alone. SEO companies continuously learn how to exploit Google’s algorithms to promote their customers web sites in your results regardless of whether their customer is the result you want to find. But I still doubt there’s a better way than PageRank and friends to determine relevance of general web content.
Social media offers the opportunity of a new form of algorithm – instead of PageRank, evaluating a page based on the incoming links to it (a link from a high profile site carries more value than a link from an unknown site), how about introducing SocialRank? Evaluate the links shared by people that are then shared by other people. Sites like Facebook, Twitter, Friendfeed, Technorati, MySpace, etc. etc. contain vast social networks. I share a link, people like it and/or share it with their connections and so the link spreads.
Can we apply a rank to a page based on how the links spread? We probably can. But it won’t work.
Take FriendFeed for example. A high profile account like Robert Scoble’s means that everything he shares has a high probability of being noticed and re-shared. Does that mean anything? The same item shared by someone else might be ignored. A computer wouldn’t show such bias. If you want an idea of how bad search results would be if based on what people share, take a look at Twitter’s trending topics at any point in time – apparently that’s what the majority of people are twittering about. Would any of it help find what you’re looking for?
Search is useful and still a key part of the Internet. But increasingly we use other means to find information. On social sites such as Twitter and Facebook, we follow people we either trust or are interested in and discover information without ever looking for it. If I want a book, I often use Amazon and check the reviews. Random stuff to buy? Try eBay first. Need information about a topic, off I go to Wikipedia. If I’m looking for something I’ve read before and liked? I use the custom Google search on my web site – which includes everything on the web site and my FriendFeed account. FriendFeed aggregates everything I share through Google Reader, Delicious, Twitter, blog, web site, Slideshare… and any other service I upload stuff to.
Google is relegated to answering one-off questions (I couldn’t remember the Mac commands for a screen capture just now – easy result on Google) and desperate searches (old news, travel information) that struggle to be found anywhere.
How can Internet search be improved?
The easiest method to improve relevance across a broad range of content is to separate the results into buckets. Within enterprise search solutions, we call this federated results. Present Twitter and other real-time snippets in a separate list to standard web page results but on the same page. Use SocialRank for time-specific results such as news and travel, see how quickly a link spreads through social networks. Nuances could be included to make it difficult to game by one individual or organisation.
To give a simple example, the image below shows federated search results from a site I host on the Internet for clients. (No prizes for guessing the software being used.) I use it to show them what Google doesn’t tell them. In this example, I just entered the name of a company – Lloyds TSB. Results have come back from Twitter, Bing, Technorati and FriendFeed. (Flickr, YouTube and others are also included but snipped from the screenshot)
The Twitter comments aren’t pleasant. But better to know what your customers are saying and deal with it than not know until they’ve all left. I did the same test for a client recently and they found out that somebody had posted a Tweet asking if anyone from said client was on Twitter. Nobody was, they are now.
In short, search does matter but no more so than yesteryear whilst the format stays the same – a single page of mushed up results served with a side and topping of ads.
Whether you use Google or Bing will mostly come down to preference. (And some of that preference is more political than technical.) The relevance is ranked slightly differently – I find Bing seems to prefer domain name matches. But the differences are incremental and barely noticeable. As demonstrated in the two images below.
Same search as before – Lloyds TSB. Both display one non-Lloyds TSB domain result and both put it in 4th place. Interesting how they both display the top result the same but with different sub links to pick from. Decide for yourself which format is the best. Looking at them side-by-side here, I actually prefer the user interface for Bing. But people don’t switch browsers or search engines to discover what’s different. They change when they hear there is an alternative that is much better and easy to use. Microsoft’s challenge is that many people don’t even know there is a difference between a browser and a search engine, and some think Google is both (and they haven’t heard of Chrome):
Final note: this is one of the most rambling posts I’ve written in a while. If you got far enough through it to be reading this, thank you! 🙂 This was one of the posts that’s been floating around my head for months and it just needed to spew out but is far from perfect… I’ll endeavour to make the next one more to the point.
Joining Dots 
The youtube video is odd. People don't know the difference between a browser and search. No wonder companies just want "google inside" when asked about their intranet pages. Results are probably skewed by the camera being controlled by Goolge.
Yes I'd agree the results are probably a little skewed. The giveaway is the guy that chuckles at the cameraman when saying Google.But still a healthy reminder to us Internet old-timers that what seems obvious to us isn't always so for others.