Tuesday, August 31, 2010

Next Generation Search and the Death of Google

Where is search going ? Real-time search ? Mood search? How about "no search" ? Shouldn't the internet learn about me and just anticipate my needs ? There are inklings of cognitive search making noises on the internet fringe. Companies such as Hunch, a collective intelligence app that delivers personalized recommendations based on individual responses to a set of taste-determining questions has taken to social search scene by storm. Google, with the introduction of "Buzz" and "Social Search" has targeted the social network as a tentpole in their bid to extend their dynasty through the Facebook era. For those unfamiliar with Google Social Search, it launched across all of Google in January, after initially rolling out as an experiment last October.When you’re signed in to Google, the service shows matching web pages and other content on the topic of your search that’s created by your friends or others that Google has determined that you’re “connected” to.


Why is social search interesting ? In a word, relevance

A lot of people write about iPhone, so if I do a search for iPhone on Google, my blog post about Facetime probably isn't going to show up on the first page of my results. Probably what I'll find are some well-known and official sites. With Social Search, the idea is you can scrape relevant public content from your friends and contacts.


From links to likes

The heir apparent to social search is without a doubt Facebook. When Facebook launched its Open Graph protocol in April,  it seemed obvious that one of the company's goals was to use the resulting behavioral data to power a social search engines. The company has confirmed that all Web pages that use the network's open graph plug-ins show up in the social network's search results in the same way traditional Facebook pages do...a real buzz-kill if you are Google.

The social network can improve, but it does not solve the question of relevance completely. In the end, relevance is subjective. You can do a search for 'coffee' in Canada and find Tim Horton's website as the most relevant. Makes sense, as that’s the most popular coffee chain in Canada, but for somebody in California, Peete's might be the most relevant result. You can do a search for the ‘49ers’ and be looking for the football team, but a historian may be looking for research material on California.

Social search presents a key evolution point in search, but we need more ! There are three cardinal steps before we realize a truly "invisible web": real-time cognition and authority.

Real-time

The Web's new currency is all about the diminishing half-life of data, and sifting through them for useful information is a challenge for search engines. Its most daunting aspect is not collecting the data. Facebook and Twitter are happy to sell access to their data feeds--or "fire hoses," as they call them--directly to search providers; the information pours straight into Google's computers.

What's really hard about real-time search is figuring out the meaning and value of those fleeting bits of information. The challenge goes beyond filtering out spam, though that's an important part of it. People who search real-time data want the same quality, authority, and relevance that they expect when they perform traditional Web searches. Nobody wants to drink straight from a fire hose.

The benefit of real-time search is a just-in-time composition of tweets and facebook posts relating to a time-sensitive topic, such as which bar is hopping tonight ? Taken to the extreme, real-time search could provide trending data that could allow predictions of where a stock could track.

Cognition

This is the secret sauce of the search world - how do you anticipate user intent ? The notion is familiar to anyone using iTunes Genius or Netflix recommendation tools. Most of these systems revolve around a process called collaborative filtering. Collaborative filtering systems discover new items you might be interested based on your historic preferences such as explicit ratings or implicit click behavior. The correlations that can be drawn can be prescient (all democrats like Star Wars, you are a democrat, so you like Star Wars), or they can be real long shots.   The end goal is to uncover new and and relevant information without you searching for it.

Indeed companies such as Hunch have bet the farm on their ability to generate these correlations using vast data sets of user preferences. Clearly, these techniques presents challenges if you are not Facebook. The key is to be able to get people to provide self-declared data and a list of likes and dislikes...not easy. The traditional method is to get users to volunteer the information through a series of mini-games, and to apply this information by signing up to a search community prior to entering the search string. It would be great if there was a "taste passport" we could port to these communities to exploit the nuanced databases that have been compiled - in fact metastructures such as "open social" is an early portent of such a universe.

Authority

Obvious right ? While social search provides trust, recommendations provide cognition, is the information authentic ? Sure I have friends who know a thing or two about music, but how authoritative are they on the topic of next generation search ? How can we create a peer-reviewed authority index, and how can it be brokered effectively. For instance, if I'm working at Ericsson and I serach for a topic in wireless networking, I want results that my CTO has deemed legitimate - or perhaps I would want a view of the material that all wireless CTOs in my industry has authentticated. A possible approach may be to elect regional, national and global indexes to track peer-reviewed experts divided into categories. IEEE may be a good start, or an analysis of papers that are most referenced (page rank for academia). The incentive would be to a create an authority marketplace or even an acution process where a user who has been deemed an authority could be part of a just-in-time vote to provide his or her "paper of choice" for a given search result (much like Google auctions advertisment) - the reward being accrual of authority credits to be used to "purchase" an authoritative search referral. Ok, maybe that last bit is going overboard.

I'm personally looking forward when the web can figure me out, and start placing that advance order for the next Cure comeback album without me having to go through the drudge of typing it myself, and maybe even cancel it when it figures out that my fellow Cure fanboys decides the album sucks. That is the world I want to live in...




0 comments:

Post a Comment