Quora: Has Social Trumped the Semantic Web?

Long long ago, in that distant, ancient era of the 2000’s, we heard unending chatter about this thing called ‘the semantic web’.

The semantic web was supposed to be like the holy grail of technology. At its heart, it was meant to do two things: to make all that impersonal data online accessible in regular, everyday language; and to make connections between that data in ways that aren’t always readily apparent.

There were a bunch of promising apps. Two of the biggest were Twine – which has since become evri – and Glue, which is now Getglue.

But now, those two semantic apps are social apps. And in 2011, talk of the semantic web has fizzled. Instead, these days, all the talk is about Facebook and, lately, Quora.

So what happened?

The Unsolvable Problem

Semantic technology has always proved incredibly difficult because, on a fundamental level, technology and humans approach data differently.

While algorithms look at data as parts of a complex relationship of certain data sets to other data sets, human beings look at data as either a single piece of information that is needed – “what is the best way to bake these muffins?” – or a part of broader story about lots of information – “what’s the best way to invest my money?”.

That information is out there on the web. But it’s hard to find, because our methods of search don’t always know what to connect to what. Does a search engine know that you want those muffins gluten free and low fat? Or that you like brown sugar on top? Or that part of your savings are in another country? Or that you plan to retire at 50? How does it know what you’re after?

That was what semantic was supposed to solve. By using complex algorithms that got smarter the more you used them, it would be able to ‘make sense’ of data in a way that was relevant to you.

Trouble is… they were never very good. ‘Training’ an algorithm to know what you want is not only incredibly time-consuming, you are also still left with less than ideal results at the end.

The bottom line is that, generally speaking, our tech just isn’t sophisticated enough to give you what you want. Sure, some basic examples like Netflix’s recommendation engine are pretty good. But even that ‘basic’ one that simply tells you about movies took years of work. So clearly, we’re a long way off from a search engine being able to tell you ‘what is the best way to arrange my furniture?’

Social Solves ‘the Data Problem’ Better than Semantic

But there’s another looming issue here: social. See, the semantic web was supposed to take data and transform it into the way humans speak.

And you know who else does that? Humans.

The whole aim of the semantic web has been trumped by social, and this happened for two reasons:

• Asking people – particularly those who know you – gives you information far more relevant than either the semantic web or Google. Simply put, the results of social are better than semantic. This has ramifications not just for the semantic, but search in general.
• Your online social circle is far bigger than your real life social circle. If in real life you could get advice from a few friends at a time, on Facebook or Twitter you can get it from many many more. The ‘weak ties‘ of social media mean individuals may not ‘know’ 800 people, but they can sure get info from them.

The capacity of social media to curate, filter and suggest information took the wind out of the sails of semantic, in large part because it simply performs those functions better. Your friends know you, and even Twitter friends you’ve never met can make (for now) make better suggestions than an algorithm because there’s more information to work with and there are more perspectives to work from.

This is one of the reasons that Quora has taken off. Google is great for snippets of simple data – movie times, directions, etc. But for questions that matter – “how do I succeed?”, “what will change in the future?” – you need to speak to other people, because only other humans will be able to provide their own perspective.

So, to trot out a tech blog cliche, is the semantic web ‘dead’?

Probably not. Google is just now starting to really struggle with how easy it is to game their search engine, and will likely start to make their search algorithm more dependent of semantic tech. Similarly, some small startup with some crack programmers may be able to finagle a new algorithm that is better at giving people the info they want.

For the time being though, people scared of the singularity take note: for now, people beat tech.