Search Engines and the Illusion of Comprehensiveness

Lots of information is ridiculously easy to find these days.  GDP of Argentina?  Check.  Last quarter’s earnings at Google?  They’re everywhere.  But as soon as you start trying to do more in-depth research, you become painfully aware of the ongoing limitations of the current search engine paradigm.

For example, try doing basic background research on, say, Oracle’s marketing strategy.  You’ll quickly find that out of the top 50 results on Google, half are either redundant or spammy (companies like Highbeam trying to get you to sign up for their subscription services), another 40% have marginally useful but redundant information (all linking to the same press release, for example), and only the remaining 10% really provide much value.

At the same time, a lot of useful sources just don’t come up.  Whether it’s a 10-K filing, market research report, case study, blog post, discussion forum, or whatever, there is a pretty much infinite set of data that will not show up on the first page of your Google search results.  There are a bunch of reasons for that.  The page may be too specialized and not have enough authority to show up on a general search.  It may be gated, although various online publishers are getting better at making sure content is indexed by the search engines.  It could be behind a pay wall.  The scenarios are practically endless.

Experienced business researchers know how to wade through this morass with a minimal amount of wasted time, but it’s still an effort and a drag on productivity.  Business analysts shouldn’t have to waste energy manually filtering search results, but that’s the reality these days.  In fact, a ballpark estimate is that 50% of a business analyst’s time spent on research is consumed by monkey work – manually checking search results, tweaking keyword combinations, and hoping against hope to find that needle in the haystack, the one crucial fact that will make the whole effort worthwhile.  That time could be better spent doing real thinking work.

Why is it that the research experience is still so bad?  Mainly, it’s because no one has really focused on business users.  Search engines are still seen as general-purpose tools, and Google makes much more money off a search for a new dishwasher than on a business query.  It’s difficult to monetize a business search with a text ad.  More importantly, Google makes money when people click on ads, even if the search result is bad.  Business users need a different way to pay for better search functionality and a service that is focused specifically on their needs.

What do you think?  Is my search engine rant on target, or am I being too pessimistic?  Are there other pains to using search engines that I neglected to mention?

Related Posts:

blog comments powered by Disqus