The Problems with Market Research Reports

Have you ever looked at the table of contents for a $4,000 market research report?  It looks so enticing, like all of your questions about the market or competitors will be answered.  Market size, growth rate, competitive landscape, customer dynamics, etc. – it’s like 300 pages of candy for someone trying to write a business plan or get data on an industry.

Unfortunately, research reports almost never live up to the hype, for a number of reasons.

Black Box Numbers

One of the most sought-after aspects of market reports is the market sizing numbers.  Everyone wants a supposedly reliable source for that information, and companies like Forrester, Gartner, and IDC are particularly happy to oblige.  Of course, estimates vary widely across sources and from year to year, and there’s no way to reconcile them because you usually don’t know how they were calculated.  Most of these market sizing figures are the equivalent of sticking your finger in the wind, so take them with a grain of salt.

Fluff

Most market research reports are written for length rather than utility, and they’re stuffed with long-winded descriptions of the methodology, company profiles that repackage annual report information, and a variety of other boilerplate.  You can expect perhaps one out of every four pages to contain something useful.  For example, a recent report on business information markets (think Thomson Reuters and Bloomberg) fills quite a few pages describing Google’s acquisition of DoubleClick and AOL’s acquisition of Bebo, despite the fact that neither has anything to do with B2B information markets.

Different Market Definition­s

You would think people define markets the same ways and that there’s little room for confusion, but quite the contrary.  I’ve had all sorts of odd experiences where the report I thought was perfect turned out to be about a related but entirely different market.  For example, I was doing research on blood serum products used in pre-clinical pharmaceutical testing and found a 400 page report on blood products.  Perfect!  The only problem was that it was about blood bank products –whoops!  I’ve had similar experiences with other areas from packaging to industrial machinery.

Shallow Analysis

Often, report writers also take the easy way out by overly simplifying their analysis or failing to thoroughly vet it.  One example might be defining industry scope and size by NAICS codes, which often fail to capture some nuances of what should be considered part of an industry.  Some analysts also pull a bait and switch.  One firm is fond of analyzing “latent demand,” which bears little resemblance to actual demand.  But then, it’s not supposed to – it’s latent.  Finally, the same business information report I cited earlier claims that advertising revenue for subscription, business-oriented services will go up with increasing broadband penetration.  That conclusion just doesn’t make sense, considering that most information-intensive firms already have broadband connections.  Consumer broadband penetration won’t make any difference.

Packaged research reports are still a necessary evil.  It’s useful to have a third-party estimate of market size and to take advantage of the information distilled in these documents.  Just be aware of their shortcomings and set your expectations very low.  If you need more than a few numbers and a few factoids, go in knowing you’ll need to do more legwork on your own.


Related Posts:

blog comments powered by Disqus