Tech Reviews: The Media Responds

The perceived conflict of interest between tech reviewers and advertisers is a frequent topic of my ranting here on Cyclocosm, and one of the nice things about having decently-sized and interested readership is that when I post something, word has a way of getting around. Occasionally, people even feel compelled to contact me and respond.

What isn’t so nice is that my fast-and-loose production style does occasionally lead me to make statements that are not entirely accurate. So first, a correction brought to my attention by BikeRadar’s Jeff Jones:

When I first examined analytic data for the three sites, this is the graph I saw, and is the basis of my assertion that people hadn’t “flocked with ovine mindlessness to the new CN, and to BikeRadar”:

However, if I had simply taken the time to look at the numerical data, I would have seen that BikeRadar’s traffic had increased substantially since the CN redesign. Viewed here without the other two sites, the trend is clear:

Jeff also had some feedback on my argument that the undue influence of advertisers is reflected in tech reviews that invariably skew positive:

As for our reviews, we will give products bad ratings if they deserve it. Most companies these days either make good stuff or they go out of business. We try to be as discerning as possible and we always stand our ground if an advertiser complains about a bad review (unless there’s an error). How do you give something a really bad mark if it does what it says for its intended users?

In a later email, he clarified:

Also as a rule, reviews and ratings should be done from the point of view of the intended user and compared to other products in its class. So a cheap bike can get 4/5 if it’s good value for money and it does the job it’s meant to. A hugelly expensive bike can get 3/5 because although it’s generally amazing and you’d ride it over the cheap bike any day, it has a few imperfections that you really think should be ironed out for the money you paid for it.

And a can of drink can get a really bad review if it’s overly hyped by the marketing company: [link]. The marketing company were quite shocked when we posted this review because every other media outlet they sent it to had clearly just based it on the press release and didn’t actually taste the stuff.

In a separate conversation, Charles Manantan of Pez Cycling News also took the time to offer some feedback. He reiterated the notion that—especially at the high end—no one makes lousy stuff anymore. While it’s possible to gather hard data (and he does measure weight, stiffness, etc), the objective differences between high-end bikes and components are so small that less readily-defined traits (road feel, handling, etc.) become far more significant.

From a consumer prospective, he suggests, this data is almost certainly more relevant. After all, a Cat 2 doing criteriums every weekend might want a lot different character / ride quality from a 6.8 kilo, $6,000 dream bike than a 50-year-old who focuses on day-long rides through the mountains. Either rider might hate the choice of the other, but that wouldnt make either dream bike bad.

Charles also hastens to add that he doesn’t collect income from his work at Pez or from bike manufactures specifically to insure free rein over the products he tests and the reviews he writes about them—and because he doesn’t want “money spoiling [his] fun”.