I’ve been waiting for these reviews from The Verge ever since I heard about these devices - the Lumia 920 and the HTC 8X. The way it seems is that there’s a strange misguided bias against Microsoft and Windows in The Verge HQ. I don’t think that Microsoft and it’s partners are getting the credit they deserve for the effort and innovation being put into so many Windows Phone.
It’s a harsh industry and I know that nobody gives you concessions for effort - you don’t get points for trying, you only get acknowledged for results. But I think Microsoft’s latest efforts are being unfairly put on the back-foot in these reviews for purely existential reasons, these new devices get a score shaved off them simply for being Windows Phones. I have absolutely no objections to The Verge grilling these devices and the platform they run on for that chronic app deficiency. It’s an issue which certainly plagues the young platform and until it gets fixed it’s an inherent short-coming in the user experience which readers should be aware of.
Where I do have a problem is The Verge’s numbered scoring system which is kind of crap for two reasons - it’s inconsistent, and it poses a perception problem. I think numbers are great when used properly, but they can be a precarious conclusion in product reviews. Engadget dumped their numbered scoring system for product reviews because of this, apparently numbers are a cheap and hasty conclusion that betray the depth and complexity of reviews. And the author’s just didn’t feel that a number could justify the weight of a product’s strengths and weaknesses because these are all relative to how the author sees it, and not perhaps how readers will feel about it.
But wait…isn’t that the point of a review? To see an author’s opinion, as opposed to an author’s opinion on what he thinks my opinion would be? After all, I think I’d be the better judge of that. I’m curious to know what different people and publications think, that’s why I read reviews. If I wanted some purely objective overview of how good something is the spec sheet would suffice. And numbers provide another immeasurable benefit - solidity. By that, I mean that numbers are numbers, they’re just digits. Unlike prose, it’s not open for interpretation or misguidance. A 7 out of 10 is a 7 out of 10 for both the enamoured and the disgusted.
So back to The Verge, I love numbers, and I embrace opinions - but I also think that publications have an obligation to some sort of pseudo-objectivity and that reviews can’t be totally hinged on the pretensions of certain individuals. I think that publications themselves - as a collective of authors - should be the voice behind product reviews instead of individual authors. I think individual perceptions and inclinations play far too big a part in reviews on The Verge.
I think that’s a problem that hurts my trust for The Verge product reviews and why many are markedly inconsistent and somewhat biased and unfair. The issue shines particularly well when it comes to Windows Phones which always seem to fall unfavourably with the Verge reviewers.
Take the HTC 8X and Nokia Lumia 920 - both are undoubtedly fantastic phones. Speaking as objectively as an opinion can possibly be, I doubt anybody can pick up one of these and say that these suck with a straight face. But their scores don’t portray them as too flash - both of these devices received a 7.8 and a 7.9 respectively. Actually these scores aren’t too bad but relative to competitors that have been reviewed recently such as the iPhone 5 with a venerable 8.8 and the new Nexus 4 with an 8.3, it’s dismal - particularly for two phones which Microsoft would happily regard as ‘flagship’.
I mean, ‘ouch’, people have obviously engineered and innovated the shit out of these two phones and to be handed a rather tame 7.8/7.9 from arguably the most influential technology blog on the planet has got to hurt. But when you look at the details the party pooper of the overall score was the ‘ecosystem’ category, which landed a 5/10 on both these phones - obviously Windows Phone’s lack of third party apps being the soul cause. Aside from that, these phones are stellar. The ecosystem category is the one that hurts all of Microsoft’s newest platforms on The Verge reviews, it creates a mockery of the system because it essentially means that it’s impossible for any Windows Phone to be notably ‘good’ which is absurd.
The fact that the ecosystem category, the single low-scoring anomaly on the latest Windows Phones is an unchangeable existential problem is something which absolutely needs to be taken into account when deciding on the final ‘Verge score’. Perhaps people will understand that these are great phones sans the ecosystem problem, but perhaps people won’t - it’s all about perception and a 7.9 doesn’t leave quite a sweet taste in the mouth when compared to the much more tried and true iPhone with an 8.8. The scores on Windows Phones need to be more heavily weighted because the ecosystem oddity is hurting them much more than it needs to.
Additionally, inconsistency is something which plagues these reviews. I took it upon myself to look at several first generation Windows Phone reviews done on The Verge and found something peculiar. Both the HTC Titan and Samsung Focus Flash which were both earlier - and now non upgradeable - Windows Phone models (running Windows Phone 7/7.5) scored 6/10 in the ecosystem category. Is The Verge really proposing that the Windows Phone ecosystem has gotten worse in Windows Phone 8? That’s a factual impossibility since all Windows Phone 7 apps are Windows Phone 8 compatible. These reviews were all written by different authors, but there needs to be a level of consistency before these reviews become worthless as an aggregated entity and worthless for product comparisons.
There also seems to be a short-sighted assumption that an ecosystem is all about apps. It’s absolutely not. In fact, it’s fair to say that ecosystem product integration in Windows Phone is second to none, this is the story of Windows Phone’s ecosystem that nobody acknowledges amidst the Marketplace bashing. Find me a platform that integrates with Xbox, Skydrive, Outlook, Office or Skype as seamlessly as a Windows Phone and then we can talk.