Seeing as how I have STILL not heard back with meaningful review from the Academy of Chocolate (it seems like they would have needed to put this information together at the time of judging, not months later) I've been thinking a lot about flawed review systems. The thread on the ethicalness of chocolate reviews only exasperated my frustration with this topic.
I should be able to say that I think Amedei is an overrated product that has never won an award that wasn't bought and paid for prior to the first sample being tasted without fear of retribution. But the fact is, I can't, none of us can. In fact the only people who can are non-industry professionals and who cares what they think?
I think I've come up with a peer review system that would actually work, I post it here looking for... peer review ;)
Members sign-up and their identity is verified and then held confidentially.
Members are assigned an ID number to identify them to the public.
Members are free to review products as they wish, with the knowledge that while anonymous, they could still be held accountable for comments that are libel.
Members are transparently linked to their products and bear the aggregate rating of those products.
Members are prevented from commenting on their own products.
Members reviews may not be isolated, minmizing inference issues.
Algorithms to detect collusion are simple enough, but most likely not needed.
I wonder how many professionals would take part in such a system?