Can you trust the ratings on Vivino, the crowdsourced wine app?

There are over 87 million wine reviews on Vivino, which claims to be the most downloaded wine app in the world. We wanted to know: Are these multi-source wine reviews reliable?

The San Francisco startup, which according to Crunchbase has raised $222 million in funding, allows users to search for specific bottles, search for similar recommendations, and find out where a wine is available to buy. Most importantly for our purposes, it invites users to upload their own individual wine ratings, on a scale of 0 to 5, and to describe the wine’s flavors as well as other physical attributes.

Just as people love to debate the merits of Yelp and Rotten Tomatoes, wine bloggers have been questioning the virtues of Vivino for years. Can we really trust the masses to rate and review something as complex as wine, they ask, or should we leave that to professional critics and sommeliers, who really know what they’re talking about?

So we did an experiment. We compared reviews of some of the top-rated Vivino wines submitted by California laymen with those of a wine critic (me).

First, we asked Vivino to share data on the top-rated wines by app users located in California. We then separate the wines into six price segments: under $10, $10-$25, $25-$50, $50-$100, $100-$200, and $200-$500. We identified the highest rated wine by California users within each segment.

After buying those six wines, we did a tasting at The Chronicle newsroom. Without looking at Vivino user ratings and descriptions, I tasted each wine and assigned my own rating, evaluating the characteristics of the wine, whether it was tannic or smooth, sweet or dry, smooth or sour, and named three main flavor descriptors.

The results surprised us.

For most of these six wines, my rating was within a few decimal points of the average California user rating, and our perceptions of the wine attributes closely matched. Additionally, when asked to name three primary flavors in a wine, the descriptors I came up with were often the same or very similar to the three most common descriptors reported by Vivino users.

I agreed with 277 Californians who reviewed the Billecart-Salmon Brut Rosé at Vivino, for example, that the champagne tasted like strawberry, lemon, and green apple. However, I enjoyed it a bit more than they did: California users gave it a 4.6 out of 5, while I gave it a 4.8. (What can I say? I love pink champagne.)

We had some notable disagreements, especially when it came to the most expensive wine in our tasting, the $200+ Caymus Special Selection Cabernet Sauvignon. Although we describe the wine using very similar terms, the people at Vivino liked the wine much more than I did.

Leave a Comment