There are over 87 million wine reviews on Vivino, which claims to be the most downloaded wine app in the world. We wanted to know: Are these multi-source wine reviews reliable?
The San Francisco startup, which according to Crunchbase has raised $222 million in funding, allows users to search for specific bottles, search for similar recommendations, and find out where a wine is available to buy. Most importantly for our purposes, it invites users to upload their own individual wine ratings, on a scale of 0 to 5, and to describe the wine’s flavors as well as other physical attributes.
Just as people love to debate the merits of Yelp and Rotten Tomatoes, wine bloggers have been questioning the virtues of Vivino for years. Can we really trust the masses to rate and review something as complex as wine, they ask, or should we leave that to professional critics and sommeliers, who really know what they’re talking about?
So we did an experiment. We compared reviews of some of the top-rated Vivino wines submitted by California laymen with those of a wine critic (me).
First, we asked Vivino to share data on the top-rated wines by app users located in California. We then separate the wines into six price segments: under $10, $10-$25, $25-$50, $50-$100, $100-$200, and $200-$500. We identified the highest rated wine by California users within each segment.
After buying those six wines, we did a tasting at The Chronicle newsroom. Without looking at Vivino user ratings and descriptions, I tasted each wine and assigned my own rating, evaluating the characteristics of the wine, whether it was tannic or smooth, sweet or dry, smooth or sour, and named three main flavor descriptors.
The results surprised us.
For most of these six wines, my rating was within a few decimal points of the average California user rating, and our perceptions of the wine attributes closely matched. Additionally, when asked to name three primary flavors in a wine, the descriptors I came up with were often the same or very similar to the three most common descriptors reported by Vivino users.
I agreed with 277 Californians who reviewed the Billecart-Salmon Brut Rosé at Vivino, for example, that the champagne tasted like strawberry, lemon, and green apple. However, I enjoyed it a bit more than they did: California users gave it a 4.6 out of 5, while I gave it a 4.8. (What can I say? I love pink champagne.)
We had some notable disagreements, especially when it came to the most expensive wine in our tasting, the $200+ Caymus Special Selection Cabernet Sauvignon. Although we describe the wine using very similar terms, the people at Vivino liked the wine much more than I did.
It’s worth noting that Vivino groups similar-sounding tasting notes into categories. So while I was more specific, the Williams Selyem Pinot specifically reminded me of maraschino cherry, Vivino goes further and groups together the commonly used terms of cherry, raspberry, and strawberry for that wine. Basically, says Vivino, the wine tastes like red berries.
Overall, these results make a powerful argument that when you get enough people together, their collective opinion of a wine will be largely indistinguishable from that of a professional critic. It doesn’t matter if the people who rated the wines are expert or experienced tasters: the crowd is likely to get it right.
I was delighted to see this. To me, it suggests that wine tasting shouldn’t be seen as the esoteric, exclusive activity it’s often made out to be, and that taking blackberry and leather notes isn’t part of some secret wine snob code.
One note: in some cases, due to product availability, the vintage of the wine I tasted was different, by a year or two, from the wine in the dataset.
These are the wines we tasted.
Under $10: Corvelia Cabernet Sauvignon ($9.99) from Paso Robles
- Ester rating: 3.8
- Californians rating: 4.2
- Overall Vivino Rating: 4.1
- Esther Descriptors: Cherry Tail, Oak, Strawberry Jam
- Vivino Descriptors: Oak/Vanilla/Tobacco, Plum/Blackberry/Black Fruit, Cherry/Raspberry/Red Fruit
I was less impressed with this affordable Paso Robles wine, which is available at Trader Joe’s, than my neighbors. Still, I agree with the many reviews that say it’s good value for money. It’s fruity, silky and approachable, with an easy-drinking quality that’s entirely appropriate for a $10 California Cab. The three flavors that caught my eye were cherry cola, oak, and strawberry jam; app users mentioned oak, plum, and cherry, so we weren’t too far behind. It came across to me as slightly sweeter and slightly less acidic than what users reported. In their reviews, some people mentioned the lack of tannins as a plus – at this price, people seem to want something soft and smooth, not sticky or astringent. I second what a user named Kent wrote: “Why buy $3 Chuck when he’s on the shelf?”
$10-$25: Malm Cellars Cabernet Sauvignon ($18.99) from Sonoma County
- Ester rating: 4.2
- Californians rating: 4.3
- Overall Vivino Rating: 4.0
- Esther descriptors: cedar, licorice, blackberry
- Vivino Descriptors: Cherry/Red Fruit, Smoke/Earth/Leather, Oak/Tobacco/Vanilla
Unlike the Corvelia Cab, which has almost no tannic bite, this Cabernet is a few steps bigger and bolder. It’s also less fruity, with notes of cedar and licorice (according to me) and vanilla and cola (according to Vivino). Our respective evaluations of this wine were very close, except for the acidity: I thought the wine was less acidic than the users. Many commenters noted that it’s a great value for a sub-$20 Sonoma wine, which is definitely true. Think of it as a nice pairing for a weeknight steak dinner—all those tannins will help balance out the fatty red meat. This wine is sold almost exclusively at Costco.
$25-$50: Merry Edwards Sauvignon Blanc ($44.99) from Russian River Valley
Happy Edwards Sauvignon Blanc
Rachel Bujalski/Special for The Chronicle 2021- Ester rating: 4.4
- Californians rating: 4.6
- Overall Vivino Rating: 4.5
- Esther Descriptors: Lemon Verbena, Grapefruit, Vanilla
- Vivino Descriptors: Peach/Pear/Melon, Citrus/Grapefruit/Lemon, Oak/Butter/Vanilla
The Merry Edwards Sauvignon Blanc is proof that sometimes a wine can taste very different from what it smells like. On the nose, this Sauvignon Blanc smells strong and herbal, which might lead you to expect a very tart, citrusy wine. But on the palate it is soft and rich. I often think of it as a Chardonnay lover’s Sauvignon Blanc; a user named Julia had the same thought. The folks at Vivino freak out: “I never thought Sauvignon Blanc could be so sophisticated,” wrote SJ. “Really great SB that lived up to the hype,” said Chris. Grapefruit and vanilla, flavors that both Vivino and I cite, should be crowd-pleasers.
$50-$100: Billecart-Salmon Brut Rosé ($88.76) from Champagne, France
- Ester rating: 4.8
- Californians rating: 4.6
- Overall Vivino Rating: 4.4
- Esther descriptors: unripe strawberry, lemon, green apple
- Vivino Descriptors: Strawberry/Raspberry/Red Berries, Citrus/Grapefruit/Lemon, Apple/Peach/Pear
I love this tart rosé sparkling wine and was heartened to see that the good folks at Vivino make it too. This is the only non-California wine that was included in our experiment, which I take as a testament to its deliciousness. Because this is made by the champagne method, its bubbles are smaller (presenting as a more fizzy wine) than what you’d find in other types of sparkling wine, such as Prosecco and pet-nat. For that reason, I rated it a 9 out of 10 on the smooth to fizzy scale, although Vivino users seemed to find it a bit smoother than I did. Our respective descriptions of the wine flavors were surprisingly similar, but if you want really detailed information, I suggest checking out Chris’s elaborate tasting note, which finds the wine to have a “dried almond sugar cookie” note, among others. foods.
$100-$200: Williams Selyem Westside Road Neighbors Pinot Noir ($120) from Russian River Valley
An older vintage of Williams Selyem Westside Road Neighbors Pinot Noir.
Peter DaSilva/Special to The Chronicle 2006- Ester rating: 4.3
- Californians rating: 4.8
- Overall Vivino Rating: 4.6
- Esther Descriptors: Maraschino Cherry, Oak Vanilla, Mushroom
- Vivino Descriptors: Cherry/Raspberry/Strawberry, Vanilla/Oak/Cola, Earthy/Smoke/Leather
Williams Selyem’s Pinot Noirs have enjoyed a devoted following since the 1980s, when two friends (one a typographer for the San Francisco Chronicle) began hobby-making wine in a garage on weekends. Now the brand has a sleek, state-of-the-art winery in Healdsburg, which is so exclusive it’s (mostly) not open to the public. If you try to buy this bottle on Wine.com, you’ll have to get a special membership (although I was able to buy it at Total Wine in San Mateo without pulling any strings). I can see why it earned an average rating of 4.8 from Vivino users: It displays the quintessential characteristics of Russian River Pinot: that bright, sweet cherry red with an earthy undertone (what I described as “mushroom”). However, this particular cuvee, a blend of Pinot Noir grapes from various vineyards along Westside Road in Healdsburg, is not my personal favorite of the Williams Selyem wines; I prefer some of their single-vineyard Pinot Noirs, like the one from Precious Mountain Vineyard. .
$200-$500: Caymus Special Selection Cabernet Sauvignon ($219.99) from Napa Valley
Various Caymus wines are on display in the Caymus-Suisun Winery retail space in Fairfield.
Jessica Christian/The Chronicle- Ester rating: 3.7
- Californians rating: 4.7
- Overall Vivino Rating: 4.7
- Esther Descriptors: Root Beer, Blackberry, Leather
- Vivino Descriptors: Vanilla/Oak/Chocolate, Blackberry/Prune/Black Fruit, Leather/Smoke/Cocoa
Few wines inspire such strong emotions as Caymus Special Selection, which has been named Wine Spectator Wine of the Year not once, but twice and is a mainstay on every high-end steakhouse wine list in the country. This is a wine with a huge fan base, and that fanaticism is clearly reflected in the highly flattering 4.7 rating from Vivino users. (I suspect the prodigious price tag might also predispose a reviewer to a higher rating.) It’s also the bottle that separated me the most from the Vivino base: to me, it tasted sweet, even syrupy, without much acidity or freshness to balance it out. That said, we find some of the same flavors in it, like leather and dark berries. The crowd also picked up some chocolate notes (both chocolate and cocoa), which I think are perfect. Many community reviews described the wine as “perfect”; Bruno said that it is the “best wine in the world for me, without a doubt”. I wish I shared Bruno’s enthusiasm.
Esther Mobley is the senior wine critic for The San Francisco Chronicle. Email: [email protected] Twitter: @Esther_mobiley
