Discover Wine Everywhere You Go

How does an app identify a wine from a photograph?

A wine app identifies a bottle by reading the label as an image, not by recognising the wine. Four separate steps run between the shutter and the recommendation, and each one fails differently.

1. Step 1 — find the bottle in the frame

   The first pass locates label-shaped regions in the photograph and straightens them. A shelf photograph typically contains eight to twenty candidate labels, most of them at an angle and partly hidden by the bottle in front. Anything the detector misses at this stage cannot be recovered later, which is why a straight-on photograph of a single bottle is far more reliable than a wide shot of a shelf.

2. Step 2 — read the text

   Optical character recognition converts each region to text. This is where the classic failures live: a serif capital on a dark background, gold foil that blows out under a shop light, a script face on a Burgundy label, and a vintage printed on the neck rather than the body. The output is a string, and a wrong character in a producer name is enough to send the next step to the wrong wine.

3. Step 3 — match the text to a wine

   The extracted text is matched against a wine database. A match needs a producer, a wine name and ideally a vintage; with only two of the three the result is a family of wines rather than a bottle. This is the step that decides whether you get "Château Something 2019" or "a Bordeaux red".

4. Step 4 — turn a name into a recommendation

   Naming the bottle and recommending one are different problems, and only the first is a vision task. A recommendation ranks the identified bottles against what you said you like — style, sweetness, budget, the occasion — which means the quality of the answer depends far more on how honestly you filled in those preferences than on how good the camera is. An app that recommends without ever asking what you like is ranking by popularity and calling it personalisation.

Where wine recognition reliably fails

It fails on damaged, wet or peeling labels, because the text is not there to read. It fails on very small producers whose wines are in no database. It fails on wines whose label carries a region and no producer name, which is common in Italy and Spain. It fails in low light, because OCR needs contrast more than it needs resolution.

It also fails quietly, which is the dangerous mode: a confident answer naming the wrong vintage of the right wine. Vintage is the single most misread field, because it is the smallest text on most labels and often sits on a separate strip.

A subtler failure is the one nobody warns about: recognition can be perfect and the recommendation still wrong, because the bottle in front of you is not in the reference data at the vintage you are holding. A 2022 of a wine the database knows only through 2019 will be described with the older wine's character, and nothing in the interface signals that substitution happened.

The practical rule: use recognition to find out what a bottle is when you have never seen it before, and check the label yourself before anything expensive depends on the answer. No image model should be the last word on an auction bid or an insurance valuation.