OCR: optical character recognition

Introduction

OCR (optical character recognition) turns a picture of text into text a computer can work with: characters you can select, copy, search and edit. The picture is usually a scan of a paper document, a photo of a page or a sign, a screenshot, or a page of a PDF that was made from an image and has no text layer at all.

Without OCR such a file is only pixels. A PDF made from scans looks right on screen, but a search finds nothing in it and no word can be copied out. After OCR the same document can be searched by keyword, indexed by a search engine, read by a screen reader, and fed to a script that extracts dates, sums or names.

The task is not trivial because the same letter looks different in every font and at every size, and worse in a photo: the page is tilted, the light is uneven, the scan is noisy or the text runs over a coloured background. OCR software earns its keep by coping with exactly that.

How it works

A modern OCR engine runs a pipeline, and knowing the steps helps when the result comes out wrong:

1. Preprocessing. The image is converted to grayscale, binarised (each pixel becomes black or white), deskewed and cleaned of noise. A bad scan is best fixed here rather than after recognition.
2. Layout analysis. The engine finds columns, blocks, lines and words, so that text is read in the right order and a table does not turn into a stream of mixed cells.
3. Recognition. Each character image is compared with the trained model of the language: the engine picks the most likely letter in this context, not just the closest shape.
4. Output. The result is plain text, or a text layer placed under the original image, which is what makes a searchable PDF.

The result is a guess, so a confidence value comes with it, and a page of dense text always needs a check. Numbers and similar-looking letters (0 and O, 1 and l) are the usual suspects.

Tools on this site

The engine covered here is Tesseract — free software under the Apache License, originally written at Hewlett-Packard and developed with Google support since 2006. It reads more than a hundred languages, installs on Windows and Linux , and is the engine inside many other programs and libraries.

A command-line run needs only two arguments, the image and the name of the output file without its extension:

tesseract scan.png out

The text lands in out.txt next to the image. On Windows the installer does not add the program folder to the search path by itself, so the command is not found until the folder is added to the PATH system variable or called with its full path.

What decides quality

Resolution comes first: around 300 dpi for a scanned page is the figure the Tesseract documentation itself recommends, and a phone photo needs to be sharp and evenly lit to get there. The language model comes second: the engine must be told which language it is reading, and a page in Russian recognised with the English model comes out as nonsense.

Page segmentation is the third setting that changes the output: a single column of text, a table, one line on a sign or a single word are different jobs, and the default assumes a page. Everything else, from preprocessing to a spell check on the result, is tuning around those three.

Article author: Arthur Isaev