Skip to content

text.md

Latest commit

 

History

History
64 lines (43 loc) · 2.64 KB

text.md

File metadata and controls

64 lines (43 loc) · 2.64 KB
Raw

Scanned Receipt OCR by Convolutional-Recurrent Neural Network

This is a pytorch implementation of CRNN, which is based on @meijieru's repository here.

Introduction

The CNN + RNN + CTC model visualisation:

We applied a modified CRNN in this task. CRNN is a conventional scene text recognition method including convolutional layers, bidirectional LSTM layers, and a transcription layer in sequence.

In scanned receipts each text usually contains several words. We add the blank space between words to the alphabet for LSTM prediction and thus improve the network from single word recognition to multiple words recognition. Moreover, we double the input image width to tackle the overlap problem of long texts after max-pooling and stack one more LSTM, enhancing the accuracy per character in the training set from 62% to 83%.

Dependency

  1. warp-ctc-pytorch
  2. lmdb

Prediction

As is shown in the introduction image, CRNN only accepts image that has single line of word. Therefore, we must provide bounding boxes for the whole reciept image.

  1. Put image under folder ./data_test/ and bounding box text file under ./boundingbox/, the name of image file and text file must correspond. The example test file can be found here

  2. Download pre-trained model from here and put the weight file under ./expr/ folder

  3. To predict, just run python main.py. You can change the code inside to visualise output or prepare result for task 3.

example result:

tan chay yee
81750 masai johor
sales persor : fatin
tax invoice
total inclusive gst:
invoice no : pegiv1030765
email: ng@ojcgroup.com
bill to"
date
description
address
total:
cashier
:the peak quarry works

Training

Training a CRNN requires converting data into lmdb format.

  1. Divide training data into training and validating dataset, put each portion into ./data_tain/ and ./data_valid. An example could be found here

  2. Run create_dataset.py to create lmdb dataset. The created dataset could be found inside ./dataset

  3. After preparing dataset, just run:

    python train.py --adadelta --trainRoot {train_path} --valRoot {val_path} --cuda

    with desired options

  4. Trained model output will be in ./expr/