A Flutter package for scanning and extracting structured data from supermarket receipts using Google's ML Kit. Ideal for building expense tracking apps, loyalty programs, or any system needing receipt parsing.
- 🧾 Detect and extract text from printed receipts
- 🛒 Optimized for typical supermarket layouts
- 🔍 Identifies line items, totals, and store names
- ⚡ Fast and efficient ML Kit text recognition
- 📱 Works on Android and iOS
- 🔧 Easy API with callback support
Add to your pubspec.yaml:
dependencies:
receipt_recognition: ^<latest_version>Then run:
flutter pub getUpdate AndroidManifest.xml:
<uses-permission android:name="android.permission.CAMERA"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>Update Info.plist:
<key>NSCameraUsageDescription</key>
<string>Camera access is needed to scan receipts.</string>
<key>NSPhotoLibraryUsageDescription</key>
<string>Photo library access is needed to select receipt images.</string>import 'package:flutter/material.dart';
import 'package:receipt_recognition/receipt_recognition.dart';
import 'package:google_mlkit_text_recognition/google_mlkit_text_recognition.dart';
// Create a receipt recognizer
final receiptRecognizer = ReceiptRecognizer(
options: {
// The 'stores' map lets you normalize store/company names detected by OCR.
// Keys: possible OCR outputs or store name variants (case-insensitive).
// Values: the canonical store name you want in the final RecognizedReceipt.
'stores': {
// Normalize multiple variations of Aldi:
'aldi nord': 'Aldi Nord',
'aldi süd': 'Aldi Süd',
'aldi': 'Aldi',
// Normalize REWE variants:
'r e w e': 'Rewe',
'rewe': 'Rewe',
// Normalize Lidl spelling:
'lidl': 'Lidl',
}
},
singleScan: true,
onScanComplete: (receipt) {
// Handle the recognized receipt
print('Company: ${receipt.company?.value}');
print('Total: ${receipt.sum?.formattedValue}');
for (final position in receipt.positions) {
print('${position.product.formattedValue}: ${position.price.formattedValue}');
}
},
onScanUpdate: (progress) {
// Track scanning progress
print('Scan progress: ${progress.estimatedPercentage}%');
print('Added positions: ${progress.addedPositions.length}');
},
);
// Process an image
Future<void> processReceiptImage(InputImage inputImage) async {
final receipt = await receiptRecognizer.processImage(inputImage);
if (receipt != null) {
// Receipt was successfully recognized
}
}
// Don't forget to close the receipt recognizer when done
@override
void dispose() {
receiptRecognizer.close();
super.dispose();
}For an advanced use case, we provide an example of using this package with a video feed. You can integrate it with a camera feed (via a package like camera), and continuously scan receipts in real time.
Refer to the example app for an implementation that uses live camera data to recognize and process receipts as they appear in the frame.
The receipt_recognition package follows a modular architecture designed to handle the complexities of receipt scanning and data extraction:
┌────────────────┐ ┌─────────────────┐ ┌────────────────┐
│ │ │ │ │ │
│ Image Capture │────▶│ Text Recognition│────▶│ Receipt Parser │
│ │ │ │ │ │
└────────────────┘ └─────────────────┘ └────────┬───────┘
│
▼
┌────────────────┐ ┌─────────────────┐ ┌────────────────┐
│ │ │ │ │ │
│ Data Consumer │◀────│ Data Optimizer │◀────│ Data Extractor │
│ │ │ │ │ │
└────────────────┘ └─────────────────┘ └────────────────┘
The main entry point for the package. It orchestrates the entire recognition process from image input to structured data output.
Leverages Google's ML Kit to perform OCR (Optical Character Recognition) on receipt images, converting the visual text into digital text.
Analyzes the raw text to identify and categorize receipt elements:
- Store/company name (e.g., Aldi, Rewe, Edeka, Penny, Lidl, Kaufland, Netto in German markets)
- Total sum ("Summe", "Gesamt", "Total")
- Line items (products and prices)
- Date and time information
A crucial part that improves recognition accuracy through several mechanisms:
┌───────────────────┐
│ ReceiptOptimizer │
└─────────┬─────────┘
│
┌───────────────┼───────────────┐
│ │ │
┌───────────▼──────┐ ┌──────▼─────┐ ┌───────▼─────┐
│ Group Management │ │Confidence │ │Stability │
│ │ │Calculation │ │Thresholds │
└──────────────────┘ └────────────┘ └─────────────┘
The optimizer:
- Groups similar receipt items together
- Applies confidence thresholds to filter out uncertain recognitions
- Uses stability measures to determine reliable data points
- Merges multiple scans for improved accuracy
- Image Acquisition: Capture receipt image from camera or gallery
- Text Detection: ML Kit processes the image to extract raw text
- Structured Parsing: A raw text is analyzed to identify receipt elements
- Optimization: Multiple scans are compared and merged for accuracy
- Data Delivery: Structured receipt data is provided via callbacks
+-------------------------+----------------+--------------------------------+
| Feature | Status | Notes |
+-------------------------+----------------+--------------------------------+
| Basic OCR | ✅ Complete | Using Google ML Kit |
| Company/Store Detection | ✅ Complete | With optimization |
| Total Sum Detection | ✅ Complete | With validation |
| Line Item Recognition | ✅ Complete | Products with prices |
| Receipt Merging | ✅ Complete | For improved accuracy |
| Product Normalization | ✅ Complete | Standardizes product names |
+-------------------------+----------------+--------------------------------+
Currently, the package has optimized recognition for:
- English receipts: Full support for standard formats
- German receipts: Full support with specialized detection patterns for:
- German market chains (Aldi, Rewe, Edeka, etc.)
- German sum labels ("Summe", "Gesamt", "Zu zahlen")
- German number formats (comma as decimal separator)
The package supports two primary scanning approaches:
Ideal for scanning from gallery images or single camera captures:
User selects image → OCR → Structure extraction → Data callback
Better for real-time scanning with a live preview:
┌────────────┐ ┌───────────┐ ┌────────────┐ ┌────────────┐
│ Camera │────▶│ Frame │────▶│ Recognition│────▶│ Confidence │
│ Stream │ │ Capture │ │ Process │ │ Check │
└────────────┘ └───────────┘ └────────────┘ └──────┬─────┘
│
┌────────────┐ ┌───────────┐ ┌────────────┐ ┌──────▼─────┐
│ Final │◀────│ User │◀────│ Preview │◀────│ Feedback │
│ Result │ │ Confirm │ │ Display │ │ Loop │
└────────────┘ └───────────┘ └────────────┘ └────────────┘
- Processing Time: Typically 0.5–2 seconds per frame depending on a device
- Memory Usage: Peak usage of ~50–100MB during recognition
- Battery Impact: Moderate when using continuous scanning
- Accuracy: ~85–95% depending on receipt quality and lighting conditions
- Lighting: Ensure good, even lighting for the best OCR results
- Alignment: Keep receipts as flat and aligned as possible
- Stability: For continuous scanning, allow 1–2 seconds of stable framing
- Multiple Scans: Use the optimizer's merging capabilities for improved accuracy
- Language Handling: For mixed-language environments, consider setting the appropriate TextRecognitionScript when initializing the recognizer
The package includes a robust validation system that verifies receipt completeness based on the match between the calculated sum (from line items) and the detected total sum. Four validation states are possible:
+-------------------------+------------------------+-------------------------+
| Validation State | Description | Match Percentage |
+-------------------------+------------------------+-------------------------+
| ReceiptCompleteness. | Perfect match between | 100% |
| complete | line items and total | |
+-------------------------+------------------------+-------------------------+
| ReceiptCompleteness. | Very close match, | 95-99% |
| nearlyComplete | acceptable for most | (configurable) |
| | applications | |
+-------------------------+------------------------+-------------------------+
| ReceiptCompleteness. | Partial recognition | <95% |
| incomplete | with significant | |
| | discrepancies | |
+-------------------------+------------------------+-------------------------+
| ReceiptCompleteness. | Missing critical data | 0% |
| invalid | (e.g., total sum) | |
+-------------------------+------------------------+-------------------------+
You can track the validation state through the onScanUpdate callback:
final receiptRecognizer = ReceiptRecognizer(
onScanUpdate: (progress) {
// Check validation status
switch (progress.validationResult.status) {
case ReceiptCompleteness.nearlyComplete:
print('Receipt is ${progress.validationResult.matchPercentage}% complete');
// Consider using acceptReceipt here if percentage is acceptable
break;
case ReceiptCompleteness.incomplete:
print('Still scanning...');
break;
// Handle other cases
}
},
);When automatic validation doesn't reach 100% match but the receipt seems adequate, you can manually accept it using the acceptReceipt method:
// Example: Accepting a nearly complete receipt when user taps "Accept"
void acceptCurrentReceipt() {
if (progress.mergedReceipt != null &&
progress.validationResult.matchPercentage! >= 95) {
final acceptedReceipt = receiptRecognizer.acceptReceipt(progress.mergedReceipt!);
// Handle the accepted receipt
}
} ┌───────────────┐
│ Scan │
│ Receipt │
└───────┬───────┘
│
▼
┌─────────────────┐ ┌───────────────┐ ┌─────────────────┐
│ │ │ Validation │ │ │
│ Invalid (0%) │◀─────┤ Process ├─────▶│ Complete (100%)│
│ │ │ │ │ │
└─────────────────┘ └───────┬───────┘ └────────┬────────┘
│ │
│ │
▼ ▼
┌─────────────────────────┐ ┌──────────────────┐
│ │ │ │
│ Incomplete (<95%) │ │ Auto-accepted │
│ │ │ │
└──────────┬──────────────┘ └──────────────────┘
│
│
▼
┌─────────────────────────┐
│ │
│ Nearly Complete (≥95%) │
│ │
└──────────┬──────────────┘
│
│
▼
┌─────────────────────────┐
│ │
│ Manual Acceptance │
│ acceptReceipt() │
│ │
└─────────────────────────┘
This workflow enables you to build UIs that show the user scanning progress and offer manual acceptance for receipts that don't achieve perfect validation but are still usable.
See the CHANGELOG.md for a complete list of updates and version history.
- Product name normalization
- Long receipt support and merging mechanism
- Multi-language receipt support (English and German)
Contributions, suggestions, and bug reports are welcome! Feel free to open an issue or PR.
This package is released under the MIT License.