xiry.

Development · Tip

Receipt OCR with Mistral OCR and Vercel AI SDK

Implementation pattern for structured receipt extraction using Mistral and Vercel AI SDK

From the Xiry library · Updated 11 October 2026

A practical code recipe showing how to parse receipt images into typed, structured JSON using Mistral's vision models and the Vercel AI SDK. It demonstrates how to combine schema-based generation (`generateObject`) with custom retry logic and timeout handling. It is pulled directly from Midday's open-source document processing pipeline.

How it works

The best OCR for extracting structured receipt data: Mistral OCR + Vercel AI SDK — insanely cheap.

Code (packages/documents/src/processors/receipt-processor.ts):

- Imports: mistral from '@ai-sdk/mistral', generateObject from 'ai', receiptPrompt, receiptSchema, GetDocumentRequest type, retryCall util.

- ReceiptProcessor class with #processDocument({ documentUrl }: GetDocumentRequest): throws if no documentUrl; calls retryCall(() => generateObject({ model: mistral('mistral-medium-latest'), schema: receiptSchema, abortSignal: AbortSignal.timeout(20000) (20s), messages: [{ role: 'system', content: receiptPrompt }, { role: 'user', content: [{ type: 'image', image: documentUrl }] }], providerOptions: { mistral: { documentImageLimit: 4 } } })); returns result.object.

- getReceipt(params: GetDocumentRequest): returns { ...result, type: 'expense', date: result.date, amount: result.total_amount, currency: result.currency, name: result.store_name, tax_amount: result.tax_amount, tax_rate: result.tax_rate, tax_type: result.tax_type, language: result.language, metadata: { register_number: result.register_number ?? null, cashier_name: result.cashier_name ?? null, email: result.email ?? null } }.

Follow-up in thread: 'code here https://github.com/midday-ai/midday' (Invoicing, Time tracking, File reconciliation, Storage, Financial Overview & Assistant for Freelancers).

What it's for

  • Extract structured transaction data from receipt images into a database
  • Implement reliable LLM API calls with retry wrappers and timeout abort signals
  • Build automated expense tracking and invoice processing workflows

Key points

  • Uses Vercel AI SDK's generateObject for strictly typed JSON output
  • Integrates Mistral vision models via @ai-sdk/mistral
  • Includes retry logic and timeout abort signals for resilient API calls
  • Normalizes receipt fields including tax rate, total amount, currency, and store metadata

Pricing

Open source

Details

Type
Tip
Category
Development
Topic
Document Processing
Open source
Yes
Platforms
Code library
For
TypeScript and AI application developers building financial or expense-tracking tools
Alternative to
Tesseract OCR, AWS Textract, Google Cloud Document AI
Tags
ocr, mistral, vercel ai sdk, typescript, receipts, ai

Why it was saved

  • @pontusab, 10 August 2025: Declares Mistral OCR the best and 'insanely cheap' option for structured receipt extraction, and the code is copy-pasteable: it parses receipt images into typed fields (date, amount, currency, store name, tax, language, metadata).

Related

Updated