---
language:
  - en
  - fr
  - zh
  - de
  - ja
  - ko
  - ar
  - hi
  - pt
  - es
license: apache-2.0
library_name: executorch
tags:
  - ocr
  - executorch
  - on-device
  - android
  - xnnpack
  - int8
  - vision-language-model
  - document-understanding
base_model: lightonai/LightOnOCR-2-1B
pipeline_tag: image-to-text
---

# LightOnOCR-2-1B ExecuTorch (Android-ready)

On-device OCR model converted from [lightonai/LightOnOCR-2-1B](https://huggingface.co/lightonai/LightOnOCR-2-1B) to ExecuTorch `.pte` format for mobile/edge deployment.

**1B parameter end-to-end OCR** — converts document images (PDFs, receipts, scans) to clean text, running entirely on-device with no cloud dependency.

## Models

| File | Size | Description | Recommended |
|------|------|-------------|-------------|
| `vision_encoder_int8.pte` | 398 MB | Vision encoder, INT8 weight-only, XNNPACK | ⭐ Yes |
| `text_decoder_int8.pte` | 1.2 GB | Text decoder, INT8 weight-only, XNNPACK | ⭐ Yes |
| `vision_encoder.pte` | 1.6 GB | Vision encoder, FP32, XNNPACK | |
| `text_decoder_4096.pte` | 2.9 GB | Text decoder, FP32, XNNPACK | |

**For on-device deployment, use the INT8 variants (1.6 GB total).**

## Quick Start — Python (ExecuTorch Runtime)

```python
from executorch.runtime import Runtime
from huggingface_hub import hf_hub_download
import torch

# Download INT8 models
vis_path = hf_hub_download("acul3/LightOnOCR-2-1B-ExecuTorch", "vision_encoder_int8.pte")
dec_path = hf_hub_download("acul3/LightOnOCR-2-1B-ExecuTorch", "text_decoder_int8.pte")

# Load ExecuTorch runtime
runtime = Runtime.get()
vision = runtime.load_program(vis_path).load_method("forward")
decoder = runtime.load_program(dec_path).load_method("forward")

# Preprocess image to fixed 1120×1540
from PIL import Image
img = Image.open("document.jpg").convert("RGB").resize((1540, 1120))
# Normalize (use HF processor or manual normalization)
pixel_values = torch.tensor(...)  # [1, 3, 1120, 1540] float32

# Run vision encoder
image_features = vision.execute([pixel_values])  # [1, 2200, 1024]

# Build input tokens + autoregressive decode (see scripts/test_e2e_v2.py for full example)
```

## Quick Start — Android (Kotlin)

```kotlin
import org.pytorch.executorch.Module
import org.pytorch.executorch.EValue
import org.pytorch.executorch.Tensor

// Load models from assets or downloaded files
val visionModule = Module.load("vision_encoder_int8.pte")
val decoderModule = Module.load("text_decoder_int8.pte")

// Preprocess: resize image to 1120×1540, normalize
val pixelTensor = preprocessImage(bitmap)  // [1, 3, 1120, 1540]

// Run vision encoder
val imageFeatures = visionModule.forward(EValue.from(pixelTensor))

// Run text decoder autoregressively
// See ExecuTorch LLM Demo App for the decode loop pattern
```

## Full E2E Pipeline

For a complete working example with autoregressive decoding, see:
- **[scripts/test_e2e_v2.py](scripts/test_e2e_v2.py)** — FP32 E2E validation
- **[scripts/test_e2e_int8.py](scripts/test_e2e_int8.py)** — INT8 E2E validation

The pipeline works as follows:

1. **Resize** input image to exactly **1120×1540** pixels
2. **Run vision encoder** → image embeddings `[1, 2200, 1024]`
3. **Build input tokens** using the chat template with 2200 `[IMG]` placeholders (token id `151655`)
4. **Scatter** vision embeddings into the `[IMG]` positions in the token embedding space
5. **Prefill** the text decoder with the full combined sequence (2260 tokens)
6. **Decode** autoregressively until EOS token (`151645`) or max length

### Token Template

```
<|im_start|>user\n[IMG][IMG]...(2200 total)...[VPAD]...(39 row separators)...[IMG_END]
OCR this document. Extract all text.<|im_end|>\n<|im_start|>assistant\n
```

### Special Token IDs

| Token | ID | Purpose |
|-------|-----|---------|
| `[IMG]` | 151655 | Image patch placeholder (2200 per image) |
| `[VPAD]` | 151654 | Vision row separator (39 per image) |
| `[IMG_END]` | 151653 | End of vision tokens |
| `<\|im_start\|>` | 151644 | Chat turn start |
| `<\|im_end\|>` | 151645 | Chat turn end / EOS |

## Architecture

```
LightOnOCR-2-1B (~1B params)
├── Vision Encoder: Pixtral ViT (400M params)
│   ├── 24 transformer layers, hidden=1024, heads=16
│   ├── Patch size: 14×14, 2D RoPE
│   └── PatchMerger: 2×2 spatial merge → 4× token reduction
├── MultiModal Projector (6M params)
│   ├── RMSNorm → PatchMerger → Linear(1024→1024) → GELU → Linear
│   └── Output: [1, 2200, 1024] for 1120×1540 input
└── Text Decoder: Qwen3 (700M params)
    ├── 28 transformer layers, GQA (16 heads, 8 KV heads)
    ├── head_dim=128, intermediate=3072, hidden=1024
    ├── QK-norm (RMSNorm on Q,K per layer)
    ├── Static KV cache, max_seq_len=4096
    └── Vocab: 151936 (Qwen2 tokenizer)
```

## Validation Results

E2E validated against the original HuggingFace model on receipt and synthetic document images:

| Variant | Receipt (360 tok) | Synthetic (76 tok) |
|---------|-------------------|---------------------|
| **FP32 modules** | ✅ Exact match (edit dist = 0) | ✅ Exact match (edit dist = 0) |
| **INT8 quantized** | ✅ Exact match (edit dist = 0) | ✅ Exact match (edit dist = 0) |

**Zero quality degradation from INT8 weight-only quantization.**

## Export Details

| Property | Value |
|----------|-------|
| ExecuTorch version | 1.1.0 |
| Backend | XNNPACK (CPU, cross-platform) |
| Quantization | torchao INT8 weight-only (per-channel, no calibration) |
| Fixed input resolution | 1120 × 1540 |
| Max sequence length | 4096 tokens |
| Source model | [lightonai/LightOnOCR-2-1B](https://huggingface.co/lightonai/LightOnOCR-2-1B) |

### Model Surgery Applied

The original HF model required several modifications for ExecuTorch compatibility:

- **PatchMerger**: Rewritten to eliminate Python loops and dynamic shapes (fixed single-image)
- **KV Cache**: HF `DynamicCache` → static tensors as explicit model inputs/outputs
- **QK-Norm**: `@use_kernel_forward_from_hub` decorator bypassed, ops inlined
- **Attention**: GQA manually implemented without SDPA dispatch (export-clean)
- **Token substitution**: Vision features scattered at `[IMG]` positions via index operations

## Scripts

| Script | Description |
|--------|-------------|
| [`scripts/export_vision.py`](scripts/export_vision.py) | Vision encoder surgery + export |
| [`scripts/export_decoder.py`](scripts/export_decoder.py) | Text decoder surgery + export |
| [`scripts/quantize_wo.py`](scripts/quantize_wo.py) | INT8 weight-only quantization |
| [`scripts/test_e2e_v2.py`](scripts/test_e2e_v2.py) | Full E2E validation (FP32) |
| [`scripts/test_e2e_int8.py`](scripts/test_e2e_int8.py) | Full E2E validation (INT8) |

## Hardware Requirements

### For on-device inference (INT8)
- **RAM**: 6 GB minimum (8 GB+ recommended)
- **Storage**: 1.6 GB for both model files
- **CPU**: ARM64 (Android) or x86_64

### Tested target devices
- **XNNPACK (current)**: Any Android/iOS device with CPU
- **QNN (planned)**: Snapdragon 8 Gen 2+ with Hexagon NPU

### For export/quantization (development)
- Jetson AGX Orin 64GB or similar (64GB+ unified memory recommended)
- Python 3.10+, PyTorch 2.10+, ExecuTorch 1.1.0

## Reproduce

```bash
# Clone and setup
git clone https://huggingface.co/acul3/LightOnOCR-2-1B-ExecuTorch
cd LightOnOCR-2-1B-ExecuTorch

# Install deps
pip install executorch transformers torchao pillow

# Download source model
huggingface-cli download lightonai/LightOnOCR-2-1B --local-dir models/LightOnOCR-2-1B

# Export (generates .pte files)
python scripts/export_vision.py
python scripts/export_decoder.py

# Quantize
python scripts/quantize_wo.py

# Validate
python scripts/test_e2e_int8.py
```

## License

Apache 2.0 (same as [source model](https://huggingface.co/lightonai/LightOnOCR-2-1B))

## Citation

```bibtex
@misc{lightonocr2_executorch_2026,
  title = {LightOnOCR-2-1B-ExecuTorch: On-Device OCR with ExecuTorch},
  author = {Samsul Rahmadani},
  year = {2026},
  url = {https://huggingface.co/acul3/LightOnOCR-2-1B-ExecuTorch},
  note = {Converted from lightonai/LightOnOCR-2-1B}
}
```

## Acknowledgments

- [LightOn AI](https://lighton.ai/) for the original LightOnOCR-2-1B model
- [PyTorch ExecuTorch](https://github.com/pytorch/executorch) team for the on-device runtime
- [torchao](https://github.com/pytorch/ao) for INT8 weight-only quantization