glm-ocr-receipt-ocr-mlx-4bit

MLX conversion of a LoRA fine-tune of GLM-OCR (0.9B) for structured receipt extraction.

Scope

Trained on CORD-v2 (Indonesian receipts) — the same recipe and data as the LFM2-VL-450M fine-tune, for direct comparison. It is not trained on Japanese receipts and does not use a Japanese-receipt schema.

Observed on a Japanese receipt: it reads real strings off the page — store name, address, invoice number, item names such as レッドブルエナジー25 — but maps them into CORD's fields imperfectly, and leaves ¥ and the 軽 tax marker inside price strings. The base model's reading ability survives the fine-tune; the schema is the mismatch.

Prompt

The CORD instruction it was trained against: JSON with menu (list of {nm, cnt, price}), sub_total, and total.

Quirk: <think></think> prefix

Output begins with an empty <think></think> block before the JSON. Anything parsing this model's output must strip a leading reasoning block, or every correct response looks like a parse failure.

Conversion

mlx_vlm.convert -q --q-bits 4 with mlx-vlm 0.6.8, run against a local copy whose config.json had torch_dtype removed from text_config/vision_config — mlx-vlm's TextConfig rejects that key and the LoRA merge writes it. Result: 9.0 effective bits/weight, 1254 MB; --q-bits 4 does not quantize everything.

Measured

Local mlx_vlm.generate on Apple silicon: 2.04 GB peak, ~220 tokens/s.