EMNLP 2026 Main Conference · Accepted

Benchmarking and Boosting Multilingual Capabilities of LVLMs via OCR-Centric Reinforcement Learning

PM4Bench supports controlled evaluation across ten languages. The experiments identify OCR as a key source of cross-lingual performance gaps and motivate an OCR-centric reinforcement-learning method.

Junyuan Gao*, Jiahe Song*, Jiang Wu‡, Runchuan Zhu, Guanlin Shen, Shasha Wang, Xingjian Wei, Haote Yang, Weijia Li, Bin Wang, Lijun Wu, Conghui He†

Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University · National University of Singapore · Tsinghua SIGS

* Equal contribution · † Corresponding author · ‡ Project lead

Paper Code Benchmark QGO-8B
PM4Bench overview: limitations of existing multilingual benchmarks, the controlled ten-language PM4Bench design, and OCR-centric training findings.
Overview. PM4Bench controls content across languages, places text inside pixels to reflect agent deployment, and evaluates four complementary tasks. The experiments identify OCR as a key bottleneck and evaluate an OCR-centric intervention on PM4Bench and external benchmarks. Open the figure for the full-resolution view.

Paper overview

Abstract

Evaluating the multilingual capabilities of Large Vision-Language Models (LVLMs) remains challenging because most benchmarks rely on non-parallel corpora, making it unclear whether cross-lingual performance gaps reflect model limitations or dataset inconsistencies. We introduce PM4Bench, a multimodal, multilingual, multi-task benchmark built on a strictly parallel 10-language corpus, enabling fair, apples-to-apples cross-lingual comparison. We further introduce a vision setting that embeds textual inputs directly into images, approximating deployment scenarios where LVLM-driven agents observe unified visual environments. Experiments with 10 LVLMs identify OCR as a key factor behind cross-lingual disparity. Motivated by this finding, we design an OCR-centric GRPO strategy using fully synthesized, label-free OCR data and no costly task-specific VQA supervision. The resulting model improves general multilingual VQA, reduces disparity under vision, and transfers gains beyond PM4Bench.

Main contributions

Controlled evaluation, OCR diagnosis, and targeted training

01

Controlled evaluation

Semantics, layout, and answers are fixed across ten languages, making language the controlled variable.

02

OCR diagnosis

Interleaved input is compared with text rendered into pixels, followed by controlled OCR analyses.

03

OCR-centric GRPO

Synthetic OCR data and a dual reward improve recognition while preserving reasoning behavior.

PM⁴Bench

Strictly parallel evaluation across ten languages

Every language receives the same underlying content. PM4Bench preserves meaningful script-level differences while removing the corpus mismatch that normally confounds multilingual comparisons.

10 languages Strictly parallel Text in pixels Diagnostic metrics
A

traditional setting

Questions remain text and visual evidence remains image input—the standard interleaved LVLM formulation.

B

vision setting

All task information is rendered into one image, matching how multimodal agents perceive interfaces and environments.

MDUR

Understanding & reasoning

1,730 college-level multimodal reasoning cases per language.

MIQA

Multi-image QA

218 open-ended questions over related images per language.

MSOCR

Multi-scale OCR

100 controlled recognition probes with decreasing text size per language.

MGUI

GUI grounding

200 indirect coordinate-grounding instructions per language.

MDUR and MIQA use an LLM-and-native-speaker translation pipeline. MGUI is generated from 100 reusable interface templates, and MSOCR provides an automated, strictly parallel recognition probe.

Key finding

OCR is a key factor in the vision-setting gap

The two settings differ mainly in where text lives. That controlled difference is analyzed through ground-truth OCR input, per-language correlations, and an OCR-focused training intervention.

90% / 80%

Disparity increases

of evaluated models show higher cross-lingual variation on MDUR / MIQA under vision.

|PCC| > 0.5

Scores move together

for most models when correlating per-language MSOCR with downstream task performance.

Ground-truth text

The gap narrows

Supplying the recognized text alongside visual input raises average performance and reduces disparity.

Together, these results identify OCR as a practical target for improving multilingual LVLMs. Base-model capability remains the upper bound.

OCR-centric training

QGO: OCR-centric GRPO with a dual reward

Starting from Qwen3-VL-8B-Thinking, QGO trains on 20,000 fully synthesized OCR examples. Its reward combines recognition accuracy with response-length and format constraints: pure accuracy optimization sharpens OCR but can collapse chain-of-thought, so both sides are necessary for downstream reasoning tasks.

QGO-8B improves average score on every PM4Bench task and setting, lowers cross-lingual variation on all four vision tasks, and concentrates gains on weaker Arabic and Thai baselines. The intervention also transfers beyond the benchmark: +4.7 NED on CC-OCR overall, no broad OCRBench regression (791 vs. 789), and +0.5 average on MTVQA trained languages.

DataSynthetic, label-free OCR
ObjectiveAccuracy + length / format
OptimizationGRPO on Qwen3-VL-8B
SupervisionNo task-specific VQA / GUI labels

Links

Paper, code, data, and model

The revised manuscript is available as arXiv v3. Code, benchmark data, training data, and the complete QGO-8B checkpoint are public.

PaperarXiv v3 CodeGitHub BenchmarkHugging Face Training dataHugging Face ModelQGO-8B

Citation

Cite PM4Bench and QGO

If this work is useful in your research, please use the following BibTeX entry.

@misc{gao2026benchmarkingboostingmultilingualcapabilities,
      title={Benchmarking and Boosting Multilingual Capabilities of LVLMs via OCR-Centric Reinforcement Learning},
      author={Junyuan Gao and Jiahe Song and Jiang Wu and Runchuan Zhu and Guanlin Shen and Shasha Wang and Xingjian Wei and Haote Yang and Weijia Li and Bin Wang and Lijun Wu and Conghui He},
      year={2026},
      eprint={2503.18484},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2503.18484v3},
}