MiniCPM-V 4.5: Efficient 8B Multimodal Model

Updated 4 July 2026

The paper presents MiniCPM-V 4.5 as an 8B multimodal model integrating unified 3D-resampling, parser-free OCR, and hybrid RL for efficient processing.
It reduces visual tokens by up to 16x compared to dense encoders, enabling compact representation of high-resolution images and videos.
The unified learning paradigm enhances document reasoning and OCR accuracy while supporting controllable reasoning with both short and long modes.

MiniCPM-V 4.5 is an 8B-parameter multimodal LLM in the MiniCPM family, designed for high efficiency and strong performance on images, documents, and videos. Its defining features are a unified 3D-Resampler for compact visual encoding, a unified learning paradigm for document knowledge and text recognition without heavy data engineering, and a hybrid reinforcement learning scheme that supports both short, efficient answers and long, step-by-step reasoning. In the reported evaluations, it reaches an OpenCompass average of 77.0, surpassing GPT-4o-latest at 75.4 and Qwen2.5-VL 72B at 76.1, while on VideoMME it achieves state-of-the-art performance among models under 30B size using 46.7% GPU memory cost and 8.7% inference time of Qwen2.5-VL 7B (Yu et al., 16 Sep 2025). Within the broader MiniCPM line, it extends an earlier emphasis on efficient end-side multimodal deployment into a model class centered on compact visual tokenization, document reasoning, and video understanding (Yao et al., 2024).

1. Historical position and design objectives

The MiniCPM-V line was introduced as a series of efficient multimodal LLMs deployable on end-side devices, with earlier variants emphasizing a SigLIP SoViT-400m/14 visual encoder, adaptive high-resolution image processing up to 1.8M pixels at any aspect ratio, and a lightweight compression layer that reduced visual token count before the LLM (Yao et al., 2024). That earlier series framed MiniCPM-V as an attempt to obtain GPT-4V-level performance in much smaller models and under device-side deployment constraints, including mobile phones.

MiniCPM-V 4.5 retains the MiniCPM emphasis on efficiency, but its objectives are broader and more explicitly structured around three bottlenecks. The first is the cost of visual token sequences, especially for high-resolution images and videos. The second is the brittleness of document pipelines that rely on PDF parsers and heavy data engineering. The third is the tendency of reinforcement-learning-based reasoning models to produce unnecessarily long chains of thought even for simple problems. The model therefore introduces three coordinated changes: a unified 3D-Resampler for images and videos, a parser-free document/OCR learning paradigm based on dynamic corruption of text regions, and a hybrid RL procedure that jointly optimizes a short mode and a long reasoning mode (Yu et al., 16 Sep 2025).

A common misconception is that MiniCPM-V 4.5 is primarily an OCR model or a narrow video model. The reported design instead positions it as a general multimodal model whose OCR, document, and video gains emerge from a single architecture and training recipe. The paper evaluates it on OpenCompass, OCRBench, DocVQA, OmniDocBench, VideoMME, LVBench, MLVU, LongVideoBench, MotionBench, and hallucination benchmarks, indicating a scope wider than document parsing alone (Yu et al., 16 Sep 2025).

2. Core architecture and visual tokenization

MiniCPM-V 4.5 consists of three modules: a lightweight visual encoder, a unified 3D-Resampler, and an 8B MiniCPM-style decoder-only LLM that consumes visual tokens and text tokens (Yu et al., 16 Sep 2025). High-resolution images are first partitioned following LLaVA-UHD, so that each slice remains close to the encoder’s pretraining resolution. The crucial architectural novelty is not the encoder alone but the resampling stage that compresses the encoder’s dense output into a compact multimodal sequence.

For images, the model uses a set of learnable query tokens in a cross-attention resampling block. Conceptually, if the encoder yields patch features $\{x_{h,w}\in\mathbb{R}^d\}$ and the resampler has learnable queries $q_i\in\mathbb{R}^d$ , each query is augmented with a 2D positional embedding and attends to the patch grid:

$z_i = \text{CrossAttn}\big(q_i + p^{(2D)}_i,\ \{x_{h,w} + p^{(2D)}_{h,w}\}\big).$

Because the number of queries is small, token count is aggressively reduced. The paper states that typical MLLMs such as Qwen2.5-VL and InternVL3 use 256 tokens for a 448×448 image, whereas MiniCPM-V 4.5 uses 64 tokens for that image, a 4× reduction relative to those baselines and up to 16× relative to dense per-patch encoders (Yu et al., 16 Sep 2025).

For videos, the same module is generalized from 2D to 3D. Frames are divided into temporal packages; each package is encoded frame-wise and then jointly resampled with spatial and temporal positional embeddings:

$z^{(k)}_i = \text{CrossAttn}\big(q_i + p^{(2D)}_i + p^{(T)}_k,\ \{x_{t,h,w} + p^{(2D)}_{h,w} + p^{(T)}_t\}_{t\in\mathcal{T}_k}\big).$

This formulation explicitly exploits temporal redundancy rather than treating video as a flat concatenation of framewise image tokens. The reported example is that 6 frames at 448×448 can be jointly compressed into 64 video tokens, and that a 6-second video at 2 fps is represented by 128 tokens instead of 1,536 tokens in Qwen2.5-VL or 3,072 tokens in InternVL3 (Yu et al., 16 Sep 2025).

The unification is architecturally significant. The same resampler and weights are used for single images, multi-image inputs, and videos. The paper presents this as enabling efficient adaptation from an existing 2D-Resampler via a short SFT stage, while also allowing knowledge transfer across modalities; it explicitly notes that the model shows “reasonable video OCR capability” despite not specifically collecting video-OCR data (Yu et al., 16 Sep 2025).

Component	Function	Reported effect
Visual encoder	Encodes image slices or video frames	Supplies patch-level features
3D-Resampler	Cross-attention compression over space or space-time	64 tokens for 448×448 image; additional temporal compression for video
8B LLM decoder	Autoregressive multimodal reasoning and generation	Consumes compact visual tokens plus text tokens

An important interpretive point is that MiniCPM-V 4.5’s efficiency is not presented as a purely systems-level optimization. The paper attributes it primarily to representation design: fewer visual tokens enter the decoder, so attention cost, KV-cache size, and downstream RL-time response cost are all reduced at the modeling level (Yu et al., 16 Sep 2025).

3. Unified document knowledge and OCR learning

MiniCPM-V 4.5 treats document knowledge acquisition and text recognition as variants of the same problem. The paper states that “the key difference between document knowledge acquisition and text recognition is the visibility of the text,” and operationalizes this insight through dynamic corruption of text regions in document images (Yu et al., 16 Sep 2025).

In this training formulation, text regions are selected as supervision targets and corrupted at different intensities. Under low corruption, text remains legible and the task is effectively robust OCR. Under moderate corruption, character-level evidence becomes ambiguous, and the model must combine noisy visual evidence with document context. Under high corruption, the region is fully masked, so the model must infer the missing text from surrounding layout, figures, charts, and general knowledge. The objective remains constant across all regimes: predict the original text of the selected region (Yu et al., 16 Sep 2025).

This design yields a continuum rather than a task boundary. Robust OCR, context-aware text reconstruction, chart and table interpretation, and document-level reasoning are trained under the same supervision principle. The model therefore does not depend on a separate parser that converts PDFs into text-plus-images. Instead, it consumes document images directly and learns to reconstruct or answer from them.

The paper contrasts this unified learning paradigm with an external parser baseline. After identical downstream SFT, the unified approach improves MMMU from 49.0 to 51.4, AI2D from 74.9 to 76.5, and OCRBench from 576 to 617 (Yu et al., 16 Sep 2025). These deltas are important because they indicate that parser-free learning is not merely a simplification of the data pipeline; it is also associated with gains in both OCR robustness and knowledge-intensive reasoning.

A second misconception is that parser-free document training necessarily sacrifices structured document knowledge. The reported ablation argues against that interpretation: the same corruption-based paradigm improves OCRBench while also improving MMMU and AI2D, suggesting that direct image-grounded supervision can strengthen both recognition and higher-level inference (Yu et al., 16 Sep 2025).

4. Training curriculum and hybrid reinforcement learning

The training recipe is staged. Pre-training begins with a warm-up stage in which only the 2D-Resampler is trained while the vision encoder and LLM remain frozen, using image-caption data such as LAION-2B, COYO, and Chinese web data. The second stage unfreezes the vision encoder while keeping the LLM frozen, using OCR-rich data and image-caption data. The third stage unfreezes everything and trains on a higher-quality multimodal mixture including text-only corpora, image-text interleaved data such as OmniCorpus and MINT-1T, video caption data such as WebVid, Vript, and OpenVid, and curated subsets from earlier stages. The scheduler is Warmup-Stable-Decay, with the learning rate fixed at $5\times 10^{-5}$ in the stable phase and decaying to $1\times 10^{-5}$ (Yu et al., 16 Sep 2025).

Supervised fine-tuning is also divided into two stages. The first uses diverse multimodal instruction-response pairs plus 10% text-only high-quality instructions, which the paper presents as preserving text-only performance while activating multimodal capabilities. The second stage simultaneously upgrades the architecture from a 2D-Resampler to a unified 3D-Resampler and introduces long chain-of-thought data, with particular emphasis on hard examples from OpenThoughts and an in-house pipeline (Yu et al., 16 Sep 2025). The paper explicitly states that filtering out easy prompts is important for effective Long-CoT training.

Reinforcement learning is then used to obtain controllable reasoning length. The model is trained in two prompt-controlled modes using the same policy network: a short mode for concise answers and a long mode for explicit chain-of-thought reasoning. Rollouts are split evenly, with 50% of prompts assigned to long reasoning and 50% to short reasoning. The algorithm is GRPO, with 128 prompts per batch, 8 responses per prompt, a maximum response length of 8192 tokens, temperature 1.0, and a fixed learning rate of $1\times 10^{-6}$ (Yu et al., 16 Sep 2025).

The reward is a weighted sum of answer accuracy, formatting, repetition control, and a standardized preference-model score:

$R = R_{\text{acc}} + R_{\text{format}} + R_{\text{rep}} + \frac{1}{2}\tilde{R}_{\text{rm}},$

with

$\tilde{R}_{\text{rm}} = \frac{R_{\text{rm}} - \bar{R}_{\text{rm}}}{\sigma(R_{\text{rm}})}.$

The preference reward is applied only to the final answer portion in long mode, not to the intermediate chain of thought, which the paper motivates as a way to avoid unstable optimization on internal reasoning tokens (Yu et al., 16 Sep 2025).

The reward design mixes rule-verified accuracy for structured answers, probability-based rewards for more complex free-form answers, and a preference reward model. The paper argues that this addresses three issues simultaneously: label accuracy, reward accuracy, and reward coverage. It further adds a distinct RLAIF-V stage after GRPO, where multiple responses are sampled for the same prompt, decomposed into atomic factual claims, verified by AI agents aligned with the image or video, scored, and then used for DPO-based preference learning (Yu et al., 16 Sep 2025).

The main RL ablation compares three settings. Short-only RL yields an OpenCompass score of 76.0 with 1.6B training tokens; long-only RL yields 77.0 with 4.4B tokens; hybrid RL yields 77.1 with 3.1B tokens (Yu et al., 16 Sep 2025). The hybrid setting therefore matches or slightly exceeds long-only performance while using fewer RL tokens, and the paper interprets this as evidence that short and long modes regularize each other rather than competing.

5. Efficiency profile and empirical scaling

MiniCPM-V 4.5’s efficiency claims are tightly tied to token compression and controllable output length. On VideoMME with subtitles, the reported comparison is 73.5 for MiniCPM-V 4.5, 73.6 for GLM-4.1V-9B-thinking, and 71.6 for Qwen2.5-VL-7B; the corresponding inference times are 0.26 h, 2.63 h, and 3.00 h, with memory usage of 28 GB, 32 GB, and 60 GB respectively. The paper summarizes this as 46.7% GPU memory cost and 8.7% inference time of Qwen2.5-VL 7B (Yu et al., 16 Sep 2025).

On OpenCompass among thinking models under 30B, MiniCPM-V 4.5 reports an average score of 77.0 with 7.5 h inference time, compared with 76.6 and 17.5 h for GLM-4.1V-9B-thinking and 76.4 and 11.0 h for MiMo-VL-7B-RL (Yu et al., 16 Sep 2025). The efficiency claim here is not that the model is universally faster than any smaller baseline in any setup, but that it attains a higher accuracy–latency operating point than contemporary open multimodal reasoning models in the evaluated configuration.

The architectural ablation on video compression is particularly revealing. After fine-tuning both variants for 300 steps after general SFT, a 2D-Resampler version obtains 65.5 on Video-MME with subtitles and uses 64.0 tokens per frame, whereas the 3D-Resampler obtains 67.3 and uses 21.3 tokens per frame (Yu et al., 16 Sep 2025). The paper therefore presents the 3D formulation as simultaneously more accurate and more efficient, rather than as an accuracy-for-speed trade-off.

Evaluation	MiniCPM-V 4.5	Comparison reported
OpenCompass avg.	77.0	GPT-4o-latest 75.4; Qwen2.5-VL 72B 76.1
VideoMME w/ subtitles	73.5	Qwen2.5-VL-7B 71.6; GLM-4.1V-9B-thinking 73.6
VideoMME inference time	0.26 h	Qwen2.5-VL-7B 3.00 h
VideoMME memory	28 GB	Qwen2.5-VL-7B 60 GB

One misconception is that these efficiency gains arise mainly from quantization or serving tricks. The paper instead attributes them chiefly to aggressive visual token reduction via cross-attention resampling, the unified 3D-Resampler, and the ability to operate in short mode for tasks that do not require long reasoning (Yu et al., 16 Sep 2025). Quantization recipes are not the center of the 4.5 paper’s argument.

6. Benchmark performance, reliability, and place in the MiniCPM family

Across benchmark groups, MiniCPM-V 4.5 is presented as an 8B-class model that is often competitive with or superior to much larger multimodal systems. On comprehensive multimodal evaluation, it reports MMVet 75.5, MMStar 72.1, MME 2500, and MMBench v1.1 84.2; on STEM-oriented benchmarks, MMMU 67.7, MathVista 79.9, AI2D 86.5, MathVerse mini 58.8, LogicVista 57.0, and EMMA 34.8; on document and OCR evaluation, OCRBench 89.0, ChartQA 87.4, TextVQA 82.2, DocVQA 94.7, and OmniDocBench 0.175 on English and 0.253 on Chinese, with lower being better for OmniDocBench (Yu et al., 16 Sep 2025).

The hallucination profile is also a central part of the model’s reported identity. On HallusionBench it scores 61.2; on ObjHalBench it reports CHAIRs 9.3 and CHAIRi 5.2, both lower than the listed Qwen2.5-VL and InternVL3 baselines; and on MMHal-Bench it reports score 5.0 and rate 19.4 (Yu et al., 16 Sep 2025). These results are presented as consequences of the RLAIF-V stage and the claim-level verification procedure, not merely of scale.

The model is also strong on multi-image and video tasks. Reported values include Mantis 82.5, MMT-Bench 68.3, RealWorldQA 72.1, MM-IFEval 66.0, Video-MME 67.9 without subtitles and 73.5 with subtitles, LVBench 50.4, MLVU 75.1, LongVideoBench 63.9, MotionBench 59.7, and FAVOR-Bench 56.0 (Yu et al., 16 Sep 2025). The paper’s interpretation is that the compact token interface does not substantially narrow the representational range of the model; instead, it enables a better compute allocation across image, document, and video workloads.

MiniCPM-V 4.5 also occupies an important position in the later MiniCPM family. MiniCPM-o 4.5, a 9B omni-modal model for real-time full-duplex interaction, is explicitly described as being based on a pretraining checkpoint of MiniCPM-V 4.5 and as reusing its visual system while expanding into streaming audio, speech generation, and omni-modal interaction (Cui et al., 30 Apr 2026). This indicates that MiniCPM-V 4.5 functions not only as a standalone vision-LLM but also as the architectural and checkpoint basis for a broader “4.5” generation within MiniCPM.

A plausible implication is that MiniCPM-V 4.5 should be understood less as a single benchmark-driven release and more as a compact multimodal backbone whose main innovations—unified resampling, unified document learning, and hybrid reasoning control—were intended to scale outward to adjacent modalities and interaction paradigms. The available evidence for that reading is the direct reuse of its checkpoint and visual design in MiniCPM-o 4.5, together with the strong document, OCR, and video results reported for the base vision-LLM itself (Yu et al., 16 Sep 2025, Cui et al., 30 Apr 2026).