Abstract
Thinking with images empowers vision-language models (VLMs) to use an external Zoom-in tool for fine-grained visual search. Recent works employ supervised fine-tuning (SFT) and reinforcement learning (RL) with tool-specific designs to enable such capabilities. However, achieving logical reasoning alongside accurate and efficient tool use remains a significant challenge. To address this, we shift the focus away from increasing RL complexity and instead identify SFT data quality as a key bottleneck. We then propose LogT (short for Logically Think with Images), a fully automated SFT data engine that explicitly enforces logic-chain-guided reasoning. LogT generates an SFT dataset featuring challenging, visually grounded questions and structured tool-use traces with human-like logic. We validate our approach by fine-tuning two base models, Qwen2.5-VL-7B and InternVL3.5-8B, on the dataset and evaluating them across seven benchmark test sets. We show that LogT-SFT based on Qwen2.5-VL-7B outperforms the state-of-the-art trained with both SFT and RL, despite using 3.1times less data and >120times fewer GPU hours. Furthermore, applying a simple RL recipe without tool-specific design on LogT-SFT yields a model that surpasses prior methods with complex RL setups by a substantial 9.59-point margin. In addition to strong performance gains, our model exhibits higher tool-use accuracy and efficiency, as well as more structured and logically consistent reasoning traces, directly validating the effectiveness of our approach.
Community
When answering questions about images, vision-language models (VLMs) can benefit from zooming in on small but important details, much like humans do. However, teaching VLMs when and how to zoom in typically requires complex post-training methods. In this work, we take a different approach: instead of increasing training complexity, we focus on improving the quality of supervised fine-tuning (SFT) data. We introduce LogT, an agentic data engine that generates high-quality SFT examples. Remarkably, models trained solely on our SFT data outperform those trained with both SFT and reinforcement learning (RL). With standard, off-the-shelf RL, our model surpasses the state-of-the-art open-source model by an average of 9.59 points across seven test sets. These results demonstrate that high-quality SFT data can be more effective than sophisticated training methods, offering a much more efficient yet powerful approach to improving visual reasoning in VLMs.
This work was accepted at NeurIPS 2026.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Distilling Visual Reasoning into Text Space (2026)
- HaPRL: Human-Anchored Process Reinforcement Learning for Visual Search Agent (2026)
- Zooming without Zooming v2: From Seeing Details to Connecting Evidence for Generalizable Multimodal Perception (2026)
- Long Listening Thoughts: Eliciting Open Auditory Reasoning with Deliberative Perception and Cognitive Refinement (2026)
- Genesis: Visual Augmentation Primitives for Multimodal Chain-of-Thought Reasoning (2026)
- Look at Where You Think: Localization-Head Alignment for Fine-Grained Perception in Thinking VLMs (2026)
- Learning Where to Look: Observation Policy Optimization for Thinking with Images (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper