V-tableR1: Process-Supervised Multimodal Table Reasoning with Critic-Guided Policy Optimization

A new framework called V-tableR1 has been developed by researchers, focusing on process-supervised reinforcement learning to enhance the reasoning capabilities of multimodal large language models (MLLMs). Traditional MLLMs, which are trained mainly on end results, often view visual reasoning as a complex issue, relying on basic pattern recognition instead of thorough multi-step reasoning. Although Reinforcement Learning with Verifiable Rewards aims to promote clear reasoning paths, applying it to visual contexts is challenging due to the difficulty in translating abstract logic into pixel data. This framework utilizes the structured format of tables as an effective visual testing ground. V-tableR1 incorporates a unique critic VLM that offers detailed feedback on the visual reasoning process produced by a policy VLM. To refine this approach, the authors introduce a new RL algorithm called Process-Guided Direct Alignment Policy Optimization (PGPO). The research is available on arXiv under the identifier 2604.20755.