GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents

要約

グラフィカルユーザーインターフェイス（GUI）エージェントの構築における既存の取り組みは、大規模な視覚言語モデル（LVLMS）で監視された微調整のトレーニングパラダイムに大きく依存しています。
ただし、このアプローチには、幅広いトレーニングデータが必要であるだけでなく、GUIスクリーンショットを効果的に理解し、目に見えないインターフェイスに一般化するのに苦労しています。
この問題は、特に高レベルのタスクでは、実際のシナリオでのアプリケーションを大幅に制限しています。
大規模な推論モデル（例えば、Deepseek-R1）の強化微調整（RFT）に触発され、実際の設定での大規模な言語モデルの問題解決機能を効率的に強化します。
By leveraging a small amount of carefully curated high-quality data across multiple platforms (including Windows, Linux, MacOS, Android, and Web) and employing policy optimization algorithms such as Group Relative Policy Optimization (GRPO) to update the model, \name achieves superior performance using only 0.02\% of the data (3K vs. 13M) compared to previous state-of-the-art methods like OS-Atlas across eight benchmarks spanning three different
プラットフォーム（モバイル、デスクトップ、およびWeb）。
これらの結果は、実世界のGUIエージェントタスクのLVLMSの実行能力を改善する際の統一されたアクション空間ルールモデリングに基づいて、強化学習の計り知れない可能性を示しています。

要約(オリジナル)

Existing efforts in building Graphical User Interface (GUI) agents largely rely on the training paradigm of supervised fine-tuning on Large Vision-Language Models (LVLMs). However, this approach not only demands extensive amounts of training data but also struggles to effectively understand GUI screenshots and generalize to unseen interfaces. The issue significantly limits its application in real-world scenarios, especially for high-level tasks. Inspired by Reinforcement Fine-Tuning (RFT) in large reasoning models (e.g., DeepSeek-R1), which efficiently enhances the problem-solving capabilities of large language models in real-world settings, we propose \name, the first reinforcement learning framework designed to enhance the GUI capabilities of LVLMs in high-level real-world task scenarios, through unified action space rule modeling. By leveraging a small amount of carefully curated high-quality data across multiple platforms (including Windows, Linux, MacOS, Android, and Web) and employing policy optimization algorithms such as Group Relative Policy Optimization (GRPO) to update the model, \name achieves superior performance using only 0.02\% of the data (3K vs. 13M) compared to previous state-of-the-art methods like OS-Atlas across eight benchmarks spanning three different platforms (mobile, desktop, and web). These results demonstrate the immense potential of reinforcement learning based on unified action space rule modeling in improving the execution capabilities of LVLMs for real-world GUI agent tasks.

arxiv情報

提供元, 利用サービス

arxiv.jp, Google