Fara-7B: Efficient Multimodal Computer Use Agent

• Fara-7B is a 7-billion parameter multimodal Transformer designed as a native computer use agent that processes screenshots and issues precise GUI actions.
• It leverages the FaraGen synthetic data pipeline to generate, verify, and train on diverse web task trajectories, achieving superior benchmark performance and cost-efficiency.
• Its streamlined, pixel-in, action-out architecture supports on-device deployment with low latency, paving the way for practical, deployable computer automation solutions.

Fara-7B is a 7-billion parameter multimodal Transformer specifically designed as a native computer use agent (CUA) that interacts with graphical user interfaces via direct pixel input and atomic action output. Trained entirely on a large-scale, high-diversity synthetic corpus generated through the FaraGen closed-loop pipeline, Fara-7B achieves state-of-the-art performance among models of comparable size on several computer use agent benchmarks and demonstrates competitive performance relative to significantly larger frontier system-of-models (SoM) agents. The model and its supporting artifacts—including the WebTailBench benchmark—are released openly to encourage reproducibility and further research (Awadallah et al., 24 Nov 2025).

1. FaraGen: Synthetic Data Generation Pipeline

FaraGen is a closed-loop, multi-agent pipeline devised to address the data bottleneck in training CUAs by systematically generating and verifying high-quality, multi-step web task trajectories (§2). Grounded in seed URLs from large web indices (ClueWeb22, Tranco), FaraGen leverages three principal modules: task proposal, task solving, and trajectory verification.

• Task Proposal combines (a) targeted URL-based intent extraction and LLM prompting (“AgentInstruct” style) for subcategory-specific task generation (28% tasks) and (b) multimodal agentic exploration, where an agent (“WebExplorer”) generates tasks during live browsing (67%). An additional 5% uses LLM-driven exemplar expansion to diversify natural language variants.
• Task Solving is mediated by the Magentic-One dual-agent system: an Orchestrator plans high-level task steps and enforces “critical-point” constraints, while a WebSurfer agent makes atomic UI tool calls (click, type, scroll, memorize, terminate) using step context and screenshots. Trajectories halt at critical points (personal data, irreversible actions) for safety and verification.
• Trajectory Verification employs three LLM-based verifiers: Alignment (textual intent satisfaction), Rubric (per-task rubric with subgoal scoring, threshold 0.8), and Multimodal (joint vision-text alignment on key screenshots).
• Throughput and Corpus Yield: With distributed orchestration (40 nodes × 4 browsers), FaraGen achieves ~600 verified trajectories/hour. The final corpus comprises 145,603 verified trajectories (1,010,797 total steps) across 70,117 unique web domains, with solution verification success rates ranging (e.g., Shopping: 9%, Flights: 3–11%). Trajectory synthesis costs are consistently ≈$1/trajectory even using state-of-the-art LLMs (e.g.,$0.59 for o4-mini, $1.08 for o3,$1.00 for GPT-5), establishing scalable feasibility.

2. Fara-7B Model Architecture and Training

Fara-7B is a supervised fine-tuned version of Qwen2.5-VL-7B with a multimodal "pixel-in, action-out" architecture (§3). The model processes the computer screen as a screenshot and generates both natural language reasoning outputs (chain-of-thought) and atomic GUI actions.

• Modal Input/Output: A vision encoder embeds the screenshot into patch tokens, which are jointly attended with initial user instructions and previous reasoning steps. At each timestep $t$, the model predicts: (1) chain-of-thought $r_t$ as text, and (2) an action $a_t$ via tool-call syntax (e.g., click(x,y), type("..."), scroll, memorize, terminate). Click locations are grounded via directly emitted $(x, y)$ coordinate tokens.
• Learning Objective: Training optimizes the standard autoregressive cross-entropy objective on joint reasoning and action token sequences:

$\log P(r_t, a_t \mid \mathbf{x}_{<t})$

where $\mathbf{x}_{<t}$ includes historic screenshots, thoughts, and actions.

• Data Mixture (Table A.1): The training set includes 1.23M trajectory steps, 0.56M grounding samples (ScreenSpot-style), 3.1K refusal/denial examples, and small UI-VQA and captioning sets (1.8K each). Trajectories are upsampled for rare/long types; grounding data is critical for precise $(x, y)$ predictions.
• Hyperparameters: AdamW with $\beta_1 = 0.9$, $1.08 for o3,$0, weight decay 0.1; cosine LR with 10% warmup to $1.08 for o3,$1; batch size 128, 28K iterations over 2 epochs; trained on 64× NVIDIA H100 GPUs using DeepSpeed Stage 3, bfloat16 precision.

3. Evaluation on Benchmarks

Fara-7B's performance is assessed on multiple benchmarks that probe web automation and agentic computer use capability (§4):

• Benchmarks:
  • WebVoyager: 97 live web navigation tasks; judged by GPT-4o.
  • Online-Mind2Web: 60 information-seeking/form-filling tasks; judged by o4-mini.
  • DeepShop: 50 e-commerce navigation and checkout tasks.
  • WebTailBench: 609 hand-verified, diverse tasks (single-site and multi-site, including under-represented domains like Real Estate, Jobs, Compositional Shopping); judged by o4-mini.
• Results (Table 5):

| Model | Params | WebVoyager | Online-M2W | DeepShop | WebTailBench | |-------------|--------|:----------:|:----------:|:--------:|:-------------:| | SoM GPT-5 | – | 90.6±0.6 | 57.7±2.1 | 49.1±3.4 | 60.4±0.8 | | SoM o3 | – | 79.3±0.6 | 55.4±6.5 | 49.7±3.3 | 52.7±1.9 | | SoM GPT-4o | – | 65.1±0.6 | 34.6±1.5 | 16.0±2.3 | 30.8±3.0 | | UI-TARS-1.5-7B | 7 B | 66.4±0.8 | 31.3±2.6 | 11.6±1.4 | 19.5±2.0 | | Fara-7B | 7 B | 73.5±1.0 | 34.1±3.7 | 26.2±2.0 | 38.4±0.7 |

Fara-7B exceeds all other 7B-parameter models by substantial margins (e.g., on WebTailBench, 38.4% vs 19.5% for UI-TARS-1.5-7B) and narrows the gap to large SoM systems (e.g., within ~3 points of OpenAI computer-use preview on WebVoyager).

• Efficiency (Table 6):

| Model | \$/task | Actions/task | In-tok/task | Out-tok/task | |---------------|:-------:|:------------:|:-----------:|:------------:| | SoM GPT-5 | 0.316 | 16.6±22.1 | 147K±249K | 13.0K±21.0K | | UI-TARS-1.5-7B| 0.082 | 41.3±37.2 | 408K±572K | 2.2K±2.8K | | Fara-7B | 0.025| 16.5±21.1 | 124K±202K | 1.1K±1.4K |

Fara-7B demonstrates 10× higher token- and cost-efficiency than SoM agents, made possible by direct coordinate emission and concise reasoning.

4. On-Device Deployment and Practical Design

Fara-7B's architecture and model size enable efficient on-device deployment, meeting key practical constraints for CUAs:

• Size and Hardware: The 7B-parameter footprint is amenable to int8 quantization, permitting single-device inference on desktop GPUs (8–12GB VRAM) or NN-accelerated mobile hardware (6–8GB VRAM).
• Inference Pipeline: By relying solely on screenshots for perception (no DOM or accessibility tree at inference), the model is robust to web markup changes and simplifies dependency management.
• Atomic Action Space: Supported actions—click, type, scroll, memorize, terminate—cover the majority of productivity and navigation tasks. Each inference step requires a single forward pass (~1 s on CPU, ~100 ms on desktop GPU).
• Limitations: The current architecture does not support drag-and-drop, multimodal input (audio/video), or the ultra-low-latency required for interactive gaming. Execution of irreversible/personal actions is halted, aligning with safety and verification design (§6).

5. Analysis, Limitations, and Future Directions

Fara-7B presents several insights into the development of agentic models for computer use:

• Synthetic Data as Enabler: FaraGen demonstrates that sufficiently diverse and high-quality synthetic data (at scale, ~145K trajectories from >70K domains) enables small LLMs to achieve robust and general CUA performance, overcoming the core data scarcity barrier.
• Modeling Choices: Pixel-in, action-out with direct coordinate grounding enables domain generalization without explicit reliance on web page structure, outperforming both prior 7B models and competitive with much larger SoM agents.
• Extensibility and Open Science: By open-sourcing Fara-7B, its evaluation harness, and WebTailBench, the release supports further research and reproducibility in small, deployable CUAs.

Known Limitations include restricted action space (no drag-and-drop, no audio/video input), lack of training beyond critical points (i.e., reservation/purchase flow endpoints), and incomplete robustness to site redesigns or edge-case error recovery. Future iterations anticipate extending end-to-end execution under simulated user consent, incorporating richer agent memory and meta-cognitive chains, and employing reinforcement learning for long-horizon optimization. Improved protocols for agent-user collaboration and periodic benchmark refreshes are also identified as research priorities (Awadallah et al., 24 Nov 2025).

6. Artifacts and Community Impact

Fara-7B, its weights, evaluation toolkit, and the WebTailBench benchmark are released on Microsoft Foundry and HuggingFace. The benchmark introduces novel, under-represented segments for CUA assessment and includes code for trajectory verification (§5). The release enables not only comparative evaluation but also supports advances in data-driven agentic modeling for practical, small-footprint deployment in personal and enterprise environments.