---
title: 'Infogent: Modular Web Aggregation Framework'
url: https://www.emergentmind.com/topics/infogent
type: topic
---

# Infogent: Modular Web Aggregation Framework

Infogent denotes a modular, agent-based framework for web information aggregation that is designed to solve complex information-seeking tasks beyond the traditional linear web navigation paradigm. Unlike conventional web agents that optimize for task completion through a sequence of actions leading to a single end state, Infogent targets open-ended scenarios where the agent must navigate, extract, and aggregate information from diverse web sources to address multi-faceted user queries. The architecture is flexible across access modalities, supporting both direct API-driven (text-based) and interactive visual (browser-based) interactions. Infogent demonstrates measurable improvements over previous state-of-the-art (SOTA) web search and task completion agents in benchmark settings [2410.19054].

## 1. System Architecture

The Infogent system is structured around three collaborating black-box modules—Navigator, Extractor, and Aggregator—coordinated in an iterative feedback-driven loop. This modular decomposition underpins flexible system composition and principled delegation of sub-tasks.

- **Navigator ($\mathcal{NG}$):** Receives the user’s task $\mathcal{T}$, current webpage context $\mathcal{W}_t$ (either as URL+snippet or screenshot+DOM), feedback message $\mathcal{F}$ from Aggregator, and partial action history. It outputs an action $a_t$ from the action space $\mathcal{A}$ (including SEARCH, CLICK, AGGREGATE, GO BACK, TERMINATE). The Navigator explores the web, initiates state transitions, and decides when to hand control to the Extractor or Aggregator.

- **Extractor ($\mathcal{ET}$):** Given the current webpage, the task, and feedback, outputs a set of relevant text passages $P = \{p_1, ..., p_k\}$, typically by prompting an LLM to select up to $k$ pertinent paragraphs.

- **Aggregator ($\mathcal{AG}$):** Given newly extracted passages and the current “stack” $S$ of aggregated items, as well as the original task, decides (a) which passages to add or replace in $S$ (avoiding redundancy and promoting topical diversity), (b) generates updated feedback $\mathcal{F}$ for the Navigator, and (c) internally determines whether information sufficiency has been met (TERMINATE).

**Coordination pseudocode** (see Algorithm 1 in [2410.19054]):

```
Input: user task 𝒯, max sites K, max steps N
Initialize: 𝒲₀ ← “Search Home”, S₀ ← [], ℱ ← “None”, k ← 0, t ← 0
While aₜ ≠ TERMINATE ∧ k < K ∧ t < N:
    aₜ₊₁ ← ℕ𝔾(𝒲ₜ, 𝒯, ℱ, {a₁…aₜ})
    if aₜ₊₁ = AGGREGATE:
        P ← ℰ𝒯(𝒲ₜ, 𝒯, ℱ)
        (Sₖ₊₁, ℱ) ← 𝒜𝔾(Sₖ, P, 𝒯)
        k ← k+1
        𝒲ₜ₊₁ ← 𝒲ₜ
    else:
        𝒲ₜ₊₁ ← Act(𝒲ₜ, aₜ₊₁)
    t ← t+1
Output: aggregated stack S
```
Each module is implemented via purpose-built LLM or (for the visual modality) LMM prompt chains, without gradient-based training.

## 2. Formal Task Definitions

While no explicit RL objectives or loss functions are introduced, Infogent formalizes three module mappings:

- **Navigator policy:**
  $$
  a_t = \mathcal{NG}(\mathcal{W}_t, \mathcal{T}, \mathcal{F}, \{a_1, ..., a_{t-1}\})
  $$
- **Extractor mapping:**
  $$
  P = \mathcal{ET}(\mathcal{W}, \mathcal{T}, \mathcal{F})
  $$
- **Aggregator update:**
  $$
  (S', \mathcal{F}) = \mathcal{AG}(S, P, \mathcal{T})
  $$
There is no inherent scoring function; aggregation quality is evaluated at the macro level via task-level metrics.

## 3. Information Access Modalities

Infogent targets two modes of web interaction, supporting both textual and visual web agent paradigms:

**Direct API-Driven Access:**  
- The Navigator (ReACT-based) issues SEARCH and AGGREGATE API calls—SEARCH returns top-5 (URL, snippet) pairs via the Google Search API; AGGREGATE invokes Extractor and Aggregator on the selected URL.
- The Extractor processes HTML content to extract relevant paragraphs.
- The Aggregator operates as previously described.

**Interactive Visual Access:**  
- The Navigator (multimodal agent, SeeAct + Playwright) consumes sequential browser screenshots and DOM element candidates, issuing actions such as CLICK, TYPE, PRESS ENTER, SELECT, AGGREGATE, or TERMINATE.
- The Extractor uses a vision-capable LLM (e.g., GPT-4V), acquiring multiple screenshots and extracting text.
- The Aggregator’s mechanism is unchanged.

The stack-based aggregation and feedback mechanism are consistent across both paradigms.

### Access Setting Comparison Table

| Setting                  | Navigator Modality   | Extraction Method         |
|--------------------------|---------------------|--------------------------|
| API-Driven Access        | LLM + Search APIs   | HTML parsing, LLM prompt |
| Interactive Visual Access| LMM + Screenshots   | Vision-LMM prompt        |

## 4. Training Paradigm and Module Implementation

Infogent is instantiated entirely via off-the-shelf OpenAI APIs (GPT-4-Turbo, GPT-4o; ablations include GPT-4o-mini, GPT-3.5 for extraction), without any supervised module fine-tuning or novel dataset annotation. All agent logic is encoded in engineered prompt templates (cf. Tables A.3–A.5 in [2410.19054]). There is no learned policy or gradient update at any stage, making the system trivially deployable on new information-seeking tasks but highly dependent on the capabilities of the underlying LLM/LMM backbone, especially for the Navigator.

## 5. Empirical Evaluation

Infogent is systematically benchmarked under both access settings against multiple SOTA and strong baseline agents.

- **Benchmarks**:
    - *FRAMES* (531 items): reasoning over web tables, temporal relationships, post-processing, and constraint satisfaction.
    - *FanOutQA* (310 multi-hop Wikipedia questions).
    - *AssistantBench* (214 browser automation tasks; both dev and test splits).
    - Human-curated FanOutQA subset (31 queries failing closed-book models).

- **Metrics**:  
    - FRAMES: Overall LLM-judge accuracy.
    - FanOutQA: Exact-match accuracy; ROUGE-1/2/L.
    - AssistantBench: Final-answer accuracy.

- **Representative Results**:
    - FRAMES (Direct API):
        - Closed-Book: 23.5%
        - MindSearch: 46.3%
        - Infogent: **53.3%** (+7.0 over MindSearch)
    - FanOutQA (dev, API):
        - Closed-Book: 46.6%
        - MindSearch: 47.3%
        - Infogent: **51.1%** (+3.8)
    - AssistantBench (test, Visual):
        - RALM-Inst: 11.7%
        - SPA: 11.0%
        - Infogent (GPT-4o): **15.3%** (+4.3 over SPA)

Ablations indicate Navigator is the most bottlenecked module: downgrading to GPT-4o-mini for navigation drops accuracy to 0% on AssistantBench, while less powerful Extractor/Aggregator models incur only minor reductions (~2–3%).

## 6. Analysis, Insights, and Limitations

**Architectural Insights:**  
- Modularity enables specialization and composability: delegating extraction and aggregation relieves the Navigator, increasing coverage and reducing error rate [2410.19054].
- The natural language feedback loop, where the Aggregator returns contextually adaptive guidance $\mathcal{F}$, repeatedly diagnoses coverage gaps and refocuses the Navigator, yielding higher-quality aggregation.

**Limitations:**  
- Performance is heavily gated by the LLM backbone, especially for navigation.
- The framework lacks explicit objective metrics for “information sufficiency”—termination is governed solely by Aggregator-issued feedback.
- No gradient-based learning, adaptation to domain shifts, or listener-specific priors; all prompt-driven.
- Fails on rich interactive UIs (geo-navigation), pop-ups, and elements that require persistent local state or real-time interactions.

**Future Directions:**  
- Introducing learned components, such as retrieval modules or RL policies, to reduce dependency on generalist LLMs.
- Development of automatic and nuanced measures of information coverage, diversity, and aggregation sufficiency.
- Extension to GUI-based information aggregation beyond the web (e.g., Slack, mobile apps), leveraging the visual interaction paradigm.
- Construction of new benchmarks for subjective and open-ended aggregation tasks beyond multi-hop QA.

## 7. Connections and Extensions

While Infogent is focused on web-scale information aggregation, its architectural principles—modular decomposition, feedback-driven control, and aggregation stack memory—resonate with broader developments in agent-based information systems. The approach is compatible with entropy-based information selection paradigms, as in the informativity-maximizing label selection exposed by [1812.10358]. A plausible implication is that integrating entropy-based selection at the Aggregator level could yield more principled aggregation in domains where coverage and diversity are quantifiable. Nonetheless, Infogent itself does not implement Shannon-entropy reduction or listener-model adaptation as formalized in [1812.10358]. This suggests a fertile intersection for future hybrid agents incorporating both modular search/aggregation and entropy-theoretic label or passage selection mechanisms.

Source: https://www.emergentmind.com/topics/infogent