AgentRxiv: Collaborative Autonomous Research

• AgentRxiv is a centralized meta-infrastructure that organizes collaborative LLM agent research via a dedicated preprint server.
• It accelerates discovery by enabling both sequential and parallel research pipelines while integrating novel self-checking and reasoning strategies like SDA.
• Quantitative evaluations demonstrate significant performance gains, underscoring the efficacy of cumulative innovation and multi-agent collaboration.

AgentRxiv is a centralized framework for collaborative autonomous research, designed to enable independent LLM agent laboratories to iteratively share, retrieve, and build on each other’s research outputs. This meta-infrastructure pursues cumulative, scalable advancement in complex tasks such as benchmark mathematics problem solving by facilitating reference, adaptation, and synthesis of methods across autonomous laboratories. By providing an agent-accessible preprint server modeled after platforms like arXiv and bioRxiv, AgentRxiv orchestrates collective progress, supporting both sequential and parallel research pipelines and fostering the emergence of new reasoning strategies and robust evaluation protocols.

1. Architecture and Collaboration Mechanism

AgentRxiv’s core construct is a centralized, archival “preprint server” specifically designed for artificial research agents. Each autonomous agent laboratory comprises an end-to-end research workflow, including literature review, experimentation, and writing of research papers. Laboratories generate and contribute these artifacts—simulated papers—by uploading to AgentRxiv.

Upon initialization, an agent laboratory receives access to a fixed subset of previously generated papers ($N=5$ for most experiments), using these as a cross-laboratory knowledge base. This enables agents to:

• Revisit and refine previously explored methodologies
• Incorporate or synthesize successful strategies across independent labs
• Accelerate progress toward improved benchmark results

Parallelization is also natively supported: multiple agent laboratories can operate concurrently, sharing knowledge through the repository. While this parallel setup increases total computational cost due to redundancy, it yields faster wall-clock convergence to higher accuracy and promotes diverse exploration strategies.

AgentRxiv Component	Function	Example Parameter
Preprint Server	Archive and serve agent-generated papers	$N=5$ context docs
Research Pipeline	Autonomously conduct literature review and experimentation	Full agent cycle
Parallel Laboratories	Concurrent experiments and shared output	$3\times$ labs

2. Novel Reasoning and Prompting Strategies

A central innovation within AgentRxiv is the emergence and refinement of new reasoning and prompting techniques to address problems such as those in MATH-500.

Early approaches included:

• Dynamic Critical Chain Prompting (DCCP): Adjusting chain-of-thought (CoT) orderings to focus on pivotal solution points
• Context-Aware Recursive Uncertainty Calibration (CRUC): Recursive prompts sensitive to stepwise model uncertainty

Iterations built increasingly sophisticated mechanisms, including:

• Dual-Rebuttal CoT Voting: Generation and comparison of independent CoT traces, with rebuttal and majority selection
• Meta-Mirror Prompting: Meta-reflection on disagreement between divergent output chains
• Dual-Role Divergence Prompting and Enhanced CoT Verification: Simultaneous, independent reasoning pathways with rigorous cross-validation

The highest-performing method, Simultaneous Divergence Averaging (SDA), operates as follows:

• For each problem, generate:
  • A Precise Solver response (low temperature; deterministic, precise)
  • A Creative Evaluator response (high temperature; diverse, potentially novel)
• Encode both responses using Sentence-BERT, yielding embeddings $E_1$ and $E_2$
• Compute cosine similarity:

$$\text{cos\_sim} = \frac{E_1 \cdot E_2}{\|E_1\| \cdot \|E_2\|}$$

• If similarity exceeds a dynamic threshold, select the higher-confidence answer; otherwise, trigger meta-reassessment for reconciliation

These techniques exemplify emergent self-checking and multi-path evaluation paradigms under collaborative agent architectures.

3. Quantitative Evaluation and Benchmarking

The efficacy of the AgentRxiv framework was systematically evaluated across prominent benchmarks.

• MATH-500: Starting from a baseline with gpt-4o mini at $70.2\%$ accuracy, cumulative research and the implementation of SDA yielded $78.2\%$, an $11.4\%$ relative gain
• Parallel Laboratories: Collective operation of three labs, each producing $40$ papers, enabled earlier milestone attainment ($76.2\%$ accuracy) compared to a sequential single-lab setup, at increased aggregate cost
• Cross-Domain Generalization: Strategies developed under AgentRxiv transferred with an average $3.3\%$ improvement across benchmarks such as GPQA, MMLU-Pro, and MedQA, across several LLM families
• Efficiency Metrics: The runtime, average wall-clock time per generated paper, and explicit computational and financial costs provide a balanced picture of research velocity versus resource utilization

4. Cumulative Innovation and Scientific Acceleration

AgentRxiv’s collaborative structure allows autonomous agents not only to explore the solution space in parallel but also to build cumulatively on established work, analogous to human scientific communities.

Key implications include:

• Cumulative Knowledge Formation: By referencing, critiquing, and extending previous research, agent labs reconstruct a collective trajectory mirroring human distributed scientific advancement
• Accelerated Discovery: Parallel research pipelines facilitate rapid identification and propagation of effective reasoning strategies, significantly reducing time to achieve new state-of-the-art results
• Generalized Reasoning Mechanisms: The cross-benchmark transferability of prompts and evaluation routines indicates a route to robust, domain-general research methodologies
• Design Principles for Future AI: The effectiveness of these multi-agent collaborative approaches suggests their suitability for AI system design in research, decision-making, and complex optimization tasks
• Self-Verification Infrastructures: Techniques incorporating multi-path and divergence-based self-checking (such as SDA) foreshadow a trajectory toward AI systems with intrinsic reliability and transparency mechanisms, critical for autonomous scientific research

A plausible implication is that these mechanisms will be foundational to trustworthy automated discovery pipelines.

5. Algorithmic Outline and Mathematical Formalism

The key operations of the Simultaneous Divergence Averaging (SDA) strategy, as codified in AgentRxiv, can be abstracted as:

For each task/problem: Response_1 = Precise Solver (temperature = T_low) Response_2 = Creative Evaluator (temperature = T_high) E1 = SentenceBERT(Response_1) E2 = SentenceBERT(Response_2) cos_sim = (E1 ⋅ E2) / (||E1|| ⋅ ||E2||) if cos_sim ≥ threshold: Select answer with higher confidence else: Trigger meta-reassessment for reconciliation

Both responses include a final answer (LaTeX-formatted) and an explicit confidence score. The dynamic divergence threshold controls the trade-off between consensus and diversity, and meta-reassessment is invoked for substantial disagreement, as detailed in the technical appendix.

6. Broader Implications for AI Research and System Design

The operational paradigm introduced by AgentRxiv demonstrates the viability and advantage of collaborative, knowledge-sharing infrastructures for autonomous research agents. Core takeaways include:

• Autonomous agent collectives can function as research accelerants, producing more rapid and robust advances than isolated agents
• The analogs to human scientific practice—archival publication, refinement of prior art, and collaborative evaluation—incline these systems toward scalable, transparent, and generalizable innovation
• The self-verification mechanisms that emerge under collaborative pressure lay groundwork for trustworthy autonomous systems, with implications for both research automation and real-world scientific deployment

This suggests a convergence of human and agent-driven discovery paradigms, where future AI systems may employ agent collectives as partners in knowledge creation, decision-making, and scientific progress.