---
title: 'SurveyGen: Automatic Survey Generation'
url: https://www.emergentmind.com/topics/surveygen
type: topic
---

# SurveyGen: Automatic Survey Generation

SurveyGen most commonly denotes **automatic survey generation (ASG)**: the end-to-end production of an academic survey article from a topic query and a literature corpus. In the recent literature, the task is treated not as unconstrained long-form generation, but as a compound pipeline involving literature retrieval, structure induction, multi-document synthesis, and citation grounding. Systems such as SurveyX, SurveyForge, SurveyGen-I, SurveyG, and DeepSurvey therefore decompose survey writing into explicit preparation and generation stages, while benchmark efforts such as SurGE, SurveyBench, SurveyEval, SGSimEval, and SurveyLens evaluate outline quality, content quality, and reference quality against human-written surveys [2502.14776] [2503.04629] [2508.14317] [2510.07733] [2605.29522] [2508.15658] [2510.03120] [2512.02763] [2508.11310] [2602.11238].

## 1. Scope and task formulation

SurveyGen is usually framed as a retrieval-grounded survey-writing problem. SurGE formalizes it as: given a topic description \(t\) and a large academic corpus \(D = \{d_1, d_2, \ldots,d_n\}\), retrieve a relevant set \(\mathcal{R}_t \subseteq D\) and generate a survey article \(S\) that provides a structured and comprehensive overview of the topic [2508.15658]. SurveyX gives a closely related formulation in which a system must identify a representative literature set, organize it into a coherent taxonomy, synthesize results across papers rather than summarize one paper at a time, and attach citations that are both relevant and sufficient [2502.14776].

This formulation distinguishes SurveyGen from ordinary summarization. The output is expected to exhibit hierarchical structure, broad coverage, topic-focused relevance, synthesis across papers, critical analysis, and citation support. SurveyLens makes this decomposition explicit by representing each survey as \(S=(O,C,R)\), where \(O\) is the hierarchical outline, \(C\) is the content, and \(R\) is the reference list [2602.11238]. The same tripartite decomposition—outline, content, and references—recurs in SGSimEval and SurveyEval, which treat survey papers as structured scholarly artifacts rather than generic prose [2508.11310] [2512.02763].

A second, narrower use of closely related terminology appears in work on **Survey Response Generation**, where the task is to simulate closed-ended survey responses from persona-conditioned prompts rather than generate academic survey papers [2510.11586]. That line of work is methodologically adjacent, but it addresses a different target object: respondent answers rather than literature surveys.

## 2. Retrieval and evidence preparation

A defining feature of SurveyGen systems is that retrieval is not a single front-end step but a layered evidence-construction process. SurveyX uses a hybrid source consisting of a local arXiv collection with **2,632,189 papers as of Feb. 10, 2025** plus a self-developed Google Scholar crawler, then applies a **Keyword Expansion Algorithm**, a **2-step filtration algorithm**, and a structured pre-processing layer called **AttributeTree**, whose outputs form an **attribute forest** used later for RAG [2502.14776]. The system’s strongest ablation result is that removing AttributeTree sharply degrades structure, synthesis, critical analysis, and especially citation metrics, indicating that structured compression is not cosmetic but central.

Other systems push retrieval toward stronger bibliometric or graph-aware selection. The paper titled "SurveyGen" introduces **QUAL-SG**, which extends naïve RAG with **co-citation expansion** and **quality-aware re-ranking**. In its baseline form, paper-topic similarity is computed as
\[
\text{Sim}(q, d_i) = \cos\left( \mathbf{v}_q, \mathbf{v}_{d_i} \right),
\]
but QUAL-SG then augments the candidate set with papers cited by at least two initially retrieved papers and scores candidates using topical relevance, academic impact, and content diversity [2508.17647]. On the paper’s Task 2 benchmark, QUAL-SG improves citation quality to **P 15.87 / R 17.71 / F1 16.73**, compared with **P 5.18 / R 6.94 / F1 5.93** for Naive-RAG and **P 11.79 / R 5.78 / F1 7.76** for Fully-LLMGen [2508.17647].

SurveyGen-I uses a **coarse-to-fine retrieval** pipeline. At survey level, papers are filtered by
\[
\cos(\mathbf{e}_{T,E}, \mathbf{e}_{a_i}) \ge \theta,
\]
with threshold **0.3**, then expanded through citation links and reranked by an LLM; during writing, subsection-specific retrieval can be triggered and merged with the global paper set \(\mathcal{P}^*\) [2508.14317]. SurveyG similarly rejects flat document sets in favor of a **hierarchical citation graph** \(G=(V,E,L)\), where nodes are papers, edges reflect citation and semantic relations, and layers are **Foundation**, **Development**, and **Frontier** [2510.07733]. DeepSurvey goes further by combining **citation-graph expansion with hybrid filtering**, full-text understanding, and repository analysis; its abstract attributes later content-depth and citation gains directly to this evidence-construction strategy [2605.29522].

The common direction is clear: recent SurveyGen systems treat retrieval as a problem of **coverage, structure, and relevance jointly**, not of semantic similarity alone. This suggests that the retrieval problem in ASG is gradually converging toward graph-backed, quality-aware, and full-text-aware evidence selection rather than plain vector search.

## 3. Planning, memory, and writing architectures

Once evidence is assembled, SurveyGen systems usually impose an explicit planning layer. SurveyX divides survey composition into **Preparation** and **Generation** phases, then performs **Outline Optimization** to improve secondary outlines using hints derived from AttributeTrees and a “separate-then-reorganize” procedure that deduplicates subsection headings before drafting [2502.14776]. SurveyForge adopts a more explicitly structure-driven view: it retrieves both topic-relevant research papers and topic-relevant survey outlines, generates first-level and second-level outline nodes together with semantic queries, and uses a memory-driven retrieval module called **Scholar NAvigation Agent (SANA)** to support subsection writing [2503.04629].

SurveyGen-I makes planning dynamic. Its **PlanEvo** module constructs a dependency-aware writing plan \(\mathcal{P}_{\text{dep}}\) and assigns stage indices by
\[
\tau(s) =
\begin{cases}
0 & \text{if } \mathrm{In}(s)=\emptyset,\\
\max_{s' \in \mathrm{In}(s)} \tau(s') + 1 & \text{otherwise},
\end{cases}
\]
so subsections with the same \(\tau(s)\) can be written in parallel, while later stages depend on earlier ones [2508.14317]. Its **memory-guided writing** stores prior subsection content and terminology, then uses that memory both to generate later subsection skeletons and to revise the remaining outline after each stage.

SurveyG also emphasizes structure before prose. It derives **horizontal summaries** from community detection within each graph layer and **vertical path summaries** from weighted traversal across layers, places those summaries into the Writing Agent’s memory, and iteratively refines both outline and subsection drafts through an Evaluation Agent that critiques quality and proposes retrieval queries [2510.07733]. DeepSurvey pushes the same logic further by building what it explicitly calls an **analysis substrate**: paper-level keynotes, cluster-level relation graphs and comparison tables, guided cross-paper QA, and then **multi-granularity refinement** for the final survey [2605.29522].

InteractiveSurvey occupies a different position in the design space. It is not primarily a planner in the same sense as SurveyGen-I or SurveyG; rather, it turns intermediate artifacts into editable UI objects. It creates a **three-level hierarchical outline** from predefined survey sections, user-guided reference categorization, and LLM-generated subsection titles, then generates section text bottom-up from retrieved content while allowing users to revise categorization, outline, text, and visual elements throughout the pipeline [2504.08762].

| System | Core mechanism | Citation |
|---|---|---|
| SurveyX | AttributeTree/attribute forest, outline optimization, RAG-based rewriting | [2502.14776] |
| SurveyForge | Outline exemplars + SANA memory-driven retrieval | [2503.04629] |
| InteractiveSurvey | Editable categorization, editable outline, bottom-up section generation | [2504.08762] |
| SurveyGen-I | Coarse-to-fine retrieval, evolving plans, memory-guided writing | [2508.14317] |
| SurveyG | Hierarchical citation graph + multi-agent writing/validation | [2510.07733] |
| DeepSurvey | Full-text keynotes, cross-paper analysis substrate, evidence-constrained refinement | [2605.29522] |

Across these systems, planning is no longer a thin prompt prefix. It is a persistent intermediate representation: outline trees, dependency schedules, graph summaries, or cluster-level analyses that constrain later writing.

## 4. Benchmarks and evaluation regimes

A large fraction of recent SurveyGen research concerns evaluation rather than generation. SurGE provides one of the clearest benchmark formulations: **205 ground-truth surveys**, **1,086,992** documents in the retrieval corpus, and automatic metrics for information coverage, document/section/sentence-level citation relevance, structural organization, and content quality [2508.15658]. Its most widely cited empirical result is that even with **Recall@100 = 0.3665** for the dense retriever, the best end-to-end generator reaches only **0.0630** reference coverage in the final survey, exposing a large gap between retrieval and usable synthesis [2508.15658].

SurveyBench shifts emphasis toward reader alignment. Built from **11,343 arXiv papers** and **4,947 high-quality surveys**, it evaluates **outline quality**, **content quality**, **non-textual richness**, and **quiz-based answerability** on **20 well-vetted topics** [2510.03120]. Its headline result is that existing LLM4Survey methods are **“on average 21% lower than human in content-based evaluation,”** and that content-only no-reference LLM judging is badly inflated: without human references, all systems score almost perfectly, even above humans on some dimensions [2510.03120]. SurveyEval pursues a related goal with **38 topics across 7 disciplines**, combining reference-guided LLM judging for overall quality, principle-based outline judging, and citation recall/precision/F1 [2512.02763].

SGSimEval argues that survey evaluation should be **multifaceted and similarity-enhanced**. It evaluates **80 highly cited survey papers** and combines intrinsic scores for outline, content, and references with two similarity-weighted variants: **Balanced Similarity Weighting** and **Human-as-Perfect Similarity Weighting** [2508.11310]. Its broad conclusion is that current ASG systems show **human-comparable superiority in outline generation**, but still leave “significant room for improvement in content and reference generation” [2508.11310].

SurveyLens adds a discipline-aware perspective. It introduces **SurveyLens-1k**, a dataset of **1,000 high-quality human-written surveys spanning 10 disciplines**, and a dual-lens evaluation framework combining **Discipline-Aware Rubric Evaluation** with **Canonical Alignment Evaluation** [2602.11238]. On macro-average rubric scores, **ASG systems average 3.02**, **vanilla LLMs 2.43**, and **Deep Research agents 3.41**, with **Gemini Deep Research** reaching **3.66** and best scores in **8 of 10 disciplines** [2602.11238]. The paper’s qualitative conclusion is that structure, content synthesis, and reference quality vary sharply by discipline, so CS-centric evaluation protocols are not sufficient.

These benchmarks collectively establish a recurring empirical picture: outline quality has improved faster than content synthesis and reference quality; human-referenced evaluation is more discriminative than no-reference LLM judging; and survey usefulness is not well captured by fluency or generic long-text metrics alone.

## 5. Human interaction and adjacent survey-system work

Although SurveyGen usually denotes academic survey-paper generation, adjacent survey-system literature addresses related problems of authoring, refinement, and interaction. InteractiveSurvey is the clearest bridge between automatic survey writing and human steering: it supports both **online retrieval and user uploads**, exposes **reference categorization**, **outline**, and **survey content** as editable intermediate artifacts, and exports the final survey as **PDF, Markdown, or LaTeX** [2504.08762]. Its usability study with **34 participants** reports a **System Usability Scale score of 84.4/100**, presented as **A+** [2504.08762].

In questionnaire methodology, "Generative AI as a Safety Net for Survey Question Refinement" studies ChatGPT not as an autonomous questionnaire writer but as a critique engine for item refinement [2509.08702]. The paper uses a preregistered \(2\times3\) factorial experiment over **GPT-3.5/GPT-4.0** and persona prompts, and concludes that an average user can expect useful but imperfect feedback from simple zero-shot prompts, with **GPT-4.0 producing 0.55 more codes on average per question-treatment pair** and the **Survey Design Expert** persona producing **0.22 more codes on average** than no persona [2509.08702]. This work is not about academic survey-paper generation, but it is methodologically adjacent because it treats survey improvement as structured critique rather than free-form rewriting.

TASSY extends survey infrastructure in a different direction by combining **text annotation** and **conventional survey questions** in one progressive web application, implemented with **Flask**, **Vue.js**, and **MySQL/MariaDB or SQLite** [2112.07391]. SurveyMan goes even further toward a programming-language view: it models surveys as programs with logic, control flow, randomization, and bugs, provides a DSL for authoring, compiles surveys into JavaScript, and performs static and dynamic analyses to detect wording bias, order effects, breakoff, and inattentive respondents [1406.5572]. The Shiny-based randomized-response system similarly shows how specialized survey designs can be embedded into a deployable interface with hidden randomization and local or remote persistence [2110.14198].

A final adjacent line is **Survey Response Generation**, which studies how LLMs simulate closed-ended survey answers in silico. That work evaluates **32 million simulated survey responses** across **8 Survey Response Generation Methods**, **4 political attitude surveys**, and **10 open-weight language models**, and concludes that **Restricted Generation Methods perform best overall**, while reasoning output does not consistently improve alignment [2510.11586]. This is not survey-paper generation, but it reinforces a broader lesson: in survey-related LLM systems, output constraints and evaluation design are often as important as the base model.

## 6. Limitations, failure modes, and research directions

The recent SurveyGen literature is unusually consistent about its failure modes. SurveyBench finds that generated surveys can score above 4/5 on content quality and still perform very poorly on topic-specific quiz answering, indicating shallow synthesis and insufficient detail despite polished prose [2510.03120]. SurveyLens shows that reference quality remains weak across paradigms and that systems often exhibit a structural-versus-synthesis trade-off, with specialized ASG systems strong on outline quality but weaker on discipline-specific content expectations outside their native domains [2602.11238]. SGSimEval reaches a similar diagnosis: current systems are strong at structure, but content and especially references remain unresolved [2508.11310].

The "SurveyGen" dataset paper draws the same conclusion from a retrieval perspective. Even when quality-aware retrieval improves citation overlap and key-point recall, the authors state that **fully automatic survey generation still suffers from low citation quality and limited critical analysis**, while **human-guided writing**—providing human-selected references and human-written outline—produces substantially better content [2508.17647]. This suggests that, at present, SurveyGen is strongest as an assistive writing technology rather than a drop-in replacement for expert survey authorship.

The main architectural response to these limits is visible in DeepSurvey. Its abstract claims the **highest content score (8.644/10)**, **12.3% and 9.3% recall and precision gains over the strongest baseline** in citation quality, stronger domain robustness measured by **0.14 vs 0.22 to 0.69 CS-to-non-CS drop**, and domain-expert preference over human-written surveys on **83.3% overall quality** and **100% content depth** [2605.29522]. Whether those gains generalize broadly remains to be established, but they indicate where current research is heading: full-text evidence rather than abstract-only RAG, cross-paper relation modeling rather than isolated paper summaries, evidence-constrained citation assignment rather than post-hoc grounding, and agentic refinement rather than one-shot drafting.

A plausible implication is that SurveyGen is becoming less a single model and more a **systems problem**. The field’s direction is toward pipelines that integrate retrieval graphs, structured intermediate representations, editable planning states, and evaluation regimes tied to human-written surveys and reader-oriented tasks. The persistent gaps in citation faithfulness, analytical depth, and discipline-aware organization suggest that these systems are moving away from “write a survey from a prompt” and toward “construct, validate, and refine a survey-specific knowledge representation before writing.”

Source: https://www.emergentmind.com/topics/surveygen