VCBench: Benchmarking LLMs in Venture Capital (2509.14448v1)

Abstract: Benchmarks such as SWE-bench and ARC-AGI demonstrate how shared datasets accelerate progress toward artificial general intelligence (AGI). We introduce VCBench, the first benchmark for predicting founder success in venture capital (VC), a domain where signals are sparse, outcomes are uncertain, and even top investors perform modestly. At inception, the market index achieves a precision of 1.9%. Y Combinator outperforms the index by a factor of 1.7x, while tier-1 firms are 2.9x better. VCBench provides 9,000 anonymized founder profiles, standardized to preserve predictive features while resisting identity leakage, with adversarial tests showing more than 90% reduction in re-identification risk. We evaluate nine state-of-the-art LLMs. DeepSeek-V3 delivers over six times the baseline precision, GPT-4o achieves the highest F0.5, and most models surpass human benchmarks. Designed as a public and evolving resource available at vcbench.com, VCBench establishes a community-driven standard for reproducible and privacy-preserving evaluation of AGI in early-stage venture forecasting.

• The paper presents VCBench, a standardized benchmark using 9,000 anonymized founder profiles to evaluate early-stage venture capital forecasting performance.
• It applies a multi-stage data cleaning and anonymization pipeline that achieves a 92% reduction in re-identification risk while preserving predictive signal.
• Benchmark experiments reveal that top LLMs, such as GPT-4o, outperform human baselines by achieving significant improvements in precision and recall trade-offs in VC forecasting.

VCBench: A Standardized Benchmark for LLMs in Venture Capital Forecasting

Introduction and Motivation

VCBench introduces a rigorous, privacy-preserving benchmark for evaluating LLMs in the context of early-stage venture capital (VC) forecasting. The benchmark addresses a critical gap: while prior datasets have focused on perception, reasoning, or medical diagnosis, VCBench targets decision-making under extreme uncertainty, where even top human experts perform modestly. The domain is characterized by sparse signals, noisy data, and rare positive outcomes, making it an ideal testbed for measuring progress toward human-level and superhuman forecasting capabilities.

Dataset Construction and Anonymization Pipeline

VCBench comprises 9,000 anonymized founder profiles, with 810 labeled as successful based on stringent criteria: acquisition or IPO above \$500M, or fundraising exceeding \$500M. The dataset is statistically representative of the U.S. startup landscape from 2010–2018, covering approximately 20% of the relevant population.

The data cleaning and anonymization pipeline is a multi-stage process:

• Coverage Improvement: Cross-referencing LinkedIn and Crunchbase to fill missing fields and enforce cross-record consistency.
• Format Standardization: Deterministic canonicalization of degree and role variants, followed by LLM-assisted flagging and exclusion of non-formal entries.
• Entry- and Dataset-Level Anonymization: Removal of all direct identifiers (names, company names, locations, dates), clustering of industries via embedding and hierarchical methods, and bucketing of rare values to prevent linkage attacks.
• Iterative Adversarial Testing: Each anonymization step is validated by adversarial re-identification experiments using both offline and online LLMs, with changes retained only if they reduce leakage while preserving predictive signal.

  Figure 1: The data cleaning pipeline integrates coverage improvement, standardization, filtering, and anonymization to produce robust, privacy-preserving founder profiles.

The final format achieves a 92% reduction in re-identification risk for successful founders, as measured by explicit adversarial unit tests. Notably, the inclusion of QS university rankings (bucketed and unbucketed) further reduces identification rates, as models misapply current rankings, enhancing anonymity without sacrificing educational prestige as a predictive feature.

Dataset Characteristics and Representativeness

VCBench profiles encode structured fields: binary success label, industry, prior IPO/acquisition experience, education records (degree, field, QS ranking), and job histories (role, company size, industry, duration). The dataset is distributed in both anonymized prose (for LLMs) and structured JSON (for custom ML models).

Industry and founding year distributions are visualized below, confirming broad coverage across sectors and temporal cohorts.

Figure 2: Distribution of industries in VCBench after clustering and bucketing.

Figure 3: Distribution of startup founding years in VCBench, reflecting the eight-year outcome horizon.

Experimental Evaluation and Leaderboard Results

Nine state-of-the-art LLMs were evaluated on VCBench using six-fold cross-validation, with the $F_{0.5}$ metric prioritizing precision over recall—reflecting the high cost of false positives in VC. The benchmarked models include GPT-4o, DeepSeek-V3/R1, Gemini-2.5-Pro/Flash, Claude-3.5-Haiku, GPT-5, o3, and GPT-4o-mini.

Figure 4: Predictive performances of nine vanilla LLMs on VCBench, compared to human-level baselines and market indices.

Key findings:

• GPT-4o achieves the highest $F_{0.5}$ (25.1), with a precision of 29.1%, representing a 3.2$\times$ improvement over the market index and exceeding the 2.9$\times$ performance of tier-1 VC firms.
• DeepSeek-V3 delivers the highest precision (59.1%) but with low recall, indicating strong selectivity but limited coverage.
• Gemini-2.5-Flash attains the highest recall (69.1%) at the expense of precision.
• Most LLMs surpass human baselines, demonstrating that anonymized founder profiles retain sufficient signal for superhuman performance in early-stage VC forecasting.

Cost and latency analyses reveal that GPT-4o-mini and DeepSeek-V3 offer favorable trade-offs between performance and inference efficiency, supporting scalable deployment scenarios.

Benchmark Validity, Limitations, and Design Trade-offs

VCBench's interpretation is bounded by several factors:

• Prevalence Shift: The benchmark's 9% success rate is inflated relative to the real-world 1.9%, stabilizing evaluation but complicating extrapolation of precision multipliers.
• Human Baseline Comparability: Structural differences in deal flow and access between VCs and LLMs may distort direct comparisons.
• Data Biases: LinkedIn and Crunchbase coverage favors technology sectors and public founders, potentially limiting generalizability.
• Temporal Bias: The eight-year outcome horizon introduces right-censoring, penalizing more recent startups.
• Residual Irregularities: Despite rigorous cleaning, large-scale founder data remains noisy and heterogeneous.

Mitigation strategies include releasing only half the dataset publicly to prevent future pre-training leakage, reserving the remainder for private leaderboard evaluation.

Implications and Future Directions

VCBench establishes a reproducible, privacy-preserving standard for evaluating AGI-level decision-making in high-stakes, real-world domains. The results indicate that LLMs, when properly benchmarked and anonymized, can outperform human experts in early-stage venture forecasting. This has significant implications for the deployment of AI in investment decision support, founder evaluation, and resource allocation under uncertainty.

Future work should focus on:

• Community-driven refinement: Iterative updates to improve coverage and reduce residual noise.
• Enhanced anonymization: Incorporation of prestige proxies and scalable clustering for high-cardinality fields.
• Advanced feature engineering: Trajectory-level features and temporal patterns to strengthen predictive accuracy while maintaining privacy.
• Expanded evaluation modes: Simulation-based tournaments and human–AI competitions to capture sequential decision-making dynamics.

Conclusion

VCBench provides the first standardized, anonymized benchmark for founder-success prediction in venture capital, validated by adversarial testing and robust against identity leakage. The benchmark demonstrates that LLMs can surpass human-expert baselines in this domain, offering a foundation for reproducible research and future advances in AI-driven decision-making under uncertainty. The public leaderboard and evolving dataset invite ongoing community participation, ensuring that VCBench remains a relevant and rigorous testbed for both academic and applied research in AI for venture capital.