LSPFuzz for Language Server Testing

• LSPFuzz is a grey-box hybrid fuzzer designed to systematically test Language Server Protocol servers by integrating source code mutations with simulated editor interactions.
• It employs a two-stage mutation pipeline using syntax-aware changes and context-aware dispatch of LSP operations to probe reliability and security.
• Evaluations show LSPFuzz boosts code coverage up to 15.1x over baselines and uncovers critical bugs, including vulnerabilities with assigned CVEs.

Searching arXiv for the LSPFuzz paper and closely related work to ground the article in current arXiv records. LSPFuzz is a grey-box hybrid fuzzer specifically designed to test Language Server Protocol (LSP) servers by systematically generating and mutating both the source code being analyzed and the editor operations performed on that code (Zhu et al., 1 Oct 2025). Its central premise is that effective LSP server testing cannot be reduced to arbitrary JSON-RPC message generation or ordinary source-code fuzzing in isolation, because many bugs arise only from the interaction between a particular code pattern and a particular operation at a specific location. In the reported evaluation on four widely used LSP servers, LSPFuzz demonstrated superior performance compared to baseline fuzzers and uncovered previously unknown bugs in real-world LSP servers, including vulnerabilities assigned CVE numbers (Zhu et al., 1 Oct 2025).

1. Definition, scope, and problem setting

The Language Server Protocol is a JSON-RPC–based protocol that standardizes how editors communicate with language analysis tools such as hover, go to definition, completion, rename, formatting, document symbols, and diagnostics. The typical interaction starts with the editor launching the LSP server, sending textDocument/didOpen with the content of a source file, and then issuing requests as the developer edits or navigates code. There are $>300$ servers and $\sim 50$ editors supporting LSP, and servers such as clangd, sorbet, solc, and verible are widely deployed (Zhu et al., 1 Oct 2025).

LSPFuzz targets both reliability and security. If an LSP server crashes, all code-intelligence features disappear, interrupting developer workflow. If the server contains memory-safety bugs such as buffer overflows or use-after-free, untrusted source code can potentially trigger remote code execution or related vulnerabilities in the developer’s environment. The reported motivation is not merely theoretical: clangd alone has $>500$ crash issues, two security issues found by LSPFuzz were assigned CVEs, and LLVM changed its VSCode extension behavior to disable clangd in untrusted workspaces after the findings (Zhu et al., 1 Oct 2025).

The authors characterize LSPFuzz as the first systematic approach specifically targeting LSP server reliability and security. That framing distinguishes it from general binary fuzzers, grammar-based protocol fuzzers, compiler fuzzers, and other multi-dimensional fuzzers that do not encode the tight coupling between program text and editor actions required by LSP semantics (Zhu et al., 1 Oct 2025).

2. Input model and architectural organization

LSPFuzz adopts a grey-box hybrid architecture. It is grey-box because it uses coverage feedback from instrumented LSP servers to guide search, and hybrid because it combines structured source-code generation and mutation with coverage-guided fuzzing techniques for seed selection and prioritization (Zhu et al., 1 Oct 2025).

Initialization begins by building a code fragment pool from real-world code, specifically examples and tests from the target LSP server repository, with optional user-supplied files. Tree-Sitter grammars are then used to generate initial source-code seeds, which are parsed into ASTs or parse trees. During fuzzing, LibAFL’s scheduler selects a seed from the corpus, after which LSPFuzz applies a two-stage mutation pipeline that yields a full LSP session input: a document-opening action containing the mutated code, followed by a sequence of LSP requests whose parameters refer to locations or ranges in that specific document (Zhu et al., 1 Oct 2025).

Execution uses an instrumented LSP server and an in-memory harness. The server processes a sequence of JSON-RPC messages written into a shared-memory buffer, while instrumentation collects control-flow edges and AddressSanitizer detects memory errors. If an execution reaches a new edge, the input is added to the corpus. If it crashes, the stack trace is hashed for deduplication, and crashes at distinct program locations are treated as bug candidates (Zhu et al., 1 Oct 2025).

This architecture reflects a session-level view of LSP testing. The input is not a single message but a stateful interaction comprising document content, protocol sequencing, positions, and dependent responses. A plausible implication is that the unit of fuzzing is closer to an editor session than to a conventional file or packet.

3. Holistic mutation and the two-stage pipeline

The key technical insight of LSPFuzz is holistic mutation: source code and editor operations are mutated in a single coordinated pipeline rather than independently. The source document is first mutated, and the operation sequence is then generated with explicit awareness of that mutated document. The evaluation reports that removing this coordination substantially reduces coverage and largely eliminates operation-triggered crashes in three of the four targets (Zhu et al., 1 Oct 2025).

Stage I performs syntax-aware source-code mutation. Using Tree-Sitter, LSPFuzz parses source code into a derivation tree, randomly selects a non-terminal node, and replaces it with a newly generated subtree rooted at the same non-terminal type. Subtree generation randomly selects production rules and recursively expands child non-terminals. When compatible, LSPFuzz may splice in a subtree from the real-world code fragment pool, with a hyperparameter whose default value is $0.2$ controlling the probability of using such fragments (Zhu et al., 1 Oct 2025).

Stage I also injects invalid code in a controlled manner, motivated by the fact that LSP servers must handle incomplete or malformed programs during ordinary editing. The reported invalid-code operators are Drop Required Nodes, Node Transplantation, and Terminal Truncation. These operators are designed to produce code that is locally broken but globally recognizable, thereby exercising partial parsing, error recovery, and related analysis paths rather than merely destroying input structure (Zhu et al., 1 Oct 2025).

Stage II performs context-aware dispatching of editor operations such as textDocument/hover, textDocument/definition, textDocument/implementation, textDocument/formatting, textDocument/documentSymbol, workspace/symbol, and call-hierarchy requests. Instead of selecting arbitrary character positions, LSPFuzz computes syntactic signatures for AST nodes. A syntactic signature of level $n$ is defined as

$\langle t_0, t_1, \ldots, t_n \rangle$

where $t_0$ is the node type and the remaining elements are ancestor types. Positions are grouped by signature, and target selection randomizes over signature groups before choosing a position within a group. This shifts randomness from character positions to syntactic categories, helping rare but semantically important constructs receive non-negligible attention (Zhu et al., 1 Oct 2025).

Stage II is further enriched by runtime information from the server. LSPFuzz sends symbol-related requests and prioritizes semantic symbols returned by textDocument/documentSymbol and workspace/symbol. It also captures textDocument/publishDiagnostics and prioritizes nodes that contain or are near diagnostics. Finally, it caches responses so that operations with dependencies can reuse prior results, as in textDocument/prepareCallHierarchy followed by callHierarchy/outgoingCalls using returned CallHierarchyItem values (Zhu et al., 1 Oct 2025).

4. Guidance mechanisms, execution semantics, and testing assumptions

Coverage feedback is the primary search signal. LSP servers are instrumented with AFL++ compatible LLVM LTO mode, and LSPFuzz tracks executed control-flow edges for each test case. Inputs that hit previously unseen edges are considered interesting and retained in the corpus. Seed prioritization is delegated to LibAFL’s scheduler, which favors seeds that hit rare edges or lead to crashes (Zhu et al., 1 Oct 2025).

Crash detection combines ordinary process-failure signals with AddressSanitizer. The reported anomalies include segmentation faults, assertion failures, aborts, buffer overflows, and use-after-free. Deduplication proceeds first by stack hash and then by program location, using file-plus-line or instruction location to avoid overcounting the same underlying bug. For bug reporting, test cases are manually minimized (Zhu et al., 1 Oct 2025).

The current oracle is intentionally narrow. LSPFuzz detects crashes and memory errors rather than functional correctness deviations. The authors explicitly identify richer functional oracles—such as consistency of go-to-definition or reference results—as future work. This matters because LSP behavior includes many semantically observable failures that do not crash the server (Zhu et al., 1 Oct 2025).

A recurrent misconception addressed by the design is that LSP can be fuzzed effectively either as plain bytes or as ordinary JSON constrained by a schema. Binary fuzzers such as AFL++ tend to remain in parsing libraries, while grammar-based fuzzers can generate syntactically valid messages but still fail to satisfy the semantic constraints linking code contents, positions, and protocol state. Likewise, generic two-dimensional fuzzers may generate syntactically valid but semantically meaningless combinations, such as hovering at a non-existent location or issuing requests before didOpen (Zhu et al., 1 Oct 2025).

5. Evaluation, baselines, and empirical findings

The evaluation covered four popular LSP servers, all implemented in C/C++ and compiled with AddressSanitizer and AFL++ compatible LLVM LTO instrumentation (Zhu et al., 1 Oct 2025).

Name	Vendor	Language / Version
clangd	LLVM	C/C++, v20.1.4
sorbet	Stripe	Ruby, v0.5.11031
verible	CHIPS Alliance	Verilog, v0.0.3157
solc	Ethereum	Solidity, v0.8.29

At the time of writing, the associated VSCode extensions had 1.8M installs for clangd, 901k for sorbet, 1.1M for verible, and 1.6M for solc. The common setup used dual AMD EPYC 7773X hardware with 128 cores and 1 TB RAM, AlmaLinux 9.5, one core per fuzzing run, 24 hours per run, and 10 repetitions per configuration, totaling 200 CPU-days. CLI parsing and configuration-loading code were removed to improve throughput (Zhu et al., 1 Oct 2025).

Three LibAFL-based baselines were used: a binary baseline based on standard AFL-style mutation, a grammar baseline derived from LSP message schemas, and a two-dimensional baseline using MultipartInput with one part for source code and one for a sequence of operations. The evaluation also included a non-holistic variant, denoted LspFuzz*, which mutates code and operations independently (Zhu et al., 1 Oct 2025).

Average edge coverage after 24 hours was reported as approximately 348,741 for clangd, 83,151 for sorbet, 29,133 for verible, and 53,720 for solc. The corresponding best baseline percentages relative to LSPFuzz ranged from 20.2% to 46.4%, while the binary baseline reached as little as 0.7% of LSPFuzz’s coverage on clangd. Summarizing across targets, LSPFuzz improved coverage over the best baseline by factors ranging from $2.2\times$ to $15.1\times$, and over binary fuzzing by up to $142.9\times$ (Zhu et al., 1 Oct 2025).

Crash results were reported in two forms. Across all four servers and ten runs each, LSPFuzz produced 991 crashes with distinct stack traces, averaging 181.1 per run, and approximately 44% were operation-triggered. The binary baseline found 4 crashes total, all in JSON libraries; the grammar baseline found 12 crashes, all code-loading; the two-dimensional baseline found 137 crashes, all code-loading; and LspFuzz* found 734 crashes, with no operation-triggered crashes in three of the four servers. When deduplicated by unique program location, LSPFuzz reached 115 distinct crash locations, compared with 4 for the binary baseline, 11 for the grammar baseline, 16 for the two-dimensional baseline, and 62 for LspFuzz* (Zhu et al., 1 Oct 2025).

The reported bug-discovery outcome was 51 bugs at unique locations from the first experiment run, of which 42 were confirmed by developers, 26 were fixed, and 2 were assigned CVEs. Two illustrative cases were highlighted: a clangd formatting crash caused by invalid decltype combined with textDocument/formatting, and a sorbet go-to-implementation crash caused by a malformed lambda combined with textDocument/implementation targeting the arrow token. The former had existed for approximately two years; the latter for approximately four years (Zhu et al., 1 Oct 2025).

6. Security implications, limitations, and research context

The security significance of LSPFuzz follows from the threat model in which the attacker controls only the source code contents opened by the developer. Under that model, crashes degrade editor functionality, while memory-safety bugs in the analysis engine can create a path toward arbitrary code execution in the editor environment. The LLVM response cited in the evaluation explicitly acknowledges that parsing untrusted code through clang or clangd can be harmful and led downstream tooling to disable clangd in untrusted VSCode workspaces (Zhu et al., 1 Oct 2025).

The limitations are equally explicit. Only four servers were evaluated, though they were chosen to be popular and diverse. Fuzzing remains stochastic despite ten repetitions per configuration. The current system focuses on single-file, static content rather than multi-file workspaces or evolving edit sequences. The oracle is restricted to crashes and memory errors, and the mutation strategy is largely syntactic rather than deeply semantic with respect to control-flow, data-flow, or cross-file relations (Zhu et al., 1 Oct 2025).

In relation to adjacent research areas, LSPFuzz extends grammar-based fuzzing by operating on source-language grammars via Tree-Sitter rather than only on LSP message schemas, and by embedding LSP-specific constraints procedurally in the mutation logic. It differs from traditional protocol fuzzers because the semantic content of the payload—the source code—is itself a primary fuzzing dimension. It also differs from compiler and static-analysis fuzzers such as CSmith, LangFuzz, Nautilus, and GrayC because it targets interactive request sequences rather than standalone program analysis. The comparison to multi-dimensional fuzzers such as Falcon, KextFuzz, and DistFuzz is similarly constrained by LSP’s location-sensitive coupling between code and operations. The authors also report that an attempt to use ChatAFL with LSP failed when LSP was used as the protocol, and they identify LLM integration as future work rather than a present capability (Zhu et al., 1 Oct 2025).

The reported future directions are multi-file and evolving-source scenarios, richer LSP-specific test oracles, semantic-aware source-code mutation, and possible LLM integration for realistic editing sequences and more complex transformations. LSPFuzz itself is implemented in 12,293 lines of Rust atop LibAFL, and both the tool and experimental data are open-sourced via Zenodo. Developers reportedly expressed interest in integrating it into CI and testing workflows, which suggests that its contribution is not only methodological but also infrastructural for subsequent LSP assurance research (Zhu et al., 1 Oct 2025).

A broader methodological implication is that LSPFuzz belongs to a family of fuzzing systems that treat testing as coordinated exploration over coupled input dimensions rather than mutation over a single flat byte stream. Related arXiv work on program repair formalizes a different but structurally analogous co-exploration of patch and input spaces (Zhang et al., 2023). This suggests that LSPFuzz’s main conceptual contribution lies not merely in domain specialization for language servers, but in demonstrating that coverage-guided fuzzing can be made effective when the search space is defined by cross-dependent structured artifacts—here, source text and editor actions—whose interactions determine reachability and failure.