From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

Abstract: The rapid expansion of the model context protocol (MCP) ecosystem enables LLM-based agents to access a wide range of external tools via a standardized interface. However, identifying appropriate MCP servers for a specific development task remains challenging. Existing studies primarily focus on measuring the MCP ecosystem or optimizing tool invocation mechanisms, while systematic recommendation frameworks and reproducible benchmarks for real-world development tasks remain largely unexplored. To address this limitation, we formulate task-oriented MCP server recommendation as a structured retrieval-and-ranking problem that jointly considers semantic relevance and engineering constraints. We first construct Task2MCP, a task-centered dataset that systematically associates taxonomy-grounded development tasks with curated MCP servers. This dataset provides structured supervision and a reproducible evaluation environment for research on MCP tool recommendations. Building on this dataset, we propose T2MRec, a task-to-MCP server recommendation model. It models semantic relevance and structural compatibility to construct an initial candidate set. Then it improves coverage and ranking quality through centroid-based candidate expansion and constrained LLM-based re-ranking. In addition, we design and implement an interactive MCP server recommendation agent prototype that operates in conversational environments to support dynamic decision-making. The agent assists developers in efficiently evaluating and integrating tools by providing recommended MCP servers together with usage guidelines.

• The paper introduces T2MRec, a multi-stage retrieval and ranking framework that aligns developer tasks with MCP servers.
• It leverages a dual-tower contrastive learning approach combined with structural constraints, achieving Recall@10 of 0.6702.
• The integration of an interactive agent offers transparent, human-guided tool selection with actionable recommendations.

Task-Aware Model Context Protocol (MCP) Server Recommendation with T2MRec: A Comprehensive Analysis

Introduction and Problem Motivation

As LLM-based agent systems become pervasive in software engineering, robust tool selection remains a fundamental bottleneck. The Model Context Protocol (MCP) defines a standardized interface for invoking heterogeneous tools, leading to an explosion in available MCP servers. However, precise alignment between developer tasks and MCP server capabilities is confounded by fragmented ecosystem metadata, superficial semantic matching, and opaque recommendation rationales. Most prior work disregards operational feasibility during recommendation and fails to provide systematic, reproducible task-to-MCP server mappings, limiting practical deployment and measurable progress.

Task2MCP Dataset: Taxonomy-Grounded Benchmark Construction

To address these limitations, the paper proposes Task2MCP, a rigorously curated benchmark linking 4,800 realistic taxonomy-grounded development tasks to 5,642 MCP servers. The dataset construction pipeline integrates multi-source MCP directory crawling, GitHub-centric metadata enrichment, and strict deduplication/normalization incorporating repository-level and system-level features. The task dataset is structured atop a two-level taxonomy based on the NIST Human–AI activity framework, providing systematic coverage across categories/domains and facilitating detailed analysis. LLMs assist with task generation, but all final samples undergo human validation with high inter-annotator reliability.

Figure 1: Overview of the construction pipeline of the Task2MCP dataset, including MCP set construction, task set construction, and task-MCP interaction set construction.

The task taxonomy's two-tier organization is visually detailed, underpinning robust categorization spanning intent and capability.

Figure 2: Two-Level task taxonomy based on the NIST human–AI activity framework.

The schema diagram exposes rich metadata: each task/MCP server is accompanied by detailed categorical, programming, and system attributes. The dataset’s structure enables explicit evaluation of coverage, cold start, and constraint satisfaction.

Figure 3: Schema and illustrative example of the Task2MCP dataset.

The T2MRec Model: Design, Scoring, and Retrival-Refinement Pipeline

Model Formulation: The recommendation task is formalized as top-$K$ retrieval of MCP servers given a task, leveraging both fine-grained semantic signals and explicit structured constraints. T2MRec employs a multi-stage retrieval-and-ranking pipeline.

1. Semantic Relevance Scoring: Input representations derive from concatenated task/server text fields (including tool lists and technical metadata). A dual-tower encoder (MLP-based) is trained with a temperature-scaled contrastive objective on task–MCP interaction labels. This formulation enables fast inference via offline MCP embedding indexation and robust alignment despite heterogeneous semantic modalities.
2. Structural Compatibility Scoring: Lightweight yet effective taxonomy-, language-, and system constraint signals are integrated as normalized and binary features, distinguishing between mere intent match and operational feasibility.
3. Candidate Refinement and LLM Constrained Re-ranking: To counteract recall losses due to sparse/ambiguous documentation and to improve ranking granularity, a centroid-based neighborhood expansion increases candidate pool diversity. A final, constrained LLM re-ranking module (using deterministic prompting and strict output control) imposes human-aligned, interpretable orderings with transparent, evidence-grounded rationales.

Figure 4: Architecture of the T2MRec model. The model progressively narrows the search space of MCP servers through semantic, structural, and candidate refinement modules.

This architectural separation ensures the bulk of scoring is efficient, relegating expensive reasoning to a small, relevant subset, and greatly enhances inference scalability.

Experimental Evaluation

Baselines and Metrics

The evaluation contrasts T2MRec with traditional (NeuMF, MSBPR, ReCAFR), graph-based (NGCF, LightGCN, GDSRec, CVH-REC), and LLM-based (TiGER, OpenP5, LC-Rec, MPT-Rec) recommenders. Standard top-$k$ ranking, precision, recall, F1, and NDCG are reported.

Results and Analysis

T2MRec achieves highest Recall@10 (0.6702), NDCG@10 (0.6633), and F1@10 (0.4973), surpassing all baselines. Notably:

• It dominates LLM-based retrieval methods, which, despite strong task language modeling, fail to enforce operational constraints and yield less stable results.
• Graph-based methods offer marginal improvements over interaction-only approaches but suffer from label and metadata sparsity.
• The structural compatibility component, while auxiliary (optimal weight 0.1), increases constraint awareness and reduces spurious recommendations.

Figure 5: Effect of the two-tower encoder on task-to-MCP alignment in the shared embedding space using violin plots. The dual-tower encoder significantly improves discriminability (Wilcoxon signed-rank tests, $p < 0.001$).

Ablation studies confirm the essentiality of contrastive training (removal yields a 32% NDCG drop), dual-tower encoding, and lexical representations. The LLM re-ranking module is less impactful than the first two but contributes to top-of-list ranking refinement.

Extensive parameter sweeps show T2MRec is robust; performance degrades smoothly outside optimal semantic/structural weighting, temperature, and learning rate settings.

Figure 6: Effect of different semantics–structure weights.

Figure 7: Effect of different temperature values in the contrastive objective.

Figure 8: Effect of different learning rates.

Interactive Recommender Agent: Practical System Integration

The paper further instantiates T2MRec as a retrieval-centered, multi-layer conversational agent (T2MAgent) that operationalizes structured task parsing, candidate retrieval, evidence-grounded LLM explanation generation, and interactive refinement. The architecture enforces strict separation between retrieval and NLG, improving both controllability and session traceability.

Figure 9: Overall system architecture of T2MAgent for retrieval-centered MCP server recommendation.

The interactive UI provides users with actionable recommendations, explicit justification, and metadata provenance, facilitating human-in-the-loop development and iterative clarifications.

Figure 10: Interactive interface of the T2MAgent agent showing MCP server recommendations and evidence-grounded explanations.

A sample case—"generate marketing-oriented captions for YouTube videos"—demonstrates context-sensitive multi-tool composition, system constraint management, and user-driven preference refinement.

Figure 11: Illustrative task used in the case study: generating marketing-oriented captions for YouTube videos.

Implications and Future Directions

Practical Implications: T2MRec materializes a scalable and robust paradigm for tool recommendation in dynamic agent ecosystems. By explicitly decomposing semantic, structural, and reasoning components, it sets a new empirical standard for measurable, reproducible progress in LLM toolchain selection. The design is modular, extensible, and amenable to integration with real-world developer workflows.

Theoretical Implications: The work demonstrates that contrastive learning on dual-tower architectures is effective for bridging the task-capability semantic gap, provided that constraint signals are not neglected. The candidate refinement pipeline underscores the necessity of hybrid retrieval-NLU strategies in high-noise retrieval/selection environments.

Future Research: The authors delineate three major axes for extension: (1) Expansion to proprietary/closed MCP ecosystems and continual benchmark augmentation; (2) Generalization to tool-composition, multi-step and feedback-driven workflows; (3) Tighter coupling of recommendation and tool execution feedback for looped agentic pipelines.

Conclusion

This paper establishes a rigorous, reproducible foundation for task-aware recommendation of MCP servers with the Task2MCP dataset and the T2MRec model. The joint modeling of semantic and structural signals, together with late-stage LLM re-ranking and interactive system design, leads to quantifiably superior retrieval and ranking performance over all studied baselines. The implications for LLM-based agent tool selection are direct: robust retrieval requires both deep task/tool understanding and explicit constraint modeling, operationalized in scalable, modular architectures. Future directions entail both dataset expansion and further integration of real-world agent execution feedback for continual refinement.

---

Reference:

"From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation" (2604.17234)