---
title: 'Orcheo: Modular Conversational Search'
url: https://www.emergentmind.com/topics/orcheo
type: topic
---

# Orcheo: Modular Conversational Search

Searching arXiv for recent papers on "Orcheo" and related uses to ground the encyclopedia entry.
Orcheo is an open-source platform for conversational search (CS) that represents an end-to-end system as a directed graph of interchangeable, single-file Python modules called “nodes.” It is intended to address two barriers identified in CS research: the lack of a unified framework for efficiently sharing contributions with the community, and the difficulty of deploying end-to-end prototypes needed for user evaluation. Its design emphasizes component reuse and reproducibility, production-ready infrastructure spanning local and remote execution, and starter-kit assets with 50+ off-the-shelf components for query understanding, ranking, and response generation. The platform is released under the MIT License at `https://github.com/ShaojieJiang/orcheo` [2602.14710].

## 1. Research context and system goals

Conversational search requires a software pipeline that integrates query reformulation, ranking, and response generation. Orcheo is positioned not as a single retrieval or generation model, but as a framework for composing, deploying, and evaluating such pipelines. The platform’s stated advantages are threefold: a modular architecture that promotes component reuse through single-file node modules, production-ready infrastructure that bridges the prototype-to-system gap via dual execution modes, secure credential management, and execution telemetry, and starter-kit assets that enable rapid bootstrapping of complete CS pipelines [2602.14710].

This positioning is significant for CS methodology. A workflow in Orcheo can be inspected at the level of individual modules, executed locally for rapid iteration, or moved to a remote backend without code changes. This suggests a deliberate attempt to align algorithmic experimentation, software engineering, and user-facing deployment within a single framework.

## 2. Node hierarchy, workflow graph, and state model

Every conversational search pipeline in Orcheo is expressed as a directed graph of nodes. The core abstraction is defined in `orcheo.nodes.base`, where three abstract base classes organize the system: `BaseNode`, `AINode`, and `TaskNode` [2602.14710].

| Class | Inheritance | Role |
|---|---|---|
| `BaseNode` | base class | Common services such as credential resolution via `[[…]]`, variable interpolation via `{…}`, and tool registration |
| `AINode` | inherits `BaseNode` | Wrappers for LLM calls, returning LangChain-style `ChatMessage` lists |
| `TaskNode` | inherits `BaseNode` | Data-processing or retrieval components, returning typed dicts merged into `State` |

Concrete examples include `QueryRewriteNode`, `DenseSearchNode`, `BM25TaskNode`, `ReRankerNode`, `GroundedGeneratorNode`, and `RetrievalEvaluationNode`. The node hierarchy described in the paper places `LLMNode` and `AgentNode` under `AINode`, and `DenseSearchNode`, `ReRankerNode`, and `RetrievalEvaluationNode` under `TaskNode`.

A workflow run carries a single `State` object, described as a subclass of LangGraph’s `MessagesState`. This object holds inputs such as the user query, accumulated outputs keyed by node name, conversation history, and runtime configuration including prompt templates and model names. The dataflow supports three edge types: sequential edges such as `A→B`, conditional edges selected on a state field, and parallel edges that fork execution and later aggregate results. Variable interpolation allows a node configuration to refer to upstream outputs, for example `retriever_index: "{dense_search.index_name}"`.

At execution time, LangGraph compiles the workflow into a `StateGraph` that handles async scheduling, checkpointing, streaming results back to clients, and instrumentation via OpenTelemetry. The practical consequence is that workflow semantics are explicit at the graph level while runtime services remain embedded in the execution substrate rather than reimplemented per pipeline.

## 3. Execution modes, infrastructure, and observability

Orcheo provides dual execution modes. In local mode, the `orcheo` CLI and Python SDK (`OrcheoClient`) allow workflows to be run on a laptop with inspection of intermediate state. In remote mode, a backend composed of FastAPI, WebSocket services, Celery workers, LangGraph runtime, PostgreSQL or SQLite, Redis, and an encrypted Credential Vault runs in the cloud or on premises. Switching between these modes requires no code changes and depends only on client URL and credentials [2602.14710].

Credential handling is integrated into the platform. The Credential Vault is an AES-256 encrypted store for API keys, OAuth tokens, and database passwords, and nodes reference secrets at runtime using syntactic forms such as `[[vault_name#field]]` or `[[api_key]]`. This matters because many CS pipelines simultaneously depend on search indexes, model APIs, analytics stores, and external tools, and the platform treats credential resolution as a first-class runtime service.

Observability is similarly built in. OpenTelemetry tracing is attached to every node execution and can be exported to Jaeger or Datadog or viewed in the built-in tracing viewer. Each trace records node-level start and end timestamps, input and output summaries, and any errors. The paper also notes LangSmith integration for logging prompt inputs and outputs and for viewing LLM usage breakdowns. Together, these mechanisms make runtime diagnosis and experiment auditing part of the default workflow rather than optional add-ons.

A distinctive infrastructure feature is built-in AI coding support, described as “vibe-coding.” An open-source Agent Skills repository provides prompts and rules so assistants such as Claude Code, GitHub Copilot, and OpenAI Codex CLI can install Orcheo, generate Docker Compose files for the full stack, and scaffold new conversational pipelines. Because the emitted artifacts are plain Python and YAML, the resulting code remains inspectable, versionable, and runnable in both local and production settings.

## 4. Starter-kit components and workflow composition

The starter kit contains 50+ off-the-shelf nodes spanning the main stages of conversational search [2602.14710]. Query understanding and rewriting are represented by components such as `ContextualRewrite` and `FewShotRewriter`; retrieval by `BM25TaskNode`, `DenseSearchNode`, and `HybridSearchNode`; re-ranking by `CrossEncoderRerankerNode` and `SPLADERerankerNode`; context management by `ContextCompressorNode` and `DeduplicationNode`; response generation by `GroundedGeneratorNode` and `TemplateGeneratorNode`; evaluation by `RetrievalEvaluationNode`, `RougeMetricsNode`, `BleuMetricsNode`, and `AnswerQualityEvaluationNode`; and integrations by `SlackToolNode` and `TelegramToolNode`.

The framework’s composition model is intentionally simple. In the example linear RAG pipeline given in the paper, a `WorkflowBuilder` defines four nodes—`QueryRewriteNode`, `DenseSearchNode`, `ContextCompressorNode`, and `GroundedGeneratorNode`—then connects them with explicit edges and executes the result through `OrcheoClient`. A key property of this arrangement is that a node can be swapped by changing its class or parameters in the builder call, with no downstream edits required. This suggests that Orcheo treats pipelines as stable graph structures whose components can be replaced parametrically.

The paper further reports an engineering efficiency claim: comparable end-to-end evaluations that once took 500–1,000 lines of custom code now fit in 85–150 lines of Orcheo workflow definitions. Built-in history tracking, prompt versioning, and automated report generation are cited as the main sources of reduced boilerplate.

## 5. Evaluation primitives and reported case studies

Orcheo is not limited to workflow execution; it also packages evaluation nodes and dataset-oriented workflows. The paper reports two case studies intended to validate modularity and ease of use [2602.14710].

| Case study | Workflow summary | Reported metrics |
|---|---|---|
| QReCC query rewriting | `QReCCDatasetNode → ConversationalBatchEvalNode` invoking a `QueryRewrite` subgraph, then `RougeMetricsNode + SemanticSimilarityNode → AnalyticsExportNode` | `75.25 ROUGE-1 Recall`, `79.00 embedding-cosine` |
| MultiDoc2Dial grounded generation | `MultiDoc2DialDatasetNode → ConversationalBatchEvalNode` wrapping a four-stage RAG subgraph, then parallel `BleuMetrics`, `Rouge-L`, `TokenF1 → AnalyticsExportNode` | `Token F1=8.34`, `SacreBLEU=13.32`, `ROUGE-L=6.82` |

For standard IR evaluation, `RetrievalEvaluationNode` provides measures including $\mathrm{NDCG}@k$ and $\mathrm{MRR}$:

$$
\mathrm{NDCG}@k = \frac{1}{Z_k}\sum_{i=1}^{k}\frac{2^{rel_i}-1}{\log_2(i+1)}
$$

$$
\mathrm{MRR} = \frac{1}{|Q|}\sum_{q=1}^{|Q|}\frac{1}{\mathrm{rank}_q}
$$

These nodes emit run files in TREC format, ready for TIREx or ranxhub ingestion. For researchers, this is consequential because the same framework that defines the workflow can also produce evaluation artifacts compatible with established IR benchmarking infrastructure.

## 6. Extensibility, debugging, and production scaling

Orcheo’s extensibility model is based on custom single-file nodes. The paper’s example creates a new class under a project file, registers it through `registry.register(NodeMetadata(...))`, installs it through standard Python mechanisms, and references it by name in workflow JSON or builder calls. The registry auto-discovers such nodes at startup, with no core edits required [2602.14710].

The best-practice guidance is pragmatic and oriented toward maintaining modularity. Node dependencies should be kept minimal, package versions should be pinned in `requirements.txt`, and semantic versioning should be used for node packages. Large models or non-Python libraries are to be wrapped behind a `TaskNode` and accompanied by health checks. This suggests that Orcheo treats the node boundary as both an algorithmic and an operational encapsulation boundary.

For debugging and monitoring, the paper recommends the tracing viewer or third-party OTel dashboards for inspecting span timelines and payload sizes, LangSmith integration for prompt and usage logging, and verbose logging through the environment variable `ORCHEO_LOG_LEVEL=DEBUG`. For production scaling, it recommends horizontally scaling Celery workers by increasing worker count or deploying across Kubernetes pods, using Celery Beat for periodic tasks such as index refresh and analytics snapshots, employing Redis or RabbitMQ clusters for broker high availability, sharding large document collections across multiple `DenseSearchNode` instances with conditional routing, and caching frequent subgraph outputs through a dedicated `CacheNode` backed by Redis or Memcached.

## 7. Distinct uses of the name and related orchestration literature

A recurrent source of confusion is that “Orcheo” does not denote a single object across the literature. In conversational search, it refers to the modular full-stack platform described above [2602.14710]. In exoplanet science, however, the name appears in a separate sense as “the mission formerly known as EChO,” a dedicated space-borne infrared spectroscopic observatory designed to characterize the atmospheres of hundreds of transiting and eclipsing exoplanets [1403.0357]. That mission concept is defined by a 1 m-class, passively cooled telescope at $L_2$, a tiered survey strategy, and the EChOSim and ETLOS toolchain; it is unrelated to the CS software platform.

The broader term “orchestration” also has separate technical meanings. In contract automata, orchestration denotes coordination by an external orchestrator that synchronizes complementary offer and request actions in a product automaton, with strong agreement and a branching condition determining when the corresponding choreography of communicating machines is deadlock-free and convergent [1410.7471]. In multimodal scientific reasoning, SciOrch describes a framework in which an 8B orchestrator model decomposes questions, delegates sub-problems to selected commercial models through API calls, and synthesizes a final answer, with training based on MCTS and GRPO-style optimization [2606.15872].

These usages are conceptually adjacent only at a high level: all concern coordination across components, but they operate in different domains, with different primitives, objectives, and evaluation criteria. For that reason, Orcheo the conversational-search platform is best understood as a software framework for composing and deploying CS pipelines, not as a generic synonym for orchestration, not as the exoplanet observatory, and not as the scientific-reasoning orchestrator.

Source: https://www.emergentmind.com/topics/orcheo