Dynamic Stream Orchestrator (DSO)

• Dynamic Stream Orchestrator (DSO) is a runtime component that enables programmable, event-triggered adaptation of continuous stream-processing applications.
• It separates control logic from data processing using ORCA logic and ORCA service, promoting flexibility and reuse in application management.
• The framework addresses adaptation challenges such as failure recovery, dynamic composition, and semantic adjustments with precise event filtering and actuation APIs.

Dynamic Stream Orchestrator (DSO) denotes a dedicated runtime component for event-driven, user-defined adaptation of continuous stream-processing applications. In the defining stream-processing work, the component is termed an orchestrator and is implemented as a first-class concept in IBM System S: developers write ORCA logic that subscribes to runtime events, receives context describing the relevant logical and physical execution slice, and invokes actuation APIs through an ORCA service to adapt application behavior at runtime (Jacques-Silva et al., 2012). The mechanism was introduced to support algorithmic adjustment, incremental job deployment, application-specific failure recovery, and multi-application lifecycle management without relying on external scripts or embedding control operators into the stream graph.

1. Concept and motivation

Stream-processing applications are commonly deployed as continuous queries that run from submission until cancellation. That execution model complicates runtime adaptation whenever an application must respond to changes in input semantics, system failures, performance anomalies, or dependencies on other streaming jobs. The orchestrator was introduced precisely to make such adaptation explicit, programmable, and application-specific, rather than incidental to the dataflow itself (Jacques-Silva et al., 2012).

The central design objective is a separation between control logic and data-processing logic. Prior practice relied either on external scripts that monitor the system and issue commands, or on control operators inserted directly into the stream graph. The former is hard to maintain and weak in reuse; the latter tangles management policy with stream semantics. The orchestrator replaces both approaches with a uniform event/actuation interface in which control routines are written once, externalized from the graph, and executed as part of the runtime infrastructure.

The adaptation problems addressed by the framework are broad but concrete. They include changes in incoming data distribution or semantics, partial fault-tolerance policies that depend on application statefulness and tolerance for tuple loss, dynamic composition of multiple applications with startup and cancellation dependencies, and reactions to system-state or performance metrics such as PE failures, built-in counters, custom application metrics, and user-injected events. The orchestrator is therefore not a generic scheduler; it is a programmable runtime layer for expressing application-specific management policies.

2. ORCA logic, ORCA service, and the programming model

The orchestrator consists of two parts. ORCA logic is user-written control code. ORCA service is a runtime daemon that detects and collects events, delivers them with context, and exposes actuation APIs. This split is foundational: ORCA logic contains policy, while ORCA service mediates access to runtime mechanisms and system metadata (Jacques-Silva et al., 2012).

Developers implement ORCA logic in C++ by inheriting from an Orchestrator class. That class provides callback signatures for event handlers. The compiled result is a shared library, .so, loaded by the ORCA service. Through the service reference, ORCA logic can register event scopes, inspect application topology and logical-to-physical mappings, submit and cancel jobs, restart processes, modify placement configuration, set exclusive host pools, manage application dependencies, and issue control commands to managed operators or components.

A key architectural point is that the ORCA service acts as a proxy for control operations. The control routine does not directly manipulate System S internals; it invokes actuation routines exposed by the service, and the service coordinates those actions in application-specific order. This enables reusable control patterns across applications while preserving separation from the underlying runtime subsystems.

The model is explicitly context-aware. Adaptation policies are usually written in terms of logical application structure—operators, composites, application names, and metric names—but actuations often target physical execution entities such as processing elements (PEs) and hosts. The orchestrator bridges that gap by combining logical filtering during subscription with runtime queries that recover the relevant physical placement when an event is handled.

3. Event semantics, filtering, and delivery

The orchestrator’s subscription abstraction is the event scope. Developers explicitly register interest in most events; the only event always in scope is the start notification. An event scope is a disjunction of subscopes: an event is delivered if it matches at least one registered subscope, and each event is delivered only once even if it matches multiple scopes (Jacques-Silva et al., 2012).

A subscope can filter on event type, application name, operator type, composite type, metric name, and other contextual attributes. The filtering semantics are precise. Filters on the same attribute are disjunctive, whereas filters on different attributes are conjunctive. This permits hierarchy-aware specifications such as “operator metric events for metric queueSize,” restricted to Split or Merge operators, contained in composite type composite1, and belonging to application Figure2.

The paper gives an SQL formulation equivalent to such a hierarchical event-scope query, emphasizing that the API is simpler than expressing the same containment logic directly over recursive composite structure:

WITH CompPairs (compName, parentName) as ( SELECT CI.compName, CI.parentName FROM CompositeInstances as CI UNION ALL SELECT CI.compName, CP.parentName FROM CompositeInstance as CI, CompPairs as CP WHERE CI.parentName = CP.compName )

When an event matches registered scopes, the ORCA service delivers two objects: scope keys, identifying the matched subscopes, and an event context describing the event and its runtime slice. Events are delivered one at a time. If more events arrive while a handler is running, they are queued in arrival order. The context contains the minimum information needed for the event type and can be used to query additional details, such as which operators are in a PE, which composites are in a PE, the enclosing composite for an operator, or the PE identifier for that operator.

Supported event sources fall into several categories. The orchestrator itself generates start signals, job submission events, job cancellation events, and timer expiration events. External runtime infrastructure contributes PE failure events, potentially host- or process-related notifications, and metric events pulled from system daemons. Built-in metrics include quantities such as tuples processed per operator, tuples sent by an operator, and tuple bytes processed by a PE. Custom application metrics can be created during execution and may encode domain semantics, such as counts of tweets with known cause or unknown cause. User-generated events may be injected through a command tool that directly calls the ORCA service.

For metric events, the ORCA service attaches an epoch value, incremented at each SRM query. Epochs function as a logical clock: a handler can determine whether multiple metric values came from the same polling round and avoid combining stale and fresh information. This mechanism is essential when a policy depends on synchronized interpretation of multiple custom or built-in metrics.

4. Runtime integration in IBM System S

The orchestrator was implemented on top of IBM System S, whose relevant subsystems are SPL, SAM, SRM, and HC. SPL is the streaming language. SAM, the Streams Application Manager, receives submissions and cancellations, spawns PEs, and can stop or restart PEs. SRM, the Streams Resource Manager, tracks hosts and component status, detects host and process failures, and collects metrics. HC, the Host Controller, is a local host daemon that starts local PEs and reports status and metrics (Jacques-Silva et al., 2012).

System S distinguishes between a logical stream graph and a physical deployment. Logically, SPL applications are composed of operators and composites. Physically, the runtime partitions them into PEs and maps those PEs to hosts. Operators from different composite instances may be co-located in one PE, while operators from the same composite may be split across multiple PEs. This mismatch is one of the main reasons the orchestrator requires explicit context objects and inspection APIs.

Operationally, the orchestrator is deployed through a user-submitted orchestrator descriptor XML and a user-written shared library. The workflow is fixed: the user submits the XML to SAM; SAM forks a new process for the ORCA service; the ORCA service loads the shared library; it then invokes the orchestrator start callback. The choice of a separate process provides memory isolation between user code and infrastructure components. System S was extended so that orchestration became a first-class runtime entity: SAM tracks orchestrators and their associated jobs, a single orchestrator can manage multiple applications, and the ORCA service interacts with SAM, SRM, and managed operators.

A major extension enabled by this integration is automatic management of application dependencies. The orchestrator maintains a dependency graph whose nodes are application configurations and whose directed edges are dependency relations annotated by uptime requirements. Each application configuration includes a string identifier, application name, parameter map, boolean garbage-collectable flag, and garbage collection timeout. Submission proceeds by snapshotting the dependency graph, cutting unrelated nodes and edges, identifying source applications with no unsatisfied dependencies, starting them through SAM, and then submitting subsequent applications after their dependency and uptime conditions are satisfied. Cancellation proceeds by checking whether an application feeds another running application, rejecting the request if starvation would result, and otherwise canceling garbage-collectable dependents after timeout while preserving applications that are non-garbage-collectable, explicitly submitted, or still required by others.

5. Representative adaptation routines

The paper demonstrates three adaptation scenarios that collectively define the practical scope of the orchestrator. They cover semantic adaptation, application-specific failover, and dynamic composition of multi-application streaming systems (Jacques-Silva et al., 2012).

Scenario	Triggering events	Core actuation
Incoming data distribution changes	Custom metrics for known-cause and unknown-cause tweets	Trigger Hadoop recomputation and reload cause model
Failures	PE failure events	Fail over among replicas and restart failed PE
On-demand dynamic composition	Custom profile-count metrics and final punctuation metric	Submit C3 applications and cancel them when complete

In the first scenario, the application is a Twitter sentiment-analysis pipeline that filters tweets by product, classifies sentiment, stores negative tweets for batch processing, associates them with a cause, aggregates causes for user frustration, and uses precomputed causes from a Hadoop job as the cause model. The adaptation logic subscribes to two custom operator metrics: count of tweets with known cause and count of tweets with unknown cause. The handler records both metric values and their epochs, compares them only when they originate from the same epoch, and triggers a Hadoop job if unknown exceeds known. To prevent thrashing, it does not retrigger if another job was started in the last 10 minutes, and it reloads the output model when the Hadoop job completes. The orchestrator logic for this scenario is reported as 114 lines of C++. Experimentally, the ratio of unknown to known causes is below 1 at startup, rises around epoch 250 when new antenna-related complaints appear, crosses threshold 1.0, and then drops below 1 again after model refresh.

In the second scenario, the application is the financial engineering Trend Calculator, which computes over a 600-second sliding window the minimum and maximum trade prices, average price, and Bollinger Bands above and below average. Because the application does not use checkpointing, PE failures cause loss of state for stateful operators and require reprocessing of 600 seconds of data. The orchestrator subscribes only to PE failure events. On startup it sets the application to run in an exclusive host pool, submits three replicas, assigns active and backup status, stores that status in a file for a GUI, and registers for PE failure notifications. Upon failure, it determines whether the failed replica is active or backup; if the active replica fails, it selects the oldest running replica as the new active one, on the rationale that it is most likely to retain the most complete sliding-window state, updates the status file, and restarts the failed PE. This logic is reported as 196 lines of C++. The experiment shows failover after killing a PE in the active replica, correct output from the backup-selected active replica, and incorrect output from the restarted replica until it reconstructs its full 600-second state.

In the third scenario, the orchestrator manages on-demand composition of a social-media profile analysis pipeline with three categories of sub-applications. C1 applications consume continuous streams such as Twitter and MySpace, extract profile and sentiment information, and identify profiles for further analysis. C2 applications depend on C1 profiles, query additional sources such as Facebook, Twitter, and blogs via BoardReader, and enrich profiles in a data store. C3 applications consume the C2 data store and correlate user sentiments with attributes such as age, gender, and location. The orchestrator registers for custom metrics from C2 applications that count profiles with a specific attribute and for the built-in final punctuation metric from each C3 sink. At startup it establishes dependency relations between C2 and C1, uses zero uptime requirement because C1 applications have no internal state, and submits all C2 applications, thereby causing the relevant C1 and C2 jobs to start. During execution, when enough new profiles with a given attribute have accumulated—illustrated in the experiment by spawning C3 around 1500 profiles—the orchestrator submits an AttributeAggregator for that attribute. On receipt of final punctuation from the C3 sink, it issues a cancellation request. This routine is reported as 139 lines of C++.

6. Significance, limitations, and terminological ambiguity

The orchestrator’s principal significance lies in making runtime adaptation a reusable systems abstraction rather than an ad hoc coding pattern. It combines event subscriptions over system and application state, context-aware event delivery, inspection APIs for both logical and physical metadata, actuation APIs for jobs, PEs, placement, and dependency management, and an execution model that isolates control logic from stream-processing logic (Jacques-Silva et al., 2012).

Several implementation boundaries are explicit. Failure events reuse existing System S detection mechanisms, so the orchestrator does not add failure-detection overhead to the managed application hot path. Metrics are pulled from SRM periodically, and user-generated events enter directly into the ORCA service. Handling can add recovery latency because of the additional control layer, but that trade-off is accepted in exchange for flexibility. The paper also notes that repartitioning operators into different PEs would require recompilation in SPL; the implemented ORCA service does not support recompilation directly, although exposing source-code annotations and triggering recompilation is mentioned as a possible extension.

A frequent misconception arises from acronym overlap. In stream processing, the relevant concept is the orchestrator described above. In power-systems literature, however, DSO generally denotes Distribution System Operator, a distinct entity involved in retail-market clearing, DER coordination, or TSO–DSO interaction rather than runtime adaptation of streaming jobs. That separate usage appears in work on comprehensive DER aggregator participation, ISO–DSO coordination, DSO-centric retail markets, and DSO-led or TSO–DSO operational planning frameworks (Mousavi et al., 2020, Mousavi et al., 2022, Haider et al., 2021, Bragin et al., 2021, García-Muñoz et al., 24 Mar 2026). For literature search and citation practice, this ambiguity is consequential: “DSO” is not a stable cross-domain identifier, whereas “orchestrator” in System S denotes a specific runtime adaptation framework with well-defined event, context, and actuation semantics.