Kieker Observability Framework v2

• Kieker Observability Framework is a modular system that collects metrics, logs, and traces to reconstruct execution flows and enable architectural analysis.
• The framework separates monitoring from analysis using a TeeTime-based pipeline, supporting diverse instrumentation for languages like Java and Python.
• Optimization strategies such as source instrumentation, record aggregation, and queue tuning reduce overhead while balancing data fidelity and runtime performance.

to=arxiv_search.search 在天天中彩票json {"query":"Kieker Observability Framework Version 2 arXiv (Yang et al., 12 Mar 2025) OR Kieker OpenTelemetry (Reichelt et al., 2024, Reichelt et al., 13 Oct 2025) Python Kieker (Larrivain et al., 31 Jul 2025) minimal overhead monitoring (Reichelt et al., 2023)","max_results":10,"sort_by":"submittedDate","sort_order":"descending"} to=arxiv_search.search ացինjson {"query":"Kieker observability framework", "max_results": 10} to=arxiv_search.search ացինjson {"query":"all:Kieker observability framework", "max_results": 10} to=arxiv_search 彩票娱乐注册json {"query":"Kieker observability framework", "max_results": 10} Kieker is an observability framework that, in Version 2, is presented as the successor of the Kieker Monitoring Framework and is explicitly oriented toward metrics, logs, and traces. Its core design separates monitoring from analysis, employs the TeeTime pipe-and-filter framework to assemble modular analysis pipelines, and emphasizes low-overhead collection with downstream reconstruction and visualization. Across recent work, Kieker appears both as a native observability stack and as an interoperability layer between its own record model and the OpenTelemetry ecosystem, including ExplorViz, while also supporting static and dynamic architecture recovery for Python systems (Yang et al., 12 Mar 2025).

1. Conceptual scope and system model

In Kieker’s terminology, observability concerns how well the internal state of a software system can be inferred from monitoring data. The Version 2 framework follows the three commonly accepted pillars of observability. Metrics are described as aggregated numerical data such as CPU usage, response times, and error rates, sampled at regular intervals; logging is treated as text-based records of discrete events or activities; tracing captures the path a request takes through a distributed system, where traces aggregate highly structured log events called spans (Yang et al., 12 Mar 2025).

Kieker is consistently described as having two major parts: monitoring and analysis. The monitoring part generates observability data, while the analysis part consumes those data for analytics. A complementary description in the interoperability work states that native Kieker records contain information about program execution, such as control flow points and system information. Control flow points denote the beginning and the end of operations or thread joins, and these are stored together with timestamps and metadata sufficient for reconstructing a call tree. System information includes CPU and disk utilization. This division shows that Kieker is not merely a tracer; it is a broader framework in which tracing is one, but not the only, signal class (Reichelt et al., 2024).

The framework’s architectural separation is also reflected in its persistent data handling. Monitoring probes and samplers collect events, a Monitoring Controller coordinates them, Monitoring Writers persist monitoring records in a monitoring log, and downstream analysis pipelines reconstruct traces and derive higher-level artifacts such as call trees and component graphs. A plausible implication is that Kieker’s framework boundary is intentionally drawn around a reusable monitoring log and stage-based analysis stack rather than around a single runtime agent or backend (Reichelt et al., 13 Oct 2025).

2. Instrumentation, records, and trace representation

Kieker supports multiple instrumentation mechanisms and several language targets. For Java, the framework provides automated or manual instrumentation through the JVM’s -javaagent interface and supports AspectJ, ByteBuddy, DiSL, and Javassist. Automated source transformation is also supported when Java sources are available, and framework-specific injection is available for servlets and Spring applications. Beyond Java, Kieker supports instrumentation of C, Fortran, and Python code (Yang et al., 12 Mar 2025).

A recurring feature of Kieker’s tracing model is its compact representation of synchronous control flow. In the OpenTelemetry-to-Kieker mapping work, Kieker invocation records are described with the fields tin, tout, eoi, ess, signature, and hostname. Here, tin and tout store start and end time, eoi is an execution order index, ess is the execution stack size, signature identifies the operation, and hostname denotes the executing host or service. Kieker reconstructs synchronous control flow using eoi and ess: ess increases by exactly 1 when a parent operation calls a child, and eoi provides global ordering within the trace. This representation avoids explicit parent references in persisted record data and is optimized for efficient serialization (Reichelt et al., 13 Oct 2025).

That compactness coexists with a more general observability model in Version 2. The tool-artifact paper states that Kieker traces consist of records that are spans, while the interoperability paper emphasizes begin and end events, thread joins, timestamps, and metadata. Read together, these descriptions indicate that Kieker’s internal representation is centered on reconstructable execution structure rather than on the in-process maintenance of OpenTelemetry-style span objects. This suggests a data model optimized for post hoc analysis and conversion.

3. Low-overhead monitoring and optimization strategies

A defining characteristic of Kieker in the recent literature is its emphasis on low overhead. Version 2 attributes this, in part, to complete decoupling of the application thread and the log writer thread: the application thread proceeds asynchronously immediately after writing a record into a queue. The OpenTelemetry export work similarly motivates Kieker by emphasizing lower performance overhead for gathering distributed execution traces, with compact records emitted by lightweight probes and correlation or reconstruction performed downstream rather than in-process (Yang et al., 12 Mar 2025).

The most detailed examination of this property appears in the study on minimal-overhead monitoring. That work identifies four optimization options realized in Kieker: source instrumentation instead of AspectJ instrumentation, reduction of measurement data, change of the queue, and aggregation of measurement data. Source instrumentation removes AspectJ runtime overhead and allows direct insertion of timing code into method bodies. Reduction of measurement data introduces DurationRecord, a minimal record storing only duration rather than the fuller OperationExecutionRecord. Queue optimization replaces LinkedBlockingQueue with a synchronized CircularFifoQueue. Aggregation introduces AggregatedWriter, which writes summaries over a fixed window instead of per-call records (Reichelt et al., 2023).

The empirical results are specific. In MooBench, with default parameters including call tree depth $d = 10$, 2,000,000 method invocations per VM, and 10 VM starts, the baseline mean was 0.0548 $\mu s$ per top-level call; AspectJ-based Kieker monitoring measured 4.7711 $\mu s$; source instrumentation with OperationExecutionRecord measured 2.4169 $\mu s$; source instrumentation with DurationRecord measured 2.3426 $\mu s$; source instrumentation with CircularFifoQueue and OperationExecutionRecord measured 1.5017 $\mu s$; and the combination of source instrumentation, DurationRecord, AggregatedWriter, and CircularFifoQueue measured 0.3897 $\mu s$. The paper highlights this as an overall reduction from 4.7711 $\mu s$ to 0.3897 $\mu s$ per top-level call for the compared configurations (Reichelt et al., 2023).

These optimizations come with explicit trade-offs. DurationRecord and aggregation reduce overhead but sacrifice metadata needed for call tree reconstruction and architecture recovery. CircularFifoQueue reduces allocation and blocking costs but can overwrite the oldest elements when capacity is exceeded, creating a risk of silent data loss. The framework therefore presents low overhead not as a single fixed property, but as a design space in which fidelity, completeness, and diagnostic granularity can be traded against runtime cost.

4. Analysis pipelines and architecture recovery

Kieker’s analysis model is built around stage composition. The framework uses TeeTime as a pipes-and-filters substrate, and the Version 2 paper explicitly states that TeeTime was incorporated to restructure and simplify analysis pipelines. Users can therefore assemble custom analysis applications by chaining compatible stages and formats (Yang et al., 12 Mar 2025).

This modularity is visible in the Python-oriented work, which extends Kieker beyond its original Java orientation. That pipeline combines dynamic analysis, static analysis, model merging, and visualization. Dynamic analysis instruments Python code through OpenTelemetry-based tooling, translates spans into Kieker records, and reconstructs a dynamic architecture. Static analysis parses Python source to extract function-call and data-flow relations, then reconstructs a static architecture. The two models are merged and visualized. The reconstructed Python architecture follows a Java-inspired structure that includes classes and operations rather than only files (Larrivain et al., 31 Jul 2025).

Tool	Function	Language
OtktInst	Semi-automated instrumentation utility for Python based on OTel	Python
Otkt DSL	OpenTelemetry-to-Kieker Translation DSL	Python/Java/Xtext
DAR	Dynamic Architecture Recovery	Java
Pyparse	AST parsing and CSV generation for function calls and data flow	Python
SAR	Static Architecture Recovery	Java
MOP	Merges and compares architecture models	Java
MVIS	Exports architecture models to DOT and GraphML	Java
GGVIS	Grouped Graph Visualizer built on Tulip	Python

The Python study reports concrete processing times and visualization behavior. Representative totals include approximately 00:12 for Anytree, 00:22 for UXsim, 00:29 for Pillow, 43:12 for Matplotlib, 33:37 for NumPy, and 29:47 for SciPy, with dynamic data-gathering time explicitly excluded. For visualization scalability, Graphviz was reported to struggle with large nested graphs, whereas Tulip-based GGVIS showed major improvements; for the NumPy architecture, fdp required approximately 16:38, while GGVIS completed in about 00:30–01:00 depending on output format (Larrivain et al., 31 Jul 2025).

A common misconception is to treat Kieker primarily as a collector. The recent literature instead presents it as a framework for architectural reverse engineering and execution analysis, in which collection is only the first stage. Call trees, component graphs, merged static–dynamic models, and software-landscape visualizations are central outputs rather than secondary features.

5. OpenTelemetry interoperability and model translation

Interoperability with OpenTelemetry is a major recent theme in Kieker research, but it is not described as one-to-one equivalence. The Kieker-to-OpenTelemetry paper states that Kieker’s custom record types are “not one-to-one compatible” with OpenTelemetry formats, even though OpenTelemetry has become the de-facto standard for observability data and is relied upon by tools such as Zipkin, Prometheus, Grafana, and ExplorViz. The implemented solution is a TeeTime-based export from reconstructed Kieker ExecutionTrace instances to OpenTelemetry spans. In the demonstrated pipeline, data flow proceeds from Kieker probes and SingleSocketTcpWriter through MultipleConnectionTcpSourceStage, ExecutionRecordTransformationStage, TraceReconstructionStage, and OpenTelemetryExporterStage, and then onward to ExplorViz. The exporter supports gRPC and Zipkin; in the ExplorViz demonstration, spans are sent via gRPC to the OpenTelemetry Collector, then via Apache Kafka through the backend, and finally visualized in ten-second time windows (Reichelt et al., 2024).

The reverse direction exposes a deeper modeling mismatch. OpenTelemetry traces are designed to represent asynchronous, distributed interactions across services, whereas Kieker traces are synchronous and encode control flow through eoi and ess instead of explicit parent references. The OpenTelemetry-to-Kieker transformer therefore maps startEpochNanos to tin, endEpochNanos to tout, OTel name to signature, and derives hostname from semantic attributes such as net.sock.peer.addr and net.peer.name. Because overlapping spans can make a strictly stack-based ess assignment inconsistent, the authors evaluated four solution options and chose to mark traces as asynchronous. As a result, analysis of OpenTelemetry-converted traces should be run with kieker-trace-analysis --asynchronousTrace (Reichelt et al., 13 Oct 2025).

Interoperability is therefore practical but partial. The export from Kieker to OpenTelemetry covers traces reconstructed from control-flow records; metrics and logs are acknowledged but not exported in that work. Conversely, the OpenTelemetry-to-Kieker work focuses on call tree reconstruction from spans and does not discuss metrics, logs, span links, sampling, dropped spans, or status and event mapping. A plausible implication is that the current interoperability layer is strongest where a trace can be represented as an operation hierarchy and weaker where signal richness depends on the broader OpenTelemetry semantic model.

6. Demonstrations, limitations, and ongoing directions

Recent demonstrations situate Kieker in microservice and polyglot settings. The Version 2 tool artifact presents an end-to-end deployment in which TeaStore, instrumented with Kieker agents, is connected through the Kieker OpenTelemetry Transformer to the OpenTelemetry Collector and then to ExplorViz; JMeter drives an initial workload so that executed services and interactions become visible in the software-landscape visualization. The same paper also provides an online service and a Docker Compose artifact for local deployment (Yang et al., 12 Mar 2025).

Other demonstrations emphasize complementary aspects of the framework. One interoperability study visualizes TeaStore traces in ExplorViz after online conversion from Kieker data to OpenTelemetry. Another transforms OpenTelemetry spans from the Astronomy Shop, a demo application with 14 services in 11 languages, into Kieker records so that Kieker’s trace analysis tool can render cross-service call trees. The Python study validates static and dynamic architecture recovery on Anytree, UXsim, Pillow, Matplotlib, NumPy, and SciPy, and also documents sustainment challenges, including non-functional tools in the official repository and reliance on archived replication packages for working binaries (Reichelt et al., 2024).

Several limitations recur across the literature. The OpenTelemetry export work does not specify exact trace-ID and span-ID mapping, status or error handling, or clock-synchronization assumptions. The OpenTelemetry import work does not provide quantitative throughput or latency evaluation and requires --asynchronousTrace for microservice traces originating in asynchronous systems. The Python work does not provide formulas for overhead, correlation, or complexity beyond reporting empirical timings. The Version 2 paper does not formalize the internal record schema, transport details, storage backends, or fault tolerance (Reichelt et al., 13 Oct 2025).

The stated research directions are correspondingly concrete. The Kieker-to-OpenTelemetry work proposed importing OpenTelemetry traces into Kieker, defining OpenTelemetry records through Kieker’s Instrumentation Record Language, and comparing the overhead of transforming Kieker records against writing OpenTelemetry records directly. The later OpenTelemetry-to-Kieker work realizes part of that interoperability agenda and further proposes combining Kieker inside applications to minimize overhead with OpenTelemetry at service boundaries to capture distributed interactions. Taken together, these developments position Kieker as a framework centered on low-overhead execution monitoring, modular analysis, and bidirectional interoperability, rather than as a closed alternative to standard observability tooling (Reichelt et al., 2024).