AgileLog: Forkable Stream Log for AI Agents

• AgileLog is a forkable shared-log abstraction designed to support AI agents with dynamic, speculative operations on live data streams.
• It introduces continuous and severed forks that provide logical isolation and performance protection via zero-data-copy and optimized metadata strategies.
• The Bolt implementation demonstrates minimal performance interference even under heavy agent workloads, validating AgileLog's efficiency and scalability.

AgileLog is a shared-log abstraction for streaming systems that adds forking as a first-class operation to support AI agents operating over live data streams. It was introduced together with Bolt, a C++ implementation built on a diskless shared-log architecture, to address two deficiencies of conventional streaming systems for agentic workloads: the lack of protection from performance interference and the lack of a safe mechanism for staging, validating, and selectively adopting agent-generated writes (Bhat et al., 16 Apr 2026).

1. Motivation and problem setting

AgileLog is grounded in the observation that AI agents are not ordinary stream-processing programs. Traditional stream applications have fixed, programmer-defined logic, whereas agents reason on the fly, choose tools dynamically, and are exploratory or speculative. In streaming settings, this creates irregular read patterns, branching behavior, retries, and agent-generated writes that may be malformed or semantically wrong. The paper identifies two core insufficiencies in conventional shared logs: no protection from performance interference when agentic workloads contend with production consumers and producers, and no safe way to handle agentic writes that may later need to be validated, discarded, or selectively adopted (Bhat et al., 16 Apr 2026).

The proposed answer is not a temporary side log or a larger client API, but a shared log that can fork itself. A fork gives an agent a logically separate, performance-isolated child copy of a log that shares some or all of the parent’s history, can be read and appended to like a normal log, and can itself be forked. This design targets use cases such as ad-hoc analytics and dashboards, real-time context retrieval, agentic stream processing, coding and testing of stream applications, and what-if simulation and counterfactual analysis. The system is therefore explicitly a storage and coordination substrate for agents on streaming data, not an agent framework (Bhat et al., 16 Apr 2026).

2. Interface, fork types, and core semantics

AgileLog extends a conventional shared log with two forking primitives and two lifecycle primitives. The interface presented in the paper is:

$B$5

Two fork types define the abstraction. A continuous fork (cFork) shares the parent’s history up to the fork point, continues to inherit all future appends on the parent, and may also receive private appends of its own. This yields unidirectional write isolation: parent appends are visible to the child, but child appends are not visible to the parent. A severed fork (sFork) shares the parent’s prefix up to the fork point and then disconnects; later parent writes are not visible to the child, and child writes remain private. sFork can also be created from a past offset rather than only from the current tail (Bhat et al., 16 Apr 2026).

The semantics of cFork require linearizable interleaving. If append $A$ completes on the parent before append $B$ starts on the child, then $B$ must appear after $A$ in the child’s total order. Forks may be nested, forming an inheritance forest of logs. Promotion is restricted to promotable continuous forks: after promote(), the child effectively becomes the parent beyond the fork point, while squash() deletes a fork and recursively clears its metadata state. Root logs cannot be squashed. The abstraction deliberately does not support arbitrary multi-branch merges; promote is a restricted merge in which one child can replace the parent beyond the fork point (Bhat et al., 16 Apr 2026).

3. Bolt implementation and data structures

Bolt implements AgileLog as a diskless shared-log system with three main components: stateless brokers, a shared object store, and a fault-tolerant metadata layer replicated with Raft. Brokers accept appends and reads, batch records, write objects to shared storage, and communicate with the metadata layer. The shared object store is MinIO in the prototype, though the design assumes any S3-compatible store. The metadata layer sequences records and maintains per-log metadata including an index mapping positions to object locations, a tail, and fork metadata. The implementation is in C++ and is approximately 21K LOC; it uses eRPC for RPC and willemt/raft for replication (Bhat et al., 16 Apr 2026).

The implementation makes forks cheap in two senses. Zero-data-copy forking is achieved because inherited history points to the same shared-storage objects as the parent. Zero-metadata-copy fork creation is achieved with a Hierarchical Log Index (HLI). Instead of copying the parent index into the child, the child starts with an empty map and recursively consults ancestors on reads of inherited positions. Because logs are append-only, inherited prefixes do not change, and this indirection is safe (Bhat et al., 16 Apr 2026).

Continuous forks introduce a harder metadata problem. A naive implementation, called BoltNaiveCF, would copy parent metadata into all descendants on every parent append. Bolt avoids this through tail-only updates: when a record is appended to a log $P$, the metadata entry is inserted only in $P$’s own index, while descendants’ tails are advanced to reflect inherited visibility. Each record’s metadata is stored in exactly one log index, namely the log where it originated. To resolve inherited positions, HLI augments local entries with a cumulative count of records locally appended up to and including a position. If $i$ is a child position and $l$ is the number of local appends before $i$, the corresponding parent position is $i - l$. Tail propagation is further optimized with a Lazy Tail Tree (LTT), represented as an Euler tour of the inheritance tree stored in a balanced BST, yielding subtree updates and point queries in $B$0 where $B$1 is the number of logs (Bhat et al., 16 Apr 2026).

4. Isolation, promotion, and operational behavior

AgileLog is designed to provide both logical isolation and performance isolation. Logical isolation depends on fork type. sFork gives bidirectional isolation after the fork point. cFork gives unidirectional isolation: parent appends remain visible to the child, but child-local writes do not leak into the parent unless the child is explicitly promoted. Performance isolation is realized in Bolt by running forks on brokers separate from the parent while reusing shared object storage, so agentic reads and appends do not interfere with production workloads at the broker layer (Bhat et al., 16 Apr 2026).

Promotion adds a semantic complication because a promotable cFork may change parent positions after the fork point. Bolt therefore tracks the earliest active promotable fork point with earliest-fp. While a promotable continuous fork is active, reads on the parent beyond the fork point are disallowed, append positions beyond that point are not returned, and non-promotable descendants are blocked beyond that point. The motivation is positional stability: after promotion, positions beyond the fork point may shift. This is one of the most important operational restrictions in the design (Bhat et al., 16 Apr 2026).

The abstraction is intended to support stateful validation. A continuous fork can contain pre-fork history, post-fork non-agentic parent updates, and the agent’s own candidate writes in one linearized stream. That makes it possible to run a copy of a downstream stateful consumer on the fork and validate behavior in full temporal context before promotion. This property distinguishes AgileLog from temporary side-log approaches, which do not provide the same temporal semantics for validation. The cost is that promotable forks can temporarily reduce parent consumer throughput because of the blocking semantics beyond earliest-fp (Bhat et al., 16 Apr 2026).

5. Experimental results and demonstrated applications

Bolt was evaluated on CloudLab xl170 with 9 nodes running MinIO, a 3-replica metadata layer, a dedicated node per broker, and 4 KB records. The principal baselines were internal variants rather than prior forkable logs, since no existing shared-log system supports this abstraction. The reported fork creation latency is about $B$2, independent of parent log length, whereas metadata-copy-based forking reaches about 100 ms with 25M entries. With 100 fork creations, Bolt leaves parent throughput essentially unaffected, while BoltMetaCpy degrades it. For 1000 continuous forks and 1M appended records, metadata memory overhead is 8 MB in Bolt and 4.4 GB in BoltNaiveCF. In nested continuous forks with 1M records per level, lookup at depth 7 is only 5.2% slower, with metadata lookup overhead $B$3 (Bhat et al., 16 Apr 2026).

The isolation results are central. In a latency-critical workload plus analysis workload on the same stream, Bolt runs analysis on an sFork via its own broker and the production workload sees no interference, whereas Kafka shows 2.5× latency degradation. With a root log appending at 13 KOps/s and a cFork also appending at 13 KOps/s, root mean and p99 end-to-end latencies remain unaffected in Bolt. Scaling experiments with 0, 10, and 100 cForks per root, and with 32 root logs, show little or no degradation (Bhat et al., 16 Apr 2026).

The paper also implements three LLM-powered applications using OpenCode with Gemini-2.5-Pro. In an ad-hoc IoT analytics agent over the first 1M records, Kafka exhibits 14× higher mean latency and 130× higher p99 latency during agent queries, while Bolt shows no interference. In a stream-processor testing agent, non-promotable cForks provide a realistic sandbox with real data and injected malformed, late, or duplicate test events, without degrading root latency. In a supply-chain restocking agent, promotable cForks stage candidate restock events and validate them with a stateful copy of the downstream consumer; a schema-error event that crashes the downstream consumer in Kafka remains isolated in Bolt until validated or squashed (Bhat et al., 16 Apr 2026).

6. Relation to adjacent research and broader interpretations

AgileLog belongs to a broader turn toward log-centric and fork-aware agent substrates, but its target is specifically streaming data. The paper distinguishes it from conventional shared logs and stream systems such as Kafka, Pulsar, Scalog, LazyLog, and Boki, which do not provide forking as a first-class primitive, and from branching/versioning databases and lakehouses, because AgileLog introduces continuous forks, in which a child continues to inherit future parent appends. Related log-centric agent runtimes pursue different abstractions: LogAct models an agent as a deconstructed state machine playing a shared log in which intentions are visible before execution and can be stopped by voters, while ActiveGraph makes the append-only event log the source of truth and the working graph a deterministic projection of that log (Balakrishnan et al., 9 Apr 2026, Nakajima, 21 May 2026).

Outside data-streaming systems, the label “AgileLog” has also been used as an interpretive frame for software-engineering work on agile process observability, adaptive logging, and issue-history mining. In continuous integration research, it has been used to describe a workflow combining Dynamic Regression, Code Bisector, and Code Quality to improve commit-to-test impact visibility and CI diagnostics (Sivanandan, 2015). In adaptive logging, Git histories and Mylyn Degree of Interest have been used to rejuvenate feature log levels as software evolves, successfully analyzing 99.22% of logging statements and increasing the focus of logs in bug-fix contexts $B$4 of the time (Tang et al., 2021). In empirical software-process mining, the TAWOS dataset provides 508,963 Jira issues, 10,023,871 changelog entries, and 1,612,399 comments from 44 open-source Agile projects, making issue and workflow histories available as a reusable relational corpus (Tawosi et al., 2022).

Within the streaming-systems sense, however, AgileLog has a narrower and more specific meaning: a forkable shared log for agents on data streams, centered on cFork, sFork, promote, and squash, and implemented in Bolt through zero-data-copy sharing, zero-metadata-copy fork creation, hierarchical indexes, tail-only updates, and lazy tail propagation (Bhat et al., 16 Apr 2026).