CogControlBench: Dual-Domain Benchmark

• CogControlBench is a dual-domain benchmark that rigorously evaluates reasoning-driven video generation and in-memory database concurrency control protocols.
• It employs professionally curated datasets and unified system components to ensure fairness, reproducibility, and fine-grained analysis.
• The platform provides actionable insights through detailed performance metrics and optimized protocols for both creative content synthesis and transactional systems.

CogControlBench (CCBench) serves two distinct but highly specialized roles within contemporary research: first, as a leading benchmark for reasoning-driven, controllable video generation using professional animation workflows; and second, as an open-source, unified evaluation platform for in-memory database concurrency control protocols on many-core servers. In both domains, CCBench is characterized by rigorously curated datasets, precisely defined evaluation metrics, and a focus on exposing nuances in high-level reasoning and system performance under real-world or industrial conditions.

1. Motivation and Design Principles

CCBench for controllable video generation was created to evaluate models' capabilities to (a) interpret sparse or abstract professional-level control inputs (such as hand-drawn storyboards and clay renders), (b) infer the underlying creative intent, and (c) generate high-fidelity videos that satisfy both pixel-level constraints and dense, semantic reasoning outputs. This addresses key gaps unfilled by generic vision-LLMs (VLMs), which struggle with multimodal ambiguity and lose alignment between high-level intent and low-level video output, and by prior benchmarks which utilized only simulated or synthetic data (Yang et al., 19 May 2026).

In the context of concurrency control, CCBench is structured to address confounding factors in in-memory database benchmarking. It ensures protocol comparisons are attributable strictly to concurrency control logic and optimizations, not to implementation idiosyncrasies, by enforcing a shared access method (Masstree), a uniform memory allocator, and modular, attachable/detachable optimization components (Tanabe et al., 2020). Goals are fairness, reproducibility, and the ability to dissect protocol performance at scale across CPU cache, conflict resolution delay, and version chain lifetime.

2. Construction and Dataset Properties

For video generation, CCBench comprises:

• Professional Workflow Data: Hand-drawn storyboard sequences and clay-render draft videos, each explicitly paired with final animation clips from internal anime production pipelines.
• General Generation Data: Supplemented by public community contributions (e.g., createwithclint.com) and established benchmarks such as VACE-Bench supplying subject-to-video, pose, and depth-to-video mappings.

The dataset consists of 200 high-resolution (720p) professionally validated validation samples, and a training corpus of approximately 50,000 mixed professional and general cases. Human annotators perform semantic alignment between control and output, followed by automated extraction of auxiliary controls (line art, depth, key poses) per frame and a final quality assurance stage (Yang et al., 19 May 2026).

For concurrency control, the CCBench platform is a single executable binary in which each worker thread plays both client and server roles, sharing core modules for key-value indexing, memory management, and transaction handling APIs. It does not provide a full database stack (notably, no SQL parser or layered storage engine), and focuses strictly on CC protocol analysis (Tanabe et al., 2020).

3. Task Definitions and Supported Protocols

In video generation, CCBench evaluates a single compositional task: given a control video ($V_{ctrl}$), a reference image ($I_{ref}$), and a text description ($T_{desc}$), the model must generate a video ($V_{gen}$) that:

• Reconciles all input cues—including resolving implicit or conflicting details,
• Preserves the identity and style of $I_{ref}$,
• Adheres to dynamic and spatial constraints of $V_{ctrl}$,
• Satisfies semantic requirements in $T_{desc}$.

Subtasks include storyboard-to-video, clay-render-to-video, and reference-to-video translation, each probing different aspects of professional intent cognition and visual controllability (Yang et al., 19 May 2026).

In concurrency control, CCBench supports implementation and evaluation of seven serializable protocols:

1. Two‐Phase Locking (2PL): Strict locking with various deadlock-resolution optimizations.
2. Silo (OCC): Optimistic concurrency control with invisible reads.
3. TicToc: OCC enhanced with per-record read/write timestamps.
4. MOCC: Adapts between OCC and 2PL based on contention “temperature”.
5. Snapshot Isolation (SI): Multi-version CC with snapshot reads and version chain GC.
6. SI+SSN (ERMIA): SI extended with Serial Safety Net for serializability.
7. Cicada: Hybrid OCC/MVCC with optimizations for timestamp management and version chain maintenance.

Each protocol is augmented by up to seven modular optimizations, such as decentralized timestamping, invisible reads, adaptive backoff, read-phase extension (deliberate validation delays), assertive version reuse, and rapid or aggressive garbage collection (Tanabe et al., 2020).

4. Evaluation Protocols, Metrics, and Benchmarks

CCBench utilizes both numerical and judge-based evaluation procedures.

For video generation:

• Condition Following: Multimodal Intent Alignment (MI), Appearance Follow (AF), Style/Content/Dynamic Follow (SF/CF/DF), all as scalar judge scores ([0, 5]) using Gemini 3.1-Pro.
• Visual Quality: Aesthetic Quality (AQ), Image Quality (IQ), Temporal Flickering (TF), Motion Smoothness (MS), Dynamic Degree (DD), Motion Naturalness (MN), Identity Consistency (IC), Dynamic Plausibility (DP), all scored [0, 5].
• Quantitative Metrics: Accuracy of condition following (aggregate correct integration), Fréchet Inception Distance (FID) for distributional similarity, and CLIP score (cosine similarity between frame embeddings and text prompt):

$$\mathrm{CLIP} = \frac{1}{N}\sum_{i=1}^N \frac{f_{\mathrm{clip}}(V_{\text{frame},i}) \cdot g_{\mathrm{clip}}(T_{\text{desc}})}{\|f_{\mathrm{clip}}(V_{\text{frame},i})\|\,\|g_{\mathrm{clip}}(T_{\text{desc}})\|}$$

For concurrency control:

• Throughput: $\Delta = C / T$ (completed transactions per unit time)
• L3 Miss Rate: $= \mathrm{L3\_Misses} / \mathrm{L3\_Accesses}$
• Abort Rate: $I_{ref}$0 (ratio of aborts to total attempts)
• Version Lifetime: $I_{ref}$1

Workloads are parameterized by key skew, record cardinality, payload size, transaction size, read/write mix, RMW semantics, and thread count, supporting fine-grained explorations of scaling, contention, and protocol robustness.

5. Benchmark Results and Empirical Insights

CogControlBench Video Generation Performance

The table below summarizes average judge scores (normalized to [0,1]) across CogControlBench’s main metrics (Yang et al., 19 May 2026):

Model	AQ	IQ	TF	MS	DD	MI	AF	SF	CF	DF	MN	IC	DP	Overall
Kling-3O	0.571	0.644	0.979	0.987	0.511	0.702	0.741	0.753	0.474	0.625	0.657	0.577	0.410	0.704
Seedance2.0	0.589	0.653	0.980	0.989	0.517	0.822	0.850	0.870	0.884	0.614	0.808	0.746	0.546	0.750
VACE-Wan2.1	0.549	0.636	0.975	0.986	0.528	0.684	0.672	0.744	0.780	0.526	0.755	0.736	0.551	0.665
VINO	0.570	0.581	0.980	0.989	0.280	0.664	0.770	0.803	0.826	0.457	0.777	0.729	0.542	0.686
OmniWeaving	0.512	0.549	0.976	0.982	0.396	0.526	0.423	0.495	0.773	0.515	0.651	0.588	0.531	0.607
CogOmniControl (Harness BoN)	0.596	0.637	0.980	0.990	0.531	0.785	0.792	0.843	0.864	0.562	0.805	0.719	0.546	0.742

Key findings include the substantial impact of VLM fine-tuning (supervised and reinforcement), the role of unified sequence conditioning, and a 2% relative gain using closed-loop Best-of-N evaluator selection. CogOmniControl narrows the performance gap with proprietary systems to ~3%.

Concurrency Control Analysis

Six insights from 224-thread benchmarking include:

1. Read-throughput Degradation: Occurs even without L3 cache misses as cardinality increases, due to L1/L2 thrashing.
2. Invisible Reads Efficiency: Silo surpasses TicToc under moderate contention by avoiding cache-line invalidations.
3. Abort/Wait Tradeoff: No universal best choice; transactional abort/adaptivity is context-dependent.
4. ReadPhaseExtension Benefit: Extra reads after validation failure can implicitly serialize concurrency, lowering abort rates.
5. Platform Biases: Direct protocol comparison mandates a unified infrastructure to avoid misleading version-cost conclusions.
6. Garbage Collection Limitations: State-of-the-art rapid GC fails under workloads with a single long transaction, necessitating aggressive GC strategies.

6. Limitations and Future Prospects

CogControlBench for video generation currently focuses on 200 anime-specific validation samples, with plans for extension to live-action, gaming, and increased sample sizes. A notable constraint remains manual semantic alignment during annotation. Real-time inference remains computationally prohibitive (32×H20 GPUs). Domain generalization to photographic or 3D CGI remains an open challenge (Yang et al., 19 May 2026).

On the concurrency control axis, CCBench does not currently implement a complete database stack or full TPC-C support, nor does it provide integrated durability or logging. Extensions to distributed concurrency control and richer transactional workloads are anticipated (Tanabe et al., 2020).

7. Comparative Context and Significance

Within the video generation landscape, CogControlBench distinguishes itself by anchoring benchmark evaluation on genuine, professional-level creative intent, both in data sourcing and task formulation. It exposes deficiencies in prior VLM-augmented workflows and prototypical adapter-based systems, and demonstrates the value of closed-loop architectures and finely tuned intent cognition for controllable synthesis.

For concurrency control, CCBench uniquely synthesizes seven modern protocols and optimizations in a single, extensible laboratory, supporting up to 224 logical threads with maximal fairness and reproducibility. This enables fine-grained isolation of the factors governing thread-scalability, cache affinity, abort dynamics, and version lifetimes, informing both system-level innovation and fundamental protocol theory.

Together, both deployments of CogControlBench provide robust testbeds—one for reasoning-driven content synthesis with human-aligned controls and another as a vehicle for dissecting the mechanics of large-scale in-memory transactional systems.