OBLIQ-Bench: Dual Benchmarking Framework

• OBLIQ-Bench is a dual benchmarking framework that rigorously evaluates both provably-secure obfuscators and latent query retrieval systems.
• In the obfuscation domain, it employs bounded-width Boolean circuits, BPW file format, and standardized cost models to measure runtime, energy, and security.
• For retrieval tasks, it assesses systems using metrics like Recall@k and NDCG@k on oblique queries that rely on latent document attributes.

OBLIQ-Bench refers to two distinct, technically significant benchmarking frameworks: one for the evaluation of provably-secure software functional obfuscators (Thomborson, 2015), and another for probing retrieval bottlenecks in information retrieval involving latent or implicit query-document relationships (Tchuindjo et al., 7 May 2026). Both frameworks are unified by rigorous benchmark construction, metrication, and the goal of exposing system limitations under challenging, non-trivial demands.

1. Foundations of OBLIQ-Bench: Dual Provenance

The term "OBLIQ-Bench" designates:

• A suite for benchmarking provably-secure obfuscators, centered on evaluating indistinguishability obfuscation (iO) under practical constraints. It provides Boolean-circuit workloads, a standard cost model, a file format (BPW), and an experimental protocol for reproducible measurement (Thomborson, 2015).
• A corpus and methodology–driven evaluation suite for information retrieval – specifically, for challenging "oblique queries," that is, retrieval tasks where relevance is determined by latent attributes, not surface-level overlap. OBLIQ-Bench tailors tasks across descriptive, analogue, and tip-of-the-tongue query forms, focusing on real-world corpora (Tchuindjo et al., 7 May 2026).

Both instantiations rely on formal problem definitions, explicit relevance or correctness conditions, and standard evaluation pipelines grounded in the respective domains.

2. Benchmarking Obfuscated Functional Circuits

OBLIQ-Bench, as introduced by A. Russell et al., is designed to empirically evaluate functional obfuscators using “bounded-width” Boolean circuit benchmarks (Thomborson, 2015).

Key components include:

• Workload model: The suite targets functionally-obscure Boolean circuits, excluding control-flow/data obfuscation.
• Parameters:
  • Circuit size $n\geq 10^6$ gates.
  • Circuit width $w\geq50$, ensuring cache/memory stress.
• File format: BPW (Bounded-Program-Width) enables portable and unambiguous representation of large circuits.
• Standard tasks:
  • Random-Circuit Benchmark: Parameterized by $(n,w,d)$; exercises memory effects and induces cache churn via COPY operations.
  • Password-Recognizer Benchmark: Models point functions for iO, where only inputs matching a secret pattern produce a non-trivial output.
• Methodology: Workflows encompass circuit generation, candidate obfuscator application (e.g., GGH-type iO, VM interpreters), evaluation of runtime, energy, and thermal metrics across multiple hardware platforms.

The benchmark design captures real-world constraints (e.g., on smartphones, desktops, GPUs) and provides validation data on the $n\sqrt{w}$ cost model.

3. OBLIQ-Bench for Retrieval: Latent and Implicit Query Benchmarking

OBLIQ-Bench, in information retrieval, targets tasks where standard search fails due to reliance on inductively weak surface-level alignment (Tchuindjo et al., 7 May 2026).

• Oblique Query Definition: An oblique query specifies relevance by reference to a latent or implicit attribute $f(d)$ of documents $d$ rather than surface-lexicon overlap.
• Three classes of obliqueness:

1. Descriptive: Retrieval by implicit property (e.g., stance or failure mode).
2. Analogue: Retrieval by abstract pattern match (e.g., proof meta-program).
3. Tip-of-the-tongue: Retrieval by vague, lossy recollection requiring scenario inference without identifiers.

• Pipeline for Golden Judgments:
  • Human defines $f$.
  • LLM annotates corpus.
  • Clustering by latent attributes to group documents.
  • LLM constructs abstract queries, prohibiting vocabulary leakage.
  • Multi-retriever pooling with LLM re-judging to maximize recall without exhaustive pairwise labeling.

Benchmarked tasks include Twitter stance classification, LLM failure mode discovery, proof meta-program identification, writing style attribution, and congressional hearing recall (each parameterized by corpus size and query count).

4. Evaluation Methodologies and Metrics

Functional obfuscator OBLIQ-Bench employs:

• Runtime cost model: $T(n,w)\approx c\,n\sqrt{w}$ (where $c$ abstracts hardware effects).
• Space metric: Approximately 8 bytes per gate for large-$w$ BPW circuits.
• Security formalism: Adopts the IND-iO security game; adversarial advantage must be negligible in $w\geq50$0.

Retrieval OBLIQ-Bench evaluates systems via:

• Retrieval system classes: BM25, dense dual-encoders, late-interaction models, agentic multi-hop pipelines, and oracle LLM rerankers.
• Metrics: Recall@k, NDCG@k, Precision@k; gold relevance is established through $w\geq50$1 annotation, with optional 3-point scales for some tasks.
• Retrieval–verification gap: Defined as $w\geq50$2 (best verifier vs. best retriever on task $w\geq50$3), revealing the system's bottleneck in surfacing relevant candidates.

5. Empirical Findings and Systematic Gaps

OBLIQ-Bench experiments on functional obfuscation confirm:

• $w\geq50$4 scaling holds across hardware with $w\geq50$5 deviation.
• GPU vs. CPU speedup is a constant factor, preserving scaling exponent.
• Hypotheses (e.g., linearity in $w\geq50$6, scaling in $w\geq50$7, power-throttling-induced regime changes) are empirically validated.

Retrieval-focused OBLIQ-Bench exposes:

Task	R_t (Retriever)	V_t (Verifier)	gap(t)
Twitter-Conflict	≈0.132	≈0.436	≈0.304
WildChat Errors	≈0.113	≈0.431	≈0.318
Math Meta-Program	≈0.207	≈0.329	≈0.122 / 0.473*
Writing-Style	≈0.164	≈0.515	≈0.351
Congress Hearings	≈0.185	≈0.957	≈0.772

*Meta-Program with solutions available to verifier: gap rises to ≈0.473.

• Even advanced retrievers (e.g., Gemini-2-Embedding) surface only a small fraction of relevant documents in oblique tasks, whereas LLM rerankers recognize latent relevance given a broad enough pool.
• For tip-of-the-tongue retrieval, LateOn (token-level) outperforms others but lags far behind verification upper bound.
• Multi-hop pipelines plateau, suggesting limitations in pipeline depth.

6. Limitations, Implications, and Future Directions

For functional obfuscation:

• The bounded-program-width regime is currently limited to combinatorial Boolean circuits; extending to streaming/sequential cases requires BPW v0x02+.
• This suggests that current cost models may evolve under non-CPU-bound or power-limited settings (e.g., $w\geq50$8).

For retrieval:

• Dominant challenge is the failure of scalable retrievers to expose latent document attributes at search time; the latent signal is available only to verifier models or after LLM-based re-labeling.
• Proposed directions include precomputing $w\geq50$9 embeddings, hybrid retrieval-reasoning architectures, contrastive dense retriever training on latent properties, multi-vector late-interaction, and hierarchical attribute-classification-first retrieval.
• A plausible implication is that retrieval architectures must transition from surface-similarity paradigms toward pipelines that incorporate attribute-centric inference and joint representation.

7. Cross-Domain Significance and Adoption

OBLIQ-Bench, in both software security and retrieval settings, delivers:

• Reproducible benchmarks and established metrics for critical, unsolved problems (provably-secure obfuscation; latent-query retrieval).
• Tools, file formats, and open-source resources designed for extensibility and community-driven evolution (Thomborson, 2015).
• Explicit construction pipelines allowing others to create new tasks, gold annotations, and performance measurements in challenging domains.

OBLIQ-Bench is positioned as a rigorous standard for evaluating extremes in both security-hardness and semantic retrieval generalization, and is actively used to motivate and measure the next generation of algorithms targeting latent, abstract, or implicit problem regimes (Thomborson, 2015, Tchuindjo et al., 7 May 2026).