PROVSEEK: Provenance Analysis & Query

• PROVSEEK is a methodology that reconstructs and explains derivational chains between artifacts and their origins using dependency-based provenance graphs.
• It supports both backward and forward analysis for cyber attack investigations by employing constrained graph traversal, formal query models, and efficient indexing techniques.
• The approach spans domains from cybersecurity to astronomy and multi-agent systems, offering scalable, declarative query languages and augmented analytic frameworks.

PROVSEEK denotes provenance-seeking: the reconstruction, querying, and explanation of derivational or causal chains that connect an observed artifact to the entities, activities, agents, and intermediate events that produced it. In cyber attack investigation, the term is defined as the analyst-driven process of reconstructing and interrogating causal chains of system activities to explain how an alert or point of interest arose and what it affected; in adjacent lines of work, closely related provenance-seeking mechanisms appear in provenance-aware graph search, astronomy data management, SPARQL query evaluation, and multi-agent analytic validation (Shao et al., 25 Oct 2025, Mukherjee et al., 29 Aug 2025, Sanguillon et al., 2018, Damásio et al., 2012, Friedman et al., 2020).

1. Conceptual scope and provenance substrate

At its core, PROVSEEK operates on provenance structures that encode dependency, lineage, or derivation. In system security, provenance analysis organizes system audit events into a provenance graph $G(E,V)$ in which nodes $v \in V$ are system entities such as processes, files, and network connections, and directed edges $e(u,v) \in E$ are system events whose direction indicates data flow. Each edge carries a time window with start $ts(e)$ and end $te(e)$, and dependency between events $e_1(u_1,v_1)$ and $e_2(u_2,v_2)$ exists when $v_1=u_2$ and $ts(e_1)<te(e_2)$. This directly supports backward provenance analysis, which searches causally upstream of a point-of-interest event, and forward provenance analysis, which searches causally downstream (Shao et al., 25 Oct 2025).

In standards-oriented provenance systems, the substrate is usually expressed through the W3C PROV model and compatible extensions. The astronomy lineage stack centered on the IVOA Provenance Data Model adopts the same class names as W3C PROV—Entity, Activity, and Agent—and relations including used, wasGeneratedBy, wasAssociatedWith, wasAttributedTo, actedOnBehalfOf, and wasDerivedFrom. It extends these with ActivityFlow, hadStep, and a separation between activity description and activity instance, thereby supporting reusable pipeline descriptions and job templates (Sanguillon et al., 2018).

In multi-agent analytic settings, provenance is again centered on PROV-O entities, activities, and agents, but augmented with meta-level judgments. The DIVE ontology adds Appraisal, Evidence, Preference, and Nexus, allowing provenance graphs to carry confidence, diagnosticity, coherence, and relative preference information in addition to derivational structure (Friedman et al., 2020).

A recurrent misconception is that provenance is merely archival metadata. The literature instead treats it as an executable analytic substrate: a structure against which one performs constrained traversal, lineage extraction, counterfactual refutation, confidence propagation, and domain-specific explanation (Shao et al., 25 Oct 2025, Friedman et al., 2020).

2. Formal representations and provenance semantics

PROVSEEK methods differ in interface and domain, but they share a concern with formally specifying how derivations compose. In cyber provenance analysis, system audit events are modeled as tuples such as $\langle sub, op, obj \rangle$ or $v \in V$0, where [sub](https://www.emergentmind.com/topics/multi-epoch-sub-millimeter-interferometry) is a process, obj is a process, file, or network entity, and op is a system-call semantic such as read, write, execve, clone, sendmsg, or recvmsg. Provenance queries therefore reason over typed vertices, typed edges, timestamps, and structural properties such as in-degree and out-degree (Shao et al., 25 Oct 2025).

For SPARQL, provenance-seeking is formalized algebraically through annotated relations over the most general $v \in V$1-semiring, enabling how-provenance for bag semantics, including non-monotonic constructs. The fundamental operations are semiring addition and multiplication together with monus $v \in V$2, a generalized subtraction. Join provenance is computed by product, union provenance by sum, projection by summation over collapsed bindings, and difference or MINUS by monus. The paper’s treatment of OPTIONAL is especially significant: for a left tuple $v \in V$3, matched provenance is $v \in V$4, while the unmatched residual is $v \in V$5, yielding the combined expression

$v \in V$6

This refutes prior claims that OPTIONAL could not be captured appropriately in abstract provenance models under SPARQL 1.1 multiset semantics (Damásio et al., 2012).

Astronomy provenance uses a different formal emphasis. There, recursive lineage for an entity $v \in V$7 is written as

$v \in V$8

which captures transitive upstream inputs and activities. The same paper also provides recursive SQL via PostgreSQL WITH RECURSIVE queries and recommends B-tree indexes on relation tables such as used(activity_id, entity_id) and wasGeneratedBy(entity_id, activity_id) for practical lineage retrieval (Sanguillon et al., 2018).

In multi-agent interpretation, formalism shifts from derivation alone to derivation plus appraisal. Confidence can be propagated over provenance paths by conservative minimum, average, or noisy-OR-style aggregation, while counterfactual sensitivity is represented through TMS-style environments. The central idea is that provenance is not only a record of what happened but also a basis for computing how confidence changes when one removes sources, agents, or operation classes (Friedman et al., 2020).

3. Dependency-centric query languages and the ProGQL operationalization

The most explicit operationalization of PROVSEEK appears in "ProGQL: A Provenance Graph Query System for Cyber Attack Investigation" (Shao et al., 25 Oct 2025). ProGQL extends Cypher with domain-specific operators for constrained BFS/DFS, edge weight computation, value propagation, and graph merging, so that provenance analysis, analyst constraints, expert knowledge, and scoring can be jointly expressed in a single query.

Construct	Function	Representative form
bfs / dfs	backward or forward dependency search	`BFS (r IN backward(f)
projection	edge weight computation	SET e.weight=projection(...)
reduce	node-value propagation	`SET u.rel=reduce(sum = 0, o IN out(u)
union / intersect	graph merging	RETURN g1 intersect ... UNION (...)

The traversal semantics are dependency-centric rather than path-centric. At each expansion step, adjacent edges are pruned by a Match plus Where filter that may depend on properties of the current and neighboring edges or nodes. A representative backward constraint is WHERE r.starttime < max(collect(vout IN out(v) | vout.endtime)), and a representative forward constraint is WHERE re.endtime > min(collect(uin IN in(u) | uin.starttime)) [AND](https://www.emergentmind.com/topics/aria-navigation-dataset-and) re.starttime < .... These expressions allow recursive expansion subject to temporal constraints derived from adjacent neighborhoods, which general-purpose languages such as Cypher, SPARQL, AIQL, and SAQL cannot express declaratively in this form (Shao et al., 25 Oct 2025).

ProGQL also incorporates edge weighting and value propagation. The paper’s main query computes per-edge features from data-flow relevance, temporal relevance, and structural context: 1/(abs(r.amount-st.amount)+0.0001), ln(1+1/abs(r.endtime-st.endtime)), and count(out(v))/count(in(v)). These are combined through projection, optionally after clustering with KMeans or DBScan and projection with LDA. Node impact then propagates by

$v \in V$9

with iterative updates until the aggregate difference between consecutive iterations falls below a threshold such as $e(u,v) \in E$0 (Shao et al., 25 Oct 2025).

The evaluation establishes both expressiveness and systems consequences. In multi-host attack cases, expressing the same attack behaviors required 1 ProGQL query versus 32 Cypher queries; on average, ProGQL used 9× fewer constraints, 15× fewer words, and 17× fewer characters. On a Neo4j backend, it executed 22.76× faster on average, at 21 seconds versus 478 seconds, and used approximately 59.8% of Cypher’s memory. In Password Crack, Data Leakage, and VPN Filter, Cypher returned 21,518, 21,556, and 24,827 edges, whereas ProGQL returned 282, 281, and 637, respectively, indicating much stronger inline pruning (Shao et al., 25 Oct 2025).

4. Search architectures: incremental engines, graph embeddings, and agentic orchestration

ProGQL’s execution model is incremental rather than monolithic. Its architecture consists of an importer, a language-parsing module, and a query engine, and it synthesizes backend-specific queries for PostgreSQL, MyRocks, MariaDB, Neo4j, and Nebula. Instead of materializing entire event streams in memory, the engine performs BFS/DFS incrementally, fetching only eligible edges at each expansion step and using backend indexes for constrained neighbor retrieval. Across 14 real attacks averaging 19 million queried events, the system used on average 5 GB of memory, finished searches in 224 seconds, and produced concise provenance graphs averaging approximately 180 edges; by contrast, DepImpact averaged 38.77 GB and encountered OOM on Trace Case 5 even with a 100 GB heap (Shao et al., 25 Oct 2025).

A distinct retrieval strategy appears in ProvG-Searcher, which reframes provenance graph search as subgraph matching in an order-embedding space. It partitions global provenance graphs into process-centric $e(u,v) \in E$1-hop ego-graphs, applies behavior-preserving reduction, and learns subgraph embeddings with a 3-layer multi-relational GraphSAGE encoder using add pooling and embedding dimension $e(u,v) \in E$2. Subgraph inclusion is then tested by coordinate-wise dominance using an order-violation loss, moving most computation offline. The reported results are accuracy exceeding 99%, false positive rate approximately 0.02%, ROC AUC 96.6–98.3 for validating subgraph relations, and approximately 48 seconds to process 10,000 samples online (Altinisik et al., 2023).

The LLM-driven framework explicitly named PROVSEEK adds a third architecture: agentic, retrieval-augmented provenance forensics. It orchestrates a Threat-Intelligence Extraction Agent, Investigation-Planning Agent, Data-Retrieval Engine, Investigation Agent, Follow-up Agent, Safety Agent, and Explanation Summary Agent. The toolchain includes Provenance SQL Explorer, Artifact Lookup, Event Lookup Resolver, Type-Aware Correlator, CTI Retriever, Plan Generator, Follow-up Planner, Plan Validator, Safety Checker, and Confidence Validator. CTI reports are embedded in ChromaDB with text-embedding-ada-002; retrieved indicators are then verified against provenance databases through type-specific SQL before appearing in prompts, and only evidence tied to concrete node or edge identifiers is surfaced to analysts (Mukherjee et al., 29 Aug 2025).

Its evaluation covers intelligence extraction and threat detection on DARPA Transparent Computing datasets. The framework reports 34% gains in contextual precision and recall versus vanilla RAG, 30% gains in relevance, and 22% and 29% higher precision and recall than both a baseline agentic AI approach and the state-of-the-art provenance-based intrusion detection systems FLASH and ORTHRUS. The design is explicitly verification-first: hallucinations are mitigated through CTI grounding, type-aware SQL, duplicate collapse, scoped matching, step budgets, blacklists, and confidence validation, though the paper notes that hallucination and overgeneralization are mitigated rather than eliminated (Mukherjee et al., 29 Aug 2025).

5. Cross-domain provenance services and interpretive systems

Outside cybersecurity, PROVSEEK-like capabilities have been implemented as provenance production, retrieval, storage, and visualization services. In astronomy, the IVOA Provenance Data Model aligned with W3C PROV is exercised in Pollux, CTA, RAVE, and CDS. The voprov Python library serializes provenance in PROV-N, PROV-JSON, PROV-XML, and VOTable, and renders graphs as PDF, PNG, and SVG. django-prov_vo maps IVOA provenance classes to relational tables and exposes REST plus ProvDAL endpoints returning PROV-N and PROV-JSON. A CDS PostgreSQL prototype implements the IVOA schema, returns PROV-N, PROV-JSON, and PROV-VOTable, and informs a preliminary ProvTAP design; OPUS emits provenance per job in PROV-JSON and PROV-XML for CTA workflows (Sanguillon et al., 2018).

This astronomy stack demonstrates that provenance-seeking need not be restricted to incident response. It can also serve standardized scientific lineage, with user-selectable detail levels, VO discovery through SSA and DataLink, and transitive closure queries over archived data products. A plausible implication is that the distinction between “forensics” and “data lineage” is largely a matter of query intent rather than provenance substrate (Sanguillon et al., 2018).

In multi-agent information analysis, provenance is used to inspect analytic integrity rather than system events. PROV-O captures the generation, use, derivation, association, and attribution relations, while DIVE appraisals and evidence links support dynamic confidence propagation and counterfactual analysis. The representative scenario around the claim “Lady Ada located in USA” shows how multiple derivational paths—GEOINT, Shipping News International, and Twitter—can be isolated, reweighted, or refuted. Under noisy-OR aggregation, removing a low-confidence SNI document has minor effect, whereas refuting the SELF-REPORT source class materially reduces the claim’s confidence (Friedman et al., 2020).

6. Evaluation themes, limitations, and open directions

Three themes recur across the literature. First, provenance-seeking is fundamentally a scalability problem. Security provenance graphs can span tens of millions of events; SPARQL provenance expressions can grow symbolically; archive lineage services must answer recursive queries over relational stores; and multi-agent provenance can become visually and computationally unwieldy. The corresponding responses are incremental indexed traversal, offline embedding and reduction, bounded agentic planning, and selective graph materialization (Shao et al., 25 Oct 2025, Altinisik et al., 2023, Mukherjee et al., 29 Aug 2025, Sanguillon et al., 2018).

Second, provenance expressiveness does not guarantee faithful explanation unless the query model is sufficiently rich. ProGQL addresses inline recursive constraints, weighted propagation, and graph merging; SPARQL provenance requires $e(u,v) \in E$3-semirings to account for OPTIONAL, MINUS, and bag semantics; multi-agent analysis requires appraisal and evidence ontologies beyond raw lineage; and astronomy services require workflow-level constructs such as ActivityFlow and hadStep (Shao et al., 25 Oct 2025, Damásio et al., 2012, Friedman et al., 2020, Sanguillon et al., 2018).

Third, provenance is only as reliable as its capture and interpretation pipeline. ProGQL notes sensitivity to weight choice, backend trade-offs, and the absence of parallelism; ProvG-Searcher assumes log integrity and may lose multiplicity information under aggressive reduction; LLM-driven PROVSEEK assumes trusted audit pipelines and does not claim prompt-injection robustness; astronomy provenance tooling notes that ProvTAP remains preliminary and that RDF support is not claimed by the implementations described; multi-agent provenance analysis depends on sound and complete logging at the correct granularity (Shao et al., 25 Oct 2025, Altinisik et al., 2023, Mukherjee et al., 29 Aug 2025, Sanguillon et al., 2018, Friedman et al., 2020).

Taken together, these works characterize PROVSEEK not as a single product but as a research program: provenance as a queryable, verifiable, and domain-adaptable substrate for explaining how outputs, alerts, conclusions, or data products came to be. The direction of travel is toward declarative constraint-rich search, hybrid symbolic and learned retrieval, standardized lineage interchange, and evidence-backed explanation rather than raw graph exposure alone (Shao et al., 25 Oct 2025, Mukherjee et al., 29 Aug 2025).