---
title: 'PoliAnalyzer: Personalized Privacy Analysis'
url: https://www.emergentmind.com/topics/polianalyzer
type: topic
---

# PoliAnalyzer: Personalized Privacy Analysis

Searching arXiv for the primary paper and closely related privacy-policy analysis work.
PoliAnalyzer is a neuro-symbolic system for personalized automated analysis of privacy policies. It is designed to compare formalized user data-sharing preferences against the natural-language privacy policies of websites or platforms, and to return a compliance report that identifies which policy segments are compatible with those preferences and which segments conflict with them. In the formulation described by the paper, the system combines LLM-based information extraction with deterministic logical inference: off-the-shelf NLP tools extract structured data-usage practices from policy text, those practices are converted into a formal app-policy representation, and a symbolic reasoner compares that representation against user preferences encoded as data policies [2507.14214].

## 1. Definition and problem setting

PoliAnalyzer addresses a specific bottleneck in privacy governance: privacy policies are long, natural-language documents, while the questions that matter to an individual are usually narrow and preference-dependent, such as whether a service shares location data with third parties or uses data for advertising. The system therefore targets personalized analysis rather than generic summarization. Its stated aim is to let a user compare their own privacy preferences against the privacy policy of a website or platform automatically and at scale [2507.14214].

The paper frames the problem as both informational and operational. The difficulty is not only that privacy policies are lengthy, but that users typically need only a small subset of the document: the passages that violate their expectations. PoliAnalyzer accordingly does not attempt to produce a one-size-fits-all rubric. Instead, it constructs a user-specific compliance/conflict report that points to concrete policy segments and preserves the original policy text for inspection. This suggests a shift from document-level summarization to segment-level, preference-conditioned auditing.

The scope of the system is limited to stated policy text. It analyzes what privacy policies say about data practices; it does not claim to verify whether real-world platform behavior matches those statements. That boundary is central to the system’s interpretation: PoliAnalyzer is a policy-text analyzer, not a behavioral measurement system [2507.14214].

## 2. System architecture and workflow

The system is organized into three major components: an NLP pipeline, a privacy policy converter, and a user-preference evaluator. On the policy side, the input is raw privacy policy text. On the user side, the input is a set of user preferences encoded formally as data policies. The output is a compliance analysis report describing which policy segments conflict with which user preferences [2507.14214].

The workflow begins by segmenting the privacy policy. In the prototype, each line of the privacy policy is treated as a segment, a choice that the paper explicitly describes as a balance between extraction accuracy and cost after pilot experiments. Each segment is then processed through a decomposed LLM-based information extraction pipeline. The subtasks are data entity recognition, data entity classification, purpose entity recognition, purpose entity classification, action recognition, party recognition, and relation identification. For data and purpose extraction, the pipeline separates recognition from classification: it first identifies the mention span and then maps that span to a canonical label.

The extracted entities and relations are grouped by segment and converted into data practices. These practices are then used to construct both an internal knowledge graph and a formal app-policy representation in an extended version of psDToU. The user-preference side is modeled separately: preferences are encoded as data policies, and multiple data policies combine into a user profile. A symbolic reasoner based on Notation3 then compares the app policy against the user’s data policies and computes conflicts. The final report identifies the segments that conflict with the selected preference profile and includes details such as the original policy text [2507.14214].

This architecture is explicitly hybrid. LLMs are used for text understanding and structured extraction, while the compliance decision itself is delegated to symbolic logic. The paper presents this division as a deliberate response to concerns about logic errors, explainability, and auditability in fully neural compliance reasoning.

## 3. Formal representation and compliance semantics

A central contribution of PoliAnalyzer is its formalization bridge between privacy policy text and machine-checkable compliance analysis. The system builds on psDToU, described as “perennial semantic Data Terms of Use,” and extends it for privacy-policy analysis. Within this framework, a platform’s privacy policy is represented as an app policy, while user preferences are represented as data policies [2507.14214].

The formal representation of a data practice includes at least the following dimensions: data type (`:data`), application port or identifier (`:port`), downstream recipients or third parties (`:downstream`), purpose (`:purpose`), action or data-practice type, and party or actor. Collection and use practices are mapped to input specifications containing `:data` and `:purpose`. Third-party sharing or disclosure practices are mapped to `:downstream` for the same data, together with the corresponding recipient and purpose.

A specific modeling choice is the use of DPV, the Data Privacy Vocabulary, for canonical data and purpose labels. This provides interoperable names and, more importantly, hierarchy-aware matching. The paper states that PoliAnalyzer extends the language and reasoner so that data policies can specify how matching against a hierarchy should work, such as exact match or matching all subclasses. Users may also define custom hierarchies in RDF by specifying subclass relations. This makes the comparison stage sensitive not only to string equality but also to semantic inclusion relations.

The paper does not present a single symbolic rule for conflict, but its operational semantics are clear. A policy segment is in conflict when the data-use practice encoded in the corresponding app-policy fragment is not compliant with the relevant user preference encoded as a data policy. Matching is driven by data type, purpose, and recipient or consumer class, with DPV hierarchy reasoning used to detect equivalence and superclass or subclass relations. A plausible implication is that the formal layer is intended to support explainable mismatch detection at the level of semantically typed practices rather than raw text spans.

## 4. NLP pipeline and dataset construction

PoliAnalyzer’s extraction layer is built around off-the-shelf LLMs, specifically the gpt-4o family in the prototype, used in a decomposed pipeline rather than a single prompt. Each task uses a system prompt describing the task and required JSON schema, a user prompt containing the policy segment, and stable decoding settings, especially `temperature = 0`. The paper also states that the models were fine-tuned for each task using a small amount of training data, primarily to improve adherence to the required output schema and to capture simple preferences that are difficult to specify through instructions alone [2507.14214].

The extraction tasks combine span extraction, entity normalization, and relation extraction. Relation identification differs slightly from the other tasks: each entity and practice is assigned a unique ID in code, and these jointly identified items are then passed to the model. Because LLM outputs can be malformed, the pipeline includes post-processing heuristics and tools such as `json-repair` before parsing.

Training and evaluation use an enriched Policy-IE dataset curated with domain-expert input. Policy-IE originally contains 31 documents with annotations for privacy-practice information such as party, action, entities, and relations. The enriched version adds canonical DPV labels for data and purpose entities and splits cases where one sentence contains two logically distinct practices even if they share the same action word. The paper reports 1087 data points in the enriched dataset, with 120 out of 1087 used for training or fine-tuning.

The evaluation uses a relaxed metric based on longest common substring ratio for tasks involving word matching. The reported metric suite includes `f1_n` for non-empty segments, `f1_e` for empty segments, `f1` as macro F1, and `rx` for relaxed matching. The abstract states that PoliAnalyzer achieved F1-score of 90–100% across most tasks, and the body text adds that best-performing models were often around 0.9 or above in macro F1, with especially high performance on empty segments and lower performance, roughly 0.5–0.7, on non-empty segments where relevant content is actually present [2507.14214].

These results delimit the strengths and weaknesses of the NLP stage. The system appears effective at avoiding hallucinated extractions in irrelevant segments, but extraction remains harder precisely where the policy contains meaningful, densely packed practices. The paper therefore treats the symbolic layer not as a replacement for accurate NLP, but as a way to make downstream reasoning deterministic once sufficient structure has been extracted.

## 5. User profiles, reasoning, and large-scale analysis

The reasoning layer is symbolic, rule-based, deterministic, and implemented in N3 logic. Its inputs are an app policy derived from a platform’s privacy policy and a user profile derived from one or more data policies. The comparison process is straightforward: convert each website policy into an app policy, encode a profile as data policies, run the N3 reasoner, and record the segments or practices that conflict [2507.14214].

Because the paper did not rely on a ready-made dataset of real-world user data-sharing preferences, it synthesized user preferences from prior literature. It reports identifying 23 distinct data types discussed in the literature, of which 15 are represented in DPV; 11 purpose types, of which 8 are in DPV; and 14 practice types. From these, the authors constructed 23 data policy sets spanning 7 data types, 6 purposes, and 2 data-consumer categories. The factors explicitly listed are data types `SocialCommunication`, `Contact`, `Data-general`, `MedicalHealth`, `Identifying`, `Location`, and `Picture`; purposes `Internal`, `Advertisement`, `Analytics`, `Research`, `SNS`, and `ProtectionOfPublicSecurity`; and data consumers `1st-party-only` and `1st-and-3rd-party`.

The paper names two profiles that recur in the analysis: `data-ad-3rd-no`, which denotes a profile disallowing data sharing with third parties for advertising, and `location-3rd-no`, which denotes a profile disallowing sharing location data with third parties for any purposes. These examples illustrate the form of the profile language: it captures restrictions over data type, purpose, and recipient class rather than only coarse, document-level preferences.

For aggregate analysis, the paper defines two violation-rate metrics. The first is

$$
R_{pp}  = \frac{1}{|W|} \sum_{w \in W} R^{w}_{pp} \quad \text{where} \quad R^{w}_{pp} = N^{w}_{con} / N^{w}_{pp}
$$

and the second is

$$
R_{cs}  = \frac{1}{|W|} \sum_{w \in W} R^{w}_{cs} \quad \text{where} \quad R^{w}_{cs} = N^{w}_{con} / N^{w}_{cs}.
$$

Here, \(W\) is the set of websites in a violation group, \(N^{w}_{con}\) is the number of conflicts for website \(w\), \(N^{w}_{pp}\) is the number of privacy-policy segments for website \(w\), and \(N^{w}_{cs}\) is the number of segments of website \(w\) that trigger some conflicts. The paper interprets \(R_{pp}\) as the likelihood that a privacy-policy segment creates a conflict, and \(R_{cs}\) as the average number of conflicts generated by each violating segment.

At scale, the system was applied to the top 100 most visited websites, selected from the Tranco list, and compared against the 23 user profiles, yielding 2300 profile-policy analyses. Across these websites, the paper reports 13205 total privacy-policy segments, 3421 segments containing valid practices, 636 segments that demonstrate conflicts, and 4083 conflicts across different profiles. The headline result is that, on average, 95.2% of a policy’s segments do not conflict with the analyzed user preferences, leaving 4.8% of segments—636 out of 13205—as the subset that violates those preferences [2507.14214].

This numerical result is the system’s main practical claim. It suggests that a user-oriented analyzer can substantially reduce cognitive burden by filtering a privacy policy down to the passages that matter for a particular preference profile. The paper presents this as the operational meaning of “personalized privacy policy analysis at scale.”

## 6. Findings, significance, and limitations

The substantive findings reported by PoliAnalyzer concern recurring patterns of conflict between platform privacy policies and literature-derived user expectations. The strongest recurring finding is related to location data. The paper states that, aside from a few isolated cases, the main source of conflict is related to location data, and that the vast majority—70%—of websites send location data to third parties for processing, either without stated purposes or with purposes the system could not recognize [2507.14214].

Another prominent pattern concerns `data-ad-3rd-no`. Violations of this profile indicate that many platforms share data with third parties for advertising purposes without specifying the data type, or that the system could not recognize the type. The paper reports that these cases average 12 conflicts per policy, substantially above the more typical 1–5 conflicting practices mentioned elsewhere. It interprets this as evidence that many websites do not clearly detail what kinds of data are shared for third-party advertising.

More generally, the reported conflict structure is organized by recipient, purpose, and data type. Third-party sharing is a major source of conflict; advertising is especially problematic as a purpose; and location stands out as a particularly conflict-prone data type. The paper also describes exploratory auditing beyond the original literature-derived profiles: by constructing a new profile for location data, the authors discovered an additional purpose, `RecordManagement`, that had not appeared in the user-preference literature they reviewed.

The paper explicitly presents several strengths. These include accessibility and scalability through off-the-shelf NLP, interoperability through DPV grounding and psDToU representation, auditability through deterministic logical reasoning, reusability of user profiles across many websites, and practical burden reduction by focusing attention on a small conflict-causing subset of policy text. It also makes explicit novelty claims: first toolkit for generating formal data-usage policies from privacy policy texts, first system for automated personalized analysis of privacy policies, and a large-scale assessment of the top 100 websites showing that on average 4.8% of policy segments violate user expectations [2507.14214].

Its limitations are equally clear. Extraction errors remain significant, especially on non-empty segments; some zero-conflict cases may reflect missed extraction rather than genuine compliance. The extracted schema and formal model do not cover everything that may matter in privacy policies. The 23 profiles are synthesized from prior literature rather than collected directly from users in a dedicated dataset. The review used to derive them is not presented as a systematic review. The paper also notes the need for better user ergonomics, including easier preference specification and graphical explanation, and it treats the current architecture as a feasibility demonstration using off-the-shelf LLMs rather than an optimized deployment stack.

Taken together, these points position PoliAnalyzer as a foundation for machine-assisted privacy-policy inspection rather than a finished universal compliance oracle. Its central methodological significance lies in the combination of LLM-based extraction and symbolic compliance reasoning: a neuro-symbolic architecture that turns privacy-policy analysis into a structured comparison between app policies and data policies, with segment-level traceability and reusable user profiles [2507.14214].

Source: https://www.emergentmind.com/topics/polianalyzer