NLP-based Automated Compliance Checking of Data Processing Agreements against GDPR

Abstract: Processing personal data is regulated in Europe by the General Data Protection Regulation (GDPR) through data processing agreements (DPAs). Checking the compliance of DPAs contributes to the compliance verification of software systems as DPAs are an important source of requirements for software development involving the processing of personal data. However, manually checking whether a given DPA complies with GDPR is challenging as it requires significant time and effort for understanding and identifying DPA-relevant compliance requirements in GDPR and then verifying these requirements in the DPA. In this paper, we propose an automated solution to check the compliance of a given DPA against GDPR. In close interaction with legal experts, we first built two artifacts: (i) the "shall" requirements extracted from the GDPR provisions relevant to DPA compliance and (ii) a glossary table defining the legal concepts in the requirements. Then, we developed an automated solution that leverages NLP technologies to check the compliance of a given DPA against these "shall" requirements. Specifically, our approach automatically generates phrasal-level representations for the textual content of the DPA and compares it against predefined representations of the "shall" requirements. Over a dataset of 30 actual DPAs, the approach correctly finds 618 out of 750 genuine violations while raising 76 false violations, and further correctly identifies 524 satisfied requirements. The approach has thus an average precision of 89.1%, a recall of 82.4%, and an accuracy of 84.6%. Compared to a baseline that relies on off-the-shelf NLP tools, our approach provides an average accuracy gain of ~20 percentage points. The accuracy of our approach can be improved to ~94% with limited manual verification effort.

• The paper presents an NLP-based methodology termed DERECHA that automates the compliance checking of DPAs against GDPR requirements.
• It uses semantic frame representation to break down legal texts into actionable semantic roles, enabling precise matching against regulatory criteria.
• Evaluation on 30 DPAs demonstrated high efficacy with an average precision of 89.1% and recall of 82.4%, significantly reducing manual processing time.

NLP-based Automated Compliance Checking of Data Processing Agreements against GDPR

The paper presents a methodology leveraging NLP for automated compliance checking of Data Processing Agreements (DPAs) against the General Data Protection Regulation (GDPR). The proposed approach aims to streamline the otherwise manual and error-prone process of ensuring legal compliance, which is critical given the complex and often convoluted language of legal documents.

Semantic Frame-based Representation

Semantic frames (SFs) are utilized to capture the necessary elements of each compliance requirement in GDPR. The methodology involves defining SFs for each requirement concerning the processing of personal data, using semantic roles (SRs) such as actor, action, object, and condition. These frames decompose requirements into meaningful phrases that can be directly compared to corresponding elements in DPAs.

Figure 1: Illustration of SF-based representation.

DPA Compliance Checking Approach

The compliance checking approach, referred to as DERECHA, consists of several pivotal steps. Initially, SF-based representations for GDPR compliance requirements are manually created. Subsequently, the textual content of DPAs is processed through an NLP pipeline to automatically generate corresponding SF-based representations.

Figure 2: Overview of our DPA compliance checking approach (DERECHA).

Text Preprocessing and Semantic Frame Generation

Each DPA is subjected to tokenization, part-of-speech tagging, and dependency parsing to prepare the data for SF generation. Custom extraction rules are then applied to translate these parsing results into SRs for each DPA statement. The enriched SR labels enable precise matching against the predefined SF representations of GDPR requirements.

Matching and Compliance Decision

Compliance checking involves matching predicates and arguments of SF-based representations between GDPR requirements and DPA statements. Checking begins by ensuring sufficient predicate similarity, followed by detailed argument matching based on text span overlap and semantic similarity. The approach calculates a "matching degree" score for each requirement, aiding in confidence estimation regarding compliance decisions.

Evaluation and Results

The evaluation involved 30 DPAs covering a wide variety of sectors and services. The results demonstrate the approach's efficacy, achieving an average precision of 89.1% and a recall of 82.4%. The introduction of confidence scores enhances the decision-making process, allowing for targeted manual verification with minimal effort while still effectively increasing accuracy.

Efficiency Considerations

DERECHA's efficiency is highlighted by its capability to analyze an average-sized DPA in approximately 2.5 minutes. This performance starkly contrasts with the labor-intensive manual compliance checking that is typical today, enhancing its practical appeal for real-world implementation where time and resource constraints are a constant concern.

Conclusion

The methodology presents a viable solution to automate the compliance checking of DPAs against GDPR requirements, significantly reducing the time and effort involved in ensuring legal compliance. The use of SFs and NLP techniques demonstrates considerable promise for scalability to other regulatory contexts and document types. Future work may explore extending the dataset and enhancing model training techniques to further refine the approach's accuracy and applicability across domains.