Python Pickle Files & Secure Deserialization

• Python pickle files are serialized object graphs produced by Python’s pickle module, enabling the preservation of both primitive and complex user-defined objects.
• They leverage a stack-based virtual machine and the __reduce__ hook to reconstruct full object states, being integral in ML workflows despite inherent security vulnerabilities.
• Security challenges such as arbitrary code execution during deserialization are mitigated by defenses like PickleBall, which enforce strict per-library and per-class load policies.

Python pickle files are serialized object graphs produced by Python’s “pickle” module, which implements a stack-based virtual machine capable of storing and reconstructing nearly arbitrary Python objects. The pickle format’s flexibility enables the faithful preservation and recovery of both primitive and user-defined types and arbitrary object structures, features that have led to its widespread adoption in ML development pipelines. However, this expressive capability exposes critical supply chain attack surfaces: by executing user-specified dynamic import and invocation opcodes, pickle deserialization is a direct vector for arbitrary code execution if attacker-controlled files are loaded. Rigorous evaluation on production-scale datasets demonstrates that, as of 2025, pickle remains the de facto serialization standard in ML repositories despite well-known security shortcomings (Kellas et al., 21 Aug 2025). Recent research, exemplified by the introduction and quantitative analysis of PickleBall, provides per-library, per-model-class load policies as a robust means to mitigate these risks, delivering substantial improvement over both static model scanners and weights-only unpicklers.

1. Design Principles and the Pickle Format

The pickle format leverages a small, stack-based interpreter called the Pickle Machine (PM). Each pickle file is a sequence of opcodes that manipulate a VM stack to push and pop values, import or invoke user-specified callables, and reconstruct arbitrary Python object graphs. Two core features underpin this flexibility:

• The ability to serialize virtually any Python object, including user-defined classes, complex containers, and native types.
• The per-class __reduce__ hook, which allows library authors to specify arbitrary Python code ($f, args, state$) necessary to reconstruct an object: f(args) re-instantiates the object; state payloads further rehydrate object attributes.

In standard ML workflows, training scripts serialize a model via pickle.dump(model), producing e.g. model.pkl for sharing or deployment. Consumers then execute pickle.load("model.pkl"), instantly recreating a live model instance, with full class and call graph fidelity. PyTorch, notably, adopts pickle as its default serializer for .pt and .pth files, reinforcing industry-wide inertia toward this format and sidestepping the need for architecture-specific bespoke serialization logic (Kellas et al., 21 Aug 2025).

2. Security Vulnerabilities in Untrusted Deserialization

Untrusted pickle deserialization directly enables arbitrary code execution, the primary attack primitive. Within a pickle file, opcodes such as GLOBAL or STACK_GLOBAL can resolve and import any Python callable, while REDUCE can invoke it with arbitrary arguments. Attackers can encode, for example, GLOBAL os.system; REDUCE('rm -rf /'), enabling file system erasure or data exfiltration as part of the deserialization itself. The presence of these primitives in the pickle VM means any untrusted pickle is potentially a code execution payload.

For evaluating defenses, common metrics include true/false positives/negatives (TP/FP/TN/FN), false positive/negative rates (FPR, FNR), and the proportion of benign models that load successfully (“load success rate”) (Kellas et al., 21 Aug 2025).

3. Existing Defensive Measures and Their Limitations

Defensive approaches against pickle-based attacks fall into three main categories:

• Safer Model Formats: Alternatives such as SafeTensors (memory-mapped, weights-only) and GGUF restrict serialization to raw numeric tensor data and are formally audited as non-executable. However, these formats cannot capture custom model logic or metadata, limiting their applicability to use cases requiring full object graphs. ONNX and TensorFlow SavedModel are more expressive but have demonstrated vulnerability to attacks via exotic operator overloading. Notably, as of March 2025, approximately 45% of popular Hugging Face repositories include pickle models, with pickle-only repositories accounting for 400 million downloads monthly (Kellas et al., 21 Aug 2025).
• Model Scanners: Tools like picklescan (Hugging Face), ProtectAI, and ModelTracer parse pickles using deny-lists of disallowed callables (e.g., os.system, subprocess.Popen). Empirical evaluation highlights key limitations: static deny-lists cannot exhaustively enumerate all attack-relevant primitives (e.g., via indirect imports), and dynamic tracing can miss attacks buried in subsequent deserialization layers. On a dataset of 336 models, picklescan achieved FPR ≈ 6.3%, FNR ≈ 10.7%; ModelTracer had FPR = 0%, FNR ≈ 47.6% (Kellas et al., 21 Aug 2025).
• Restrictive Loading Policies: PyTorch’s “weights-only” unpickler permit-list approach restricts deserialization to core tensor constructors and select functions, enhancing security but causing incompatibilities: 15% of popular pickle repositories cannot load under this policy, resulting in a load failure rate of 79.6 million monthly downloads (Kellas et al., 21 Aug 2025). Manual policy extensions are burdensome and rigid, and non-tensor metadata support is lost.

Approach	FPR	FNR	Benign Load Success	Key Limitation
PickleBall	20.2%	0%	79.8%	Static analysis under-approximation
Weights-only Unpick.	37.6%	0%	62.3%	Rigid, loses metadata
picklescan	6.3%	10.7%	—	Deny-list incompleteness
ModelTracer	0%	47.6%	—	High false negative rate

4. PickleBall: Source-Driven Policy Generation and Enforcement

PickleBall represents a two-phase defense—a per-library/per-class policy generator and a hardened drop-in loader (Kellas et al., 21 Aug 2025):

1. Offline Policy Generation: Leveraging the Joern code-property-graph toolchain extended for Python, PickleBall performs static analysis on the ML library’s source and the target ModelClass. For each class, it tracks all reachable constructors and callables (direct and transitive), with special handling for user-defined __reduce__ hooks to identify instantiation functions. The resultant policy enumerates two sets: AllowedImports and AllowedInvocations (i.e., which qualified names can be imported and called during deserialization).
2. Dynamic Enforcement: A forked PickleMachine interpreter replaces GLOBAL/STACK_GLOBAL with policy lookups—allowed names are resolved; others yield stub objects that raise pickleball.SecurityError on use. REDUCE/invocation opcodes are similarly checked. Enforcement is “lazy”: loading proceeds as long as disallowed functions are not actually invoked, enhancing benign model compatibility.

Pseudocode for policy generation:

Let AllowedImports ← ∅ Let AllowedInvocations ← ∅ Let Worklist ← [ModelClass] While Worklist not empty do C ← pop(Worklist) if C implements __reduce__ then (f, args, state) ← analysis of C.__reduce__ AllowedImports ∪= {f} AllowedInvocations ∪= {f} for each argument type T in args ∪ state: enqueue T in Worklist else AllowedImports ∪= {C} for each subclass S of C: enqueue S for each attribute A declared in C: let T = type(A); enqueue T end while

Result: Two sets of qualified names for imports and invocations during secure load time (Kellas et al., 21 Aug 2025).

5. Evaluation: Security, Compatibility, and Overheads

Quantitative assessment utilizes a benchmark of 336 pickle-based models (252 benign, 84 malicious, spanning 16 ML libraries, real-world and synthetic malware):

• Detection Accuracy: PickleBall achieved TP=84/84, TN=201/252, FP=51, FN=0, corresponding to FPR=20.2%, FNR=0%. All malicious models were blocked; 79.8% of benign models loaded. For comparison, weights-only unpickler loads only 62.3% of benign models, and picklescan/ModelTracer suffered from high FNR or FPR, as documented above.
• Performance: Policy generation per library is a one-time, offline cost (median 14 s; range 9–30 s). Median load-time overhead is +1.75% (0.42 ms per load), with worst-case slowdown ≈2.62× on select models (Kellas et al., 21 Aug 2025).
• Workflow Integration: Installing and activating PickleBall is designed as a drop-in operation:

  from pickleball import PolicyGenerator policy = PolicyGenerator().with_library_source("/path/to/library").for_class("module.submodule.ModelClass").generate() import pickleball as pickle from pickleball import enforce_policy enforce_policy(policy) model = library.load("model.pkl") result = model.predict(x)

  Security violations result in explicit exceptions, detailing the disallowed import or invocation.

6. Limitations, Trade-offs, and Future Prospects

While PickleBall significantly raises the bar for model supply chain threats, key limitations remain:

• Static Analysis Soundness: Incomplete modeling of dynamic Python features (attribute writes, C-extensions, generics) can result in omission of legitimate callables, leading to stub-injection. PickleBall’s lazy enforcement mitigates failure rates for many benign models but under-approximation remains. Over-approximation may broaden the allowed set, slightly increasing the residual attack surface. Prospective improvements to AST and type recovery, and leveraging docstring-driven annotations, are suggested mitigations.
• Residual Attack Surface: Even with correct policies, attackers might chain permitted calls for advanced “property-oriented programming” or “return-to-libc” style attacks, though no such exploits are currently documented. Future work includes attaching richer semantic properties to permitted callables and implementing read-only/weight-only restrictions on certain modules.
• Policy Maintenance: Evolving ML libraries require periodic policy regeneration, a process readily integrated into CI/CD workflows. Policy sets for successive versions of a model class are typically highly similar (90% Jaccard similarity across observed FlagEmbedding releases).
• Generalizability: This two-phase approach could be extended to formats with comparable expressive power, such as TensorFlow SavedModel and ONNX, if they embed arbitrary callable references. For tensor-only formats without “callable” primitives, such as SafeTensors and GGUF, PickleBall is unnecessary.
• Prototype Preprocessing: Prototype deployments necessitated minor library code edits (<100 LoC per library), primarily the removal of generics and propagation of type annotation information via docstrings.

7. Significance and Continuing Developments

PickleBall constitutes the first system to automate per-library and per-model-class load policies, grounded in static analysis, for secure pickle deserialization. It demonstrably blocks all known malicious real-world and crafted pickle payloads in an extensive dataset, maintaining high compatibility with widely used benign models, and surpasses state-of-practice methods on both axes (Kellas et al., 21 Aug 2025). Policy-generation and runtime overheads are modest for both ML library maintainers and end-users. The approach represents a practical elevation of the security baseline for Python-based ML deployments, reducing supply chain risk from “any Python code can execute” to “only the precise set of reconstructed callables may run”—a fundamental tightening of the serialization attack surface.