SecureTF Architecture

• SecureTF Architecture is a distributed secure machine learning framework that enables unmodified TensorFlow execution within Intel SGX enclaves on untrusted cloud infrastructures.
• It employs a layered security design with components like the CAS enclave and secureTF Controller to provide confidentiality, integrity, authenticity, and freshness for data, code, and model parameters.
• Performance evaluations show minimal inference overhead and scalable distributed training, with trade-offs linked to enclave memory limitations and EPC size.

secureTF is a distributed secure machine learning framework designed to execute unmodified TensorFlow applications with end-to-end security in untrusted cloud infrastructures. Built atop the Intel SGX Trusted Execution Environment (TEE) and the SCONE shielded-execution runtime, secureTF addresses confidentiality, integrity, authenticity, and freshness for data, code, and model parameters, while closely matching native TensorFlow performance and maintaining transparency for application developers. It uniquely extends SGX's trust boundary from a single enclave to a fully distributed cluster, establishing a cohesive trust model for secure training and inference across multiple machines (Quoc et al., 2021).

1. Security Objectives and Threat Model

secureTF is engineered to assure five primary system goals:

• Confidentiality: User data (raw input, intermediate tensors, model parameters) and application code (e.g., Python graph definitions) are consistently encrypted when persisting to disk or transiting outside enclaves.
• Integrity and Authenticity: Any modification to code, data, or parameters on disk or in transit must be detectable by the enclave, with authenticated metadata ensuring freshness and preventing rollback attacks.
• Transparency: Existing TensorFlow applications (Python or C++) execute unchanged, leveraging the native TensorFlow API.
• Accuracy: ML outputs—both numerics and overall accuracy—are guaranteed to be bit-for-bit identical to native TensorFlow, with no loss stemming from approximation or added privacy noise.
• Performance: Overhead is minimized, with classification/inference latency typically under 10% compared to native runs. Training overhead scales linearly in a distributed setting and is chiefly dependent on enclave memory limitations.

The threat model assumes a powerful adversary with ring-0 control of the cloud OS, hypervisor, file system, network, and I/O stacks outside enclaves—including the standard runtime libraries (glibc/musl) and interpreters. The adversary can access or modify any memory beyond enclave boundaries and manipulate the network (drop, replay, reorder packets); side-channel attacks are out of primary scope, though hooks for mitigations (SpecLH, SGXBounds) are available. The adversary cannot compromise SGX's hardware protections or CPU root secrets.

2. Architectural Components and Roles

secureTF is architected as four logical layers, providing a layered security and orchestration stack:

Layer	Description	Core Security Role
Configuration & Attestation Service	CAS enclave boots the trust anchor, manages secrets and audit log	Root of trust, key/cert provisioning
secureTF Controller	SCONE-modified musl libc/shim inside each enclave	Syscall/I/O mediation, shields
secureTF TensorFlow Library	CPU-only TF and TF-Lite for enclave execution, compiled for SCONE	ML functionality inside TEE
Orchestration Layer	Container-based deployment and workflow management outside enclaves	Launch, scaling, non-sensitive control

• CAS Enclave: Receives attestation requests from new secureTF enclaves, performs SGX remote attestation (via Intel Attestation Service or local IAS-proxy), verifies enclave measurements (MRENCLAVE), creates and provisions session keys, TLS certificates, and file-shield keys, and maintains a sealed, append-only audit log for all provisioning events.
• secureTF Controller: Intercepts and mediates all syscalls and I/O via a SCONE-modified musl libc, uses user-level threading to batch syscalls, provides network and file-system shields (converting socket traffic to TLS; encrypting/HMAC'ing file data), and guarantees data never traverses the enclave boundary in plaintext.
• secureTF TensorFlow Library: Provides the full (CPU-only) TensorFlow for training and TensorFlow Lite for inference, compiled using SCONE tools. Supports the Python API inside enclaves, and authenticates all loaded shared objects via file-shield. Model graphs and checkpoints are imported/exported using stable Protocol Buffer formats.
• Orchestration Layer: Deploys secureTF nodes in Docker containers, each waiting for key/configuration from CAS before launching jobs, with all distributed traffic shielded by default.

3. Trust Establishment and Key Management

secureTF's hierarchical key management process centers on the CAS enclave:

• Root Keys: CAS enclave maintains a long-lived root key ($K_{root}$) sealed with SGX.
• Remote Attestation Sequence:

1. New enclave instance $E$ sends its SGX quote $Q(E)$ (signed by the CPU's EPID key) to CAS.
2. CAS verifies $Q(E)$ via IAS or a local proxy, establishes trust in the enclave image and configuration.

• Secret Provisioning: CAS derives per-enclave session and operational keys via a KDF:

$$K_{session} = \text{KDF}(K_{root} \parallel N), \quad K_{TLS} = \text{KDF}(K_{session} \parallel \text{``tls''}), \quad K_{FSP} = \text{KDF}(K_{session} \parallel \text{``fsp''})$$

These keys are securely transmitted over the mutually attested channel.

• Remote Attestation Signature:

$$\sigma = \text{Sign}_{SK_{TPM}}(\text{hash}(\text{enclave\_image} \parallel \text{enclave\_config}))$$

ensuring authenticity of the enclave image and metadata.

• Audit Log: CAS maintains an append-only monotonic counter per enclave, included in the KDF nonce, to ensure freshness and enable anti-rollback guarantees.
• TLS Certificates: CAS acts as a cluster CA, generating root and per-enclave leaf certificates tightly bound to enclave measurements (MRENCLAVE), supporting mutual TLS across enclaves.

4. Secure Communication, Execution, and Integration with TensorFlow

secureTF enforces security properties in distributed execution through shielded communication and transparent integration with unmodified TensorFlow:

• Network Shield: All cross-enclave communications use mutually authenticated TLS with ECDHE key exchange; every application socket call is intercepted and TLS-encrypted using AES-GCM (128-bit tag). Protobuf-encoded messages (e.g., TF data batches, gradients, checkpoints) are wrapped in AEAD, with replay protection via TLS sequence numbers and AEAD nonces.
• File-System Shield: All persistent I/O (open/read/write) is intercepted; outgoing data is encrypted or authenticated chunkwise (per configuration), and ingested data is decrypted and validated before delivery to the application.
• TensorFlow Execution: Full TensorFlow (training) or TensorFlow Lite (inference) is compiled to run completely within enclaves. File operations (e.g., TFRecord ingestion, checkpointing) are shielded; gRPC and RPC internal communications are transparently TLS-protected. User Python or C++ code is loaded inside the enclave and authenticated by the file-shield. No changes are necessary to TensorFlow source code, only to the build toolchain and launcher.
• Session Management: Native TensorFlow operations (e.g., tf.train.Saver().save(), restore()) are transparently shielded by interceptors and mapped to AEAD-encrypted storage and retrieval.

5. Distributed Trust Extension Across Cluster

secureTF overcomes the typical locality of SGX's trust boundary—normally restricted to a single CPU—through cluster-wide attestation and key distribution:

• Cluster Root-of-Trust: All enclaves mutually attest to the CAS enclave, which both users and enclaves trust, acting as the singular trust anchor for the distributed cluster.
• Elasticity: New enclaves (spun up for autoscaling or fault recovery) re-attest to CAS, receive fresh keys/certs, and seamlessly join the trust domain.
• Coordination: Model updates and state synchronization utilize TensorFlow-native mechanisms (Parameter-Server, ring-AllReduce, gRPC), secured by the same per-peer mutual TLS guarantees. Byzantine-fault tolerant consensus is not required, as all enclaves are mutually attested.
• Anti-Rollback: CAS’s global, monotonic audit log prevents attackers from reusing historical sealed state or manipulating enclave instantiation sequences.
• Operational Topology: The main communication path follows: Client ↔ CAS enclave ↔ (Docker/Kubernetes Orchestration) ↔ Coordinator/Worker enclaves, with all communication on mutually attested TLS channels. A plausible implication is that secureTF fits with standard orchestration workflows for cloud machine learning.

6. Performance and Scalability Metrics

secureTF's performance is characterized by empirical measurements across attestation, inference, and training scenarios:

• Remote Attestation: End-to-end quote verification and key provisioning take ~17 ms using local CAS, and ~325 ms via Intel IAS—constituting a 19× speedup. Freshness/anti-replay features do not impact attestation latency.
• Inference Latency (Single-Image):
  • Native TF-Lite glibc: baseline $T_0$
  • Native TF-Lite musl: $T_0 + 2\%$
  • secureTF simulation mode: $T_0 + 5\%$
  • secureTF hardware mode: $1.1 \times T_0$ (model fits in EPC) to $E$0 (model exceeds EPC)
  • Graphene-SGX: $E$1–$E$2
• Training Throughput (MNIST, $E$3, $E$4):
  • Native TF (1 node, 4 cores): $E$5
  • secureTF hardware mode: $E$6
  • secureTF simulation mode: $E$7
  • secureTF sim + network-shield: $E$8
• Scalability: Near-linear scale-out up to three nodes (hardware mode), achieving 2.6$E$9 throughput. For $Q(E)$0 workers, speedup is $Q(E)$1 where $Q(E)$2 in simulation, $Q(E)$3 in hardware per additional worker.

This performance profile reflects enclave memory constraints: when model/data surpass ~94 MB (enclave page cache or EPC), secureTF incurs increased paging and slows by 10–15$Q(E)$4 in pure hardware mode. Mitigations include an enclave simulation mode, future CPUs with larger EPC (e.g., Ice Lake), and data pruning or normalization.

7. Trade-Offs, Deployment Considerations, and Guarantees

secureTF eliminates the need for full homomorphic encryption or secure multiparty computation (MPC), achieving native TensorFlow accuracy and transparency by relying on SGX enclaves for TEE properties. The principal constraints are:

• EPC Size: Overhead arises primarily when models/data exceed available EPC memory, significantly affecting training speed for large-scale models; simulation mode or future hardware can mitigate this.
• Deployment: SecureTF is fully compatible with standard orchestration (Docker/Kubernetes) and supports elasticity and autoscaling. Each enclave image must be signed, and deployment requires the SCONE toolchain and a running CAS enclave (which may use a local attestation proxy to avoid external IAS calls).
• Openness: No modifications are needed to TensorFlow source or user code; only build environment (scone-gcc/scone-g++) and entrypoint scripts are altered.

secureTF’s guarantees rest on hardware-enforced confidentiality and integrity of enclaves, fine-grained key management by the CAS enclave, and comprehensive shielding of all I/O and inter-enclave communications. Rollback protection and auditability further reinforce the trust model, while the performance overhead remains minimal for inference, scaling linearly for distributed training, subject to EPC memory size (Quoc et al., 2021).