Theseus Operating System

• Theseus Operating System is an open-source OS in Rust that encodes OS invariants into its type system for robust resource management.
• It employs hybrid verification by combining Rust’s static checks, selective formal proofs, and informal reasoning to drastically reduce proof effort.
• Its architecture leverages intralingual design techniques, including linear types and typestate, to enable safe, efficient, and verifiable OS operations.

Theseus is an open-source operating system written entirely in Rust, designed to provide strong lightweight correctness guarantees while avoiding the prohibitive proof effort associated with full end-to-end formal verification. By combining Rust’s affine (linear) type system with selective formal verification and informal reasoning—a methodology termed hybrid verification—Theseus encodes invariants directly into its type system and leverages compiler checks to enforce a variety of resource-management, aliasing, and functional correctness properties. The architecture and methodologies developed for Theseus represent an evolution from traditional C-based OS designs, introducing intralingual design as a paradigm in which many correctness properties are lifted into the language’s static type system.

1. Design Goals and Architecture

Theseus targets enhanced OS correctness through two primary strategies: (1) maximally offloading invariant enforcement to Rust’s static type system—termed intralingual design—and (2) combining, rather than replacing, type checking, formal verification (via SMT solvers such as Prusti), and informal reasoning in a hybrid workflow.

Resources such as memory frames and device queues are modeled as first-class Rust types, each encoding their ownership, typestate transitions, and cleanup protocol via the type system. Resource management leverages both Rust’s Drop trait and typestate programming (phantom type parameters denote resource states such as Free, Allocated, Mapped, Unmapped), ensuring that the compiler statically enforces safe usage, exclusive ownership, and correct cleanup order. The OS consists of a minimal trusted core (“Nucleus”), and there is no run-time garbage collection; safe concurrency is enforced at the language level via Rust’s borrow checker.

Traditional C/C++ OSes generally separate specification from implementation and rely on dynamic checks or documentation to uphold functional protocols (such as lock acquisition or cleanup order). In contrast, Theseus internalizes many of these specifications into its type system: cleanup ordering, exclusive resource ownership, aliasing constraints, and even protocol-level correctness are statically guaranteed by the compiler.

2. Intralingual Design Techniques

Three major intralingual design techniques enable Theseus to encode and enforce OS-level invariants at compile time.

2.1 Intralingual Representation System (IRS)

A “representation” in Theseus is a linear-typed Rust value whose sole owner has authority over an OS resource, and the type system prohibits two overlapping representations from coexisting. Typestate is modeled by parameterizing structures with phantom types; for example:

struct Pages<S: State> { range: RangeInclusive<usize>, _marker: PhantomData<S>, } enum State { Free, Allocated, Mapped, Unmapped }

Only Pages<Mapped> exposes methods for memory access, enforcing access control at the type level. Typestate transitions (e.g., mapping/unmapping) consume and return mutually-exclusive typestates. Custom Drop implementations ensure correct cleanup, so dropping a mapped page triggers a cascade of cleanup actions (e.g., clearing page-table entries) through all predecessor states.

2.2 Linear Types as Proof-of-Work Tokens

Zero-sized linear types are used as “tokens” that can only be constructed by specific control-flow paths, enforcing protocols such as register-write order in device drivers. For example:

pub struct RxCtrlDisabled(()); pub struct FilterCtrlSet(()); impl Registers { fn rxctrl_rx_disable(&mut self) -> RxCtrlDisabled { /* ... */ } fn fctrl_write(&mut self, flags: FilterCtrlFlags, _t: RxCtrlDisabled) -> FilterCtrlSet { /* ... */ } fn rxctrl_rx_enable(&mut self, _t: FilterCtrlSet) { /* ... */ } }

This approach statically enforces that certain hardware registers are manipulated only in allowed sequences.

2.3 Intralingual Hardware Abstraction Layers

Memory-mapped IO (MMIO) regions are abstracted using Rust #[repr(C)] struct types wrapped with capability markers (ReadOnly<T>, WriteOnly<T>, ReadWrite<T>) that restrict permissible volatile accesses at compile time. Bitfields are constrained using the bitflags! macro or enums, limiting the set of allowed values. Combining this with proof-of-work tokens ensures inter-field and operation-order constraints are statically checked.

3. Hybrid Verification Methodology

Full formal verification of an OS kernel has long been considered prohibitively expensive. Theseus employs a hybrid approach, weakening some guarantees but dramatically reducing the overall proof obligation and developer effort.

3.1 Selective Application of Verification

Type-system reasoning and informal (prose) proofs are cheap but less expressive, while SMT-based formal verification (using Prusti) is reserved for essential properties—such as uniqueness of resource representations—that cannot be guaranteed by the type system alone. This approach increases the trust base (including Rust core/alloc, Prusti’s backend, and a minimal set of unverified modules and specs), but reduces proof effort by one to two orders of magnitude compared to end-to-end verification.

3.2 Rules for Extending Verified Invariants

Three pragmatic rules amplify the reach of verified invariants:

1. Invariant Preservation: Once an invariant is verified for a type, all fields are made private and invariance-violating traits (e.g., Clone, DerefMut) are forbidden, allowing the compiler to statically maintain the invariant.
2. Composition: If a type $T$ is proven to satisfy property $P$, and $U$ is structurally composed only of Ts, then $U$ inherits $P$ without need for additional proof.
3. Reuse: Generics verified once (e.g., a representation creator function) propagate their proofs to all monomorphized instantiations via Rust’s compilation model.

3.3 Intralingual Specifications and Proof Obligations

Rust procedural macros provide compile-time checks for meta-properties; e.g.,

assert_not_impl_any!(Pages: DerefMut, Clone); #[private_fields("range")] #[nomutates("self.list", "self.array")]

These guard against accidental invariant violations in unverified code paths. For critical methods (e.g., unique representation creation), proof obligations are explicitly annotated:

#[ensures( result.is_ok() ==> (forall(|i: usize| i < old(self.list.len()) ==> !old(self.list.lookup(i)).overlaps(&id))) && result.is_ok() ==> self.list.lookup(0) == id )] pub fn create_unique_representation(&mut self, id: T) -> Result<R, Error> { … }

4. Case Studies

4.1 Memory Subsystem

The primary invariant for memory is bijective mapping: every virtual page maps to exactly one physical frame and vice versa, formulated as

$$\forall p_1, p_2: \text{Pages}<\text{Mapped}>,\, p_1.\text{range} \cap p_2.\text{range} \neq \varnothing \implies p_1 \equiv p_2$$

This is realized with IRS types for both frames and pages, with only the Mapped typestate permitting memory access. Intralingual HALs encode page-table walking and field-level PTE access. SMT verification (via Prusti) is used on the generic representation creator to guarantee uniqueness.

Effort and performance data:

Component	Implementation SLOC	Proof SLOC	Verification Time	Performance
RepCreator<T,R>	392	7	~191 s	1.99 μs (per op, 4k map)
Frame/page allocators	293	0–8	~135 s	—
Memory casting functions	1076	8	~459 s	—

The proof-to-implementation ratio is approximately 1:171 for the memory subsystem (excluding RepCreator; including it: 1:117). Microbenchmarks show no measurable performance degradation for core memory operations.

4.2 10 Gb Ethernet Driver (ixgbe)

Correctness properties include datasheet-compliant bit-level communication, safe management of descriptor rings and buffers, and integrity of driver state. The intralingual HAL statically enforces access restrictions, and IRS ensures cleanup and re-initialization semantics for all device objects. Typestates capture mutually exclusive configuration options and guarantee correct send/receive enablement.

Key driver routines (e.g., send_batch, receive_batch, add_filter) are formally verified using Prusti, with index bounds and logical preconditions checked. Intralingual specs prevent unverified code from invalidating critical invariants.

Effort and performance data:

Subsystem	Impl SLOC	Verified SLOC	Proof SLOC	Verification Time	Throughput vs. DPDK
ixgbe driver total	~2000	141	17	~115 s	19 Gbps vs. 20 Gbps
Prusti specs			41

The proof-to-implementation ratio for the verified portion is ≈ 1:8.3; spec-to-implementation is ≈ 1:3.4. The driver achieves maximum bidirectional throughput within 5% of the DPDK baseline and exhibits comparable latency to peer research drivers. Retrofitting the Theseus intralingual HAL into other Rust OS ixgbe drivers resulted in detection of four previously unreported bugs at compile time, such as reserved-bit writes and ordering violations.

5. Trade-offs, Trust, and Limitations

The trade-off in Theseus is between guarantee strength and proof effort: slightly weakening the logical strength of system-wide guarantees (by allowing some properties to be maintained by type checking or informal reasoning rather than full formal proofs) reduces proof effort by one to two orders of magnitude. Consequently, the trusted computing base (TCB) expands to include Rust’s core libraries, the Prusti verification backend, and a select set of unverified modules or specifications. While this is larger than that of formally verified microkernels, it remains significantly below the size of typical OS stacks.

Performance measurements indicate that intralingual design and selective verification produce negligible or modest runtime overhead. Core memory operations show unchanged performance, while the throughput of verified NIC drivers remains within ~5% of highly optimized C-based drivers.

6. Significance for OS Correctness in Rust

Theseus systematically demonstrates that a language with a strong affine type system—in this case, Rust—can be leveraged to enforce a wide spectrum of OS invariants at compile time, significantly reducing the proof burden compared to interactive theorem proving. The methodology combines IRS, proof-of-work tokens, intralingual HALs, and procedural-macro-based specifications, establishing a pattern for lightweight yet expressive OS verification that can be adopted by other Rust-based system projects. This approach bridges the spectrum between purely static typing and heavyweight formal methods, suggesting a practical path toward scalable, lightweight verification for real-world systems.