Ara Compiler: LLM Translation & Artifact Compilation

• Ara Compiler is a dual-functional system that converts legacy research artifacts and Arabic algorithmic texts into machine-executable packages and Python code.
• It employs a multi-layered workflow integrating semantic deconstruction, cognitive mapping, and physical grounding to ensure reproducibility and robust artifact generation.
• Utilizing LLM-driven translation and structured parsing, it optimizes research documentation and Arabic programming syntax for safe and efficient execution.

The Ara Compiler denotes two distinct compiler paradigms within contemporary computational research: (1) a LLM-driven semi-compiler enabling programmatic translation from Arabic-language algorithmic text to Python code ("APL compiler"), and (2) a multi-source research artifact compiler, central to the Agent-Native Research Artifact (Ara) ecosystem, that distills legacy narrative research documents and code repositories into a machine-executable, multi-layered package optimized for agent comprehension and reproducibility.

1. Definitions and Context

The term "Ara Compiler" primarily designates an agentic compilation pipeline that processes a heterogeneous mix of scientific PDFs, code repositories, reproduction rubrics, and experiment logs into an AraPackage, a rigorously structured, machine-native directory embracing cognitive, physical, exploration, and evidence layers (Liu et al., 27 Apr 2026). In parallel, "Ara-Compiler" also refers to the APL compiler, a system architected to ingest Arabic-script, curly-braced program statements and—via prompt-engineered LLM mediation—emit semantically equivalent Python code for execution within a controlled runtime (Sibaee et al., 2024). These two instantiations share the unifying theme of transformation: they bridge human-oriented or non-English textual/algorithmic representations and execution-ready, agent-native artifacts.

2. System Architectures

Ara Compiler (Research Artifact Domain)

The Ara Compiler operates as a single agent skill applying a top-down, multi-stage workflow:

• Semantic Deconstruction: Extracts structured facts from narrative PDFs, scrapes appendices, parses code comments, and reconstructs trace events from experimental logs.
• Cognitive Mapping: Populates the cognitive layer (/logic) by instantiating problems, claims, experiment protocols, related work graphs, and heuristics with formal schemas.
• Physical Grounding: Converts algorithms and architecture descriptions into parameterized code templates (src/kernel) or annotates pre-existing repositories (src/repo), consolidating all hyperparameters and experimental environments.
• Exploration Graph Extraction: Synthesizes a decision/experiment trace as a directed acyclic graph (trace/exploration_tree.yaml) from MALT logs and code-diff patterns, preserving dead ends, pivots, and ultimately rejected approaches.
• Evidence Dumping: Extracts tables, logs, and metrics into structured, queryable CSV and JSONL files in the evidence layer.
• Validation Loop: Execute up to three cycles of ARA Seal Level 1 validation, enforcing schema conformance and cross-referencing.

Ara-Compiler (Arabic Programming Language)

This system decomposes into three logical subsystems:

• Front-end / Parser: Tokenizes Arabic APL source, normalizes whitespace, and marshals each line into a JSON payload format for model ingestion.
• LLM Semi-Compiler: Employs a GPT-4-class model with a fixed translation prompt, effecting the mapping function $\mathrm{M}:$ APL $\to$ Python at the token and expression level.
• Executor / PyRunner: A sandboxed Python harness that restricts built-ins, disables filesystem/network access, and enforces execution timeouts, capturing standard output and diagnostic data.

A schematic block diagram:

┌─────────────┐ ┌────────────────────┐ ┌────────────┐ │ APL Source │─►│ LLM Prompt & │─►│ Python │ │ (Arabic) │ │ GPT-4 Translator │ │ Executor │ └─────────────┘ └────────────────────┘ └────────────┘ │ Parser/Formatter │ │ ▼ │ ▼ JSON payload Translated block Execution result {code:APL} {code:Py} {output,err}

3. Input/Output Specifications and Internal Data Structures

Ara Compiler

Inputs:

• Narrative PDFs (e.g., ICML, NeurIPS)
• Git-based code repositories
• PaperBench-format reproduction rubrics
• RE-Bench MALT transcripts and supplementary data

Outputs:

AraPackage directory, strictly adhering to protocol-specified layout:

Layer	Directory	Representative Files
Root Manifest	PAPER.md	layer index (YAML), metadata
Cognitive (/logic)	logic/	problem.md, claims.md, experiments.md
Physical (/src)	src/	kernel/ or repo/, configs/, environment.md
Exploration (/trace)	trace/	exploration_tree.yaml, sessions/
Evidence	evidence/	tables/, logs/
Staging	staging/	unpromoted observations

All files conform to formally defined schemas. For instance, each claim in claims.md documents: Statement, Status, Falsification criteria, Proof; each experiment in experiments.md records: Verifies, Setup, Procedure, Metrics, Expected outcome.

Ara-Compiler

Inputs: APL code (Arabic script, structured by curly braces), possibly containing variables, I/O instructions, arithmetic/logical expressions, conditional branches, loops, function definitions, and simple TXT-file reading.

Outputs: Valid Python source code, with structural mappings:

APL Construct	Arabic Example	Python Translation
Print	اطبع “مرحباً بالعالم”	print("مرحباً بالعالم")
Assignment/Loop	ابدأ i = 0 … بينما (i<5) {...}	i=0; while i<5: …

Keyword translation leverages a fixed mapping table (e.g., "اطبع"→print, "اذا"→if); curly braces are mapped to Python indentation.

4. Workflow Methodologies

Ara Compiler

The core pipeline is expressed functionally as:

$$\mathrm{Compile}: \mathrm{PDF} \times \mathrm{Repo} \times \mathrm{Rubric} \times \mathrm{TraceLogs} \rightarrow \mathrm{AraPackage}$$

Algorithmic steps:

1. Deconstruction: Translate prose and repo content into fact-dense bullet points, extract for every numeric result, equation, and dependency.
2. Mapping: Schema-conformant population of logic and code, with explicit cross-layer pointers (e.g., claims referencing proof and experiment data).
3. Validation and Fix: A generate→validate→repair loop via ARA Seal Level 1 checks, up to three cycles.
4. Certification: Emission of a signed Seal Certificate in PAPER.md, denoting structural integrity and schema adherence.

Ara-Compiler

1. Tokenization: Split APL source into individual lines; detect Arabic keywords and variables using a lookup table.
2. Prompt Engineering: Structure a prompt instructing the LLM to translate identifiers, replace Arabic control flow constructs, check syntax, and output only Python source.
3. Model Inference: LLM processes the prompt and returns Python code; in case of suspected syntax errors, emits an explanatory comment atop the code.
4. Sandboxed Execution: Python code is compiled and executed in a namespace with whitelisted built-ins, time-limited by a watchdog, and with redirected output/error streams.

5. Evaluation and Empirical Payoffs

Ara Compiler

Evaluation leverages two public benchmarks:

• PaperBench/RE-Bench (Understanding Layer): Baseline accuracy with PDFs/repos is 72.4%; AraCompiler–produced packages raise this to 93.7% (+21.3pp), with Ara usage averaging 114K tokens per question (baseline: 109K).
• RE-Bench (Reproduction Layer): On 150 sub-tasks across 15 papers, difficulty-weighted success goes from 57.4% to 64.4% (+7.0pp), with an 8/5/2 win/tie/loss record. On hard subtasks, Ara yields +8.5pp.
• Extension Layer Tasks: On five RE-Bench open-ended extensions, Ara agents reach task initialization earlier, with sustained advantage in several domains (e.g., rust_codecontests, nanogpt_chat_rl, fix_embedding).

Ara-Compiler

Qualitative validation is reported over 20 toy programs: translation accuracy for basic constructs is ≈95%, with primary errors resulting from mis-nested braces. LLM translation latency is reported as ≈150–200 ms per request; Python execution overhead for small scripts is negligible. Coverage is limited to six categories: variables, operations, TXT file reading, conditions, loops, and functions (Sibaee et al., 2024).

An illustrative table (from (Sibaee et al., 2024), future work):

Construct	Samples	Accuracy (%)	Latency (ms)
Print	5	100	140
Arithmetic	5	100	155
Loops	5	92	165
Functions	5	88	180

6. Integration Points and Ecosystem Interactions

• Live Research Manager: Ara Compiler and LRM both instantiate AraPackages. If a project is LRM-born, compilation validates and augments the package, using identical schemas—agents may switch between real-time capture and retrospective compilation seamlessly (Liu et al., 27 Apr 2026).
• Ara-Native Review System: Compiled Ara artifacts trigger automated Level 1 Seal checks; passing artifacts are eligible for review. Downstream systems query Seal Certificates via a uniform file-based API.
• Bidirectional Feedback (Ara-Compiler, proposed): Future work includes propagating Python runtime exceptions back into the LLM for debug-oriented retranslation (Sibaee et al., 2024).

7. Limitations and Prospective Directions

Ara Compiler

• Domain Scope: Validated primarily on empirical CS research; adapting to wet-lab protocols or theory papers is unresolved.
• Fidelity Ceiling: Extraction is source-bounded; omissions in PDFs/repos lead to absent fields—Live Research Manager can mitigate but cannot eliminate such gaps in purely legacy-driven flows.
• Deployment Prerequisites: Prototype lacks hardened sandboxing, fine-grained evidence-layer access control, and schema migration support.
• Scalability and Maintenance: Growth of the Ara corpus demands lineage tracking, cross-artifact reconciliation, and automated self-healing.

Ara-Compiler

• Coverage: Limited to a select set of programming constructs; fragile when handling deeply nested or idiomatic Arabic.
• Reliance on Proprietary LLM APIs: Cost and latency constraints.
• Error Reporting: Dependent on LLM self-diagnosis; vagueness remains an issue.
• Directions: Fine-tuning on Arabic-aware open-source LLMs, extended language support (data structures, exception handling), pre-LLM static analysis, and bidirectional debugging.

• "LLMs as Compiler for Arabic Programming Language" (Sibaee et al., 2024)
• "The Last Human-Written Paper: Agent-Native Research Artifacts" (Liu et al., 27 Apr 2026)