AI Native-First Philosophy

• AI Native-First Philosophy is a paradigm that reimagines systems by fusing AI into core design, emphasizing model-centric co-design and multi-tenant efficiency.
• It drives modular ML contracts and serverless elasticity, enabling rapid scaling, efficient batching, and reduced per-inference costs.
• The approach leverages advanced ML runtime, distributed resource coordination, and microservices integration to support continuous learning and emergent AI workloads.

The AI Native-First Philosophy defines a paradigm in which AI is integrated as the core organizing principle and operational substrate of computational systems, infrastructure, and software design. Rather than treating AI as a peripheral service or a bolted-on augmentation to existing cloud-native or distributed architectures, this philosophy calls for a fundamental rethinking of system, runtime, and software co-design: ML model concerns are treated as primary, and infrastructure is architected around the unique demands of large, generative models and their workflows. Such an approach is distinguished by tight, reciprocal integration of ML runtime, horizontal and multi-tenant scaling, and the systemic pursuit of resource efficiency and accessibility at scale (Lu et al., 17 Jan 2024).

1. Contrasting Cloud-Native and AI-Native Paradigms

Cloud-native architectures are designed to maximize scalability, resilience, and modularity in classical software workloads via containerization, orchestration (e.g., Kubernetes), microservices, serverless computing, and multi-tenancy. In this model, ML operations are typically treated as black-box workloads atop a general-purpose, resource-shared substrate optimized for vertical scaling and stateless function endpoints.

AI-native computing, in contrast, explicitly fuses cloud-native design with advanced ML runtime considerations. AI-native systems support co-design and deep coupling of distributed/cloud infrastructure and model execution, with architectural priorities such as:

• Horizontal scale-out for thousands of model variants (not just large monolithic deployments)
• Multi-tenant operation in terms of both users and model adaptors (e.g., LoRA fine-tunings)
• Elastic, rapid model scheduling and batched inference to drive cost-of-goods-sold (COGS) efficiency and improve accessibility

Aspect	Cloud-Native	AI-Native
Focus	General software/app infra	Large model (LMaaS) workloads
Scaling	Vertical (per app)	Horizontal (multi-model/tenant)
Black-box ML	Yes	No — ML runtime co-design
Multi-tenancy	Per container, database	Multi-model, LoRA adaptors, users
Serverless	Stateless functions	Elastic LLM endpoints
COGS optimization	Autoscaling, containers	Model batching, batched inference
ML runtime	Untouched	Batched LoRA, MoE, memory opt

2. Architectural and Technological Shifts

AI-native systems introduce several architectural advances beyond typical cloud-native stacks:

• Modular ML Contracts: The AI-native workflow decomposes model operations into formal contracts:
  • Training: $$\mathcal{W}_m \leftarrow \mathrm{train}(m, \mathcal{D}, \mathcal{C})$$
  • Fine-tuning (e.g., LoRA): $$\Delta m, \mathcal{W}_{\Delta m} \leftarrow \mathrm{finetune}(m, \mathcal{W}_m, \mathcal{D}', \mathcal{C}'')$$
  • Inference: $$r \leftarrow \mathrm{inference}([m,\Delta m], [\mathcal{W}_m, \mathcal{W}_{\Delta m}], q)$$

This separation allows shared base models with user-specific, resource-light adaptors, supporting efficient multi-tenant fine-tuning and serving workflows.

• Advances in ML Runtime: Innovations such as batched LoRA inference enable leveraging a single base model in conjunction with many lightweight LoRA adaptors. Punica system achieves a 14x throughput increase (batch size 32) over state-of-the-art frameworks on the 7B Llama-2 model on A100 GPUs via custom CUDA kernels.
• Serverless Elasticity: AI-native serverless orchestration enables loading/unloading heavy model checkpoints efficiently as load fluctuates, maximizing GPU utilization and minimizing idle costs. Model startup latency is in the seconds range on modern high-speed interconnects.
• Distributed Resource Coordination: Systems such as JellyBean schedule inference or training jobs across heterogeneous, geo-distributed resources (e.g., edge, spot, data center), optimizing for cost, availability, and task requirements; serving costs can be reduced by up to 58%.

3. Resource Efficiency, COGS Optimization, and Accessibility

AI-native design addresses the unsustainable per-inference costs of large models ($>1$ cent/query for GPT-4 or Llama-2, often exceeding typical web revenue per click) by:

• Model specialization (deploying specialized or fine-tuned models for specific use-cases)
• Multi-tenant batching (consolidating multiple user queries/adaptors per GPU/model load)
• Serverless elasticity (dynamically allocating/removing GPU/model resources for peak efficiency)

The system must handle spot and ephemeral resource pools, introducing mechanisms for fast checkpointing, rapid model loading/unloading, and cost-aware job scheduling to exploit transient availability in distributed (possibly geo-scattered) environments.

4. Microservices, Workflow Decomposition, and RAG Integration

AI-native extends microservice design to the end-to-end ML pipeline:

• Decomposes not only infrastructure, but also data curation (embedding generation), fine-tuning, inference, and vector database operations (as in Retrieval-Augmented Generation, RAG).
• Vector databases (e.g., Milvus, Pinecone) are organized as microservices integral to RAG workflow, enabling caching, fast embedding lookup, and improving retrieval-driven inference.

RAG-as-a-Service pipelines resemble BI-as-a-Service architectures, mapping data extraction, vectorization, storage, and inference to modular, orchestrated services. This analogy underscores the transferability of cloud-native principles but highlights the need for ML-specific workflow optimization.

5. ML Runtime and Continuous Learning Innovations

AI-native runtime advances go beyond batch inference to leverage new compute and memory optimization strategies:

• Mixture-of-Experts (MoE) architectures, FlashAttention, PagedAttention, speculative decoding mechanisms, and continuous online learning systems push hardware and throughput efficiency by exploiting underutilized resources and dynamic workload patterns.
• Continuous learning support enables real-time data ingestion and in situ adaptation (e.g., LoRA adapters updated continuously from streaming data), with runtime systems coordinating efficient integration of new knowledge akin to streaming ETL systems in databases.

6. Design Principles and Future Implications

AI-native design implies:

• Deep ML and infrastructure co-design: ML runtimes and distributed systems are engineered in tandem to optimize resilience, efficiency, and cost.
• Model-centric serverless microservices: Models become elastic endpoints, with fine-tuning, adaptation, and inference orchestrated as modular services.
• First-class multi-tenancy: Supporting multiple users, tenant-specific models/adaptors, and concurrent serving/fine-tuning at scale.
• System+ML joint optimization: Techniques such as shared memory paths, batched multi-tenant inference, and workload-aware resource allocation.
• Support for agentic and emergent workloads: Flexible support for emerging AI agents, complex model chains, and multi-role deployments.

Aspect	Cloud-Native	AI-Native
Focus	General app/server infra	Large model (LMaaS) workloads
Scaling	Vertical	Horizontal multi-model/tenant
Black-box ML	Yes	No, deep runtime integration
Multi-tenancy	Database/container-based	Multi-model, multi-adaptor
Serverless	Stateless func endpoints	Elastic LLM endpoints
COGS optim.	Autoscaling	Model batching, LoRA batched inf
ML runtime	Ignored	Batched LoRA, MoE, memory opt

7. Summary and Trajectory

AI-native computing as a philosophy posits a fundamental redesign of distributed systems to meet the emerging demands of large, multi-tenant AI workloads. It advocates for a shift from black-box ML atop generic cloud infrastructure to integrated, co-designed ML+system architectures with deep runtime coupling, efficient batching, elastic orchestration, and explicit resource control. This new paradigm targets accessibility, affordability (COGS), and extensibility, enabling the full benefit of generative AI at scale—heralding a new era in distributed software and ML system design (Lu et al., 17 Jan 2024).