HoRA: Multi-Domain Framework

• HoRA is a multi-domain framework that integrates neural fine-tuning, ancient Jyotiṣa astronomical calculus, and rapid robotic motor adaptation to address complex adaptation challenges.
• In neural networks, HoRA employs a joint hypernetwork for cross-head low-rank adaptation, yielding polynomial sample complexity and tangible accuracy improvements over traditional LoRA.
• The framework bridges historical scientific insights with modern engineering, leveraging proto-calculus in Jyotiṣa and proprioception-based control in robotics to drive robust, efficient performance.

HoRA encompasses a spectrum of concepts across distinct research fields, with the term designating three influential yet disparate frameworks: (1) cross-head hyper-shared low-rank adaptation in large neural networks, (2) the Jyotiṣa Hora system underpinning proto-physical and proto-calculus ideas in ancient Indian astronomy/astrology, and (3) rapid motor adaptation for robust in-hand object rotation in robotics. Each context is described below, presenting the key constructs, methodologies, theoretical contributions, and empirical outcomes associated with "HoRA."

1. Hyper-shared Low-Rank Adaptation in Transformers

1.1 Motivation and Theoretical Foundation

HoRA ("Hyper-shared Low-Rank Adaptation") is a parameter-efficient fine-tuning (PEFT) methodology devised to overcome the limitations of Multi-Head LoRA (MH-LoRA) in Transformer multi-head self-attention (MHA) modules. Traditional LoRA ([Low-Rank Adaptation]) adapts a pre-trained matrix $W_0 \in \mathbb{R}^{m\times n}$ using a low-rank update, $\Delta W = AB$ with $A \in \mathbb{R}^{m \times r}$, $B \in \mathbb{R}^{r \times n}$, $r \ll \min(m, n)$, optimizing only $A,B$ while freezing $W_0$. In MH-LoRA, each attention head possesses independently trained low-rank adapters for $W_{Q,i}$, $W_{V,i}$. This per-head independence neglects information sharing and synergies across heads, leading to redundant adapters and poor sample efficiency, especially in low-data regimes.

The theoretical lens frames MHA as a hierarchical mixture-of-experts (HMoE): heads constitute expert groups, with position-wise softmax as static low-level gating. Analyses reveal that the non-shared (per-head) low-rank parameterization creates flat directions in Fisher information due to parameter block disjointness ("PDE interaction"), incurring a minimax lower bound rate of $n^{-1/2}$ for estimating the optimal mixture of experts—implying exponential sample complexity for $\Delta W = AB$0-optimal adaptation.

1.2 Joint Hypernetwork Parameterization

HoRA introduces a two-stage hypernetwork architecture that generates per-head low-rank adaptation matrices from shared parameters and learnable, head-specific embeddings. This enables cross-head adapter coupling while keeping the parameter tally modest.

For each attention layer:

• A "shared" linear layer computes $\Delta W = AB$1 with $\Delta W = AB$2, $\Delta W = AB$3, $\Delta W = AB$4 an activation function, and $\Delta W = AB$5 the head embedding.
• A second "shared" linear layer computes $\Delta W = AB$6 with $\Delta W = AB$7 and $\Delta W = AB$8 layer normalization.
• For head $\Delta W = AB$9, $A \in \mathbb{R}^{m \times r}$0 and $A \in \mathbb{R}^{m \times r}$1 are block-sliced, updating each head as $A \in \mathbb{R}^{m \times r}$2.

This hypernetwork requires only $A \in \mathbb{R}^{m \times r}$3 additional parameters per layer, versus $A \in \mathbb{R}^{m \times r}$4 for MH-LoRA.

1.3 Theoretical Performance and Sample Efficiency

Under the MoE regression model with Gaussian noise $A \in \mathbb{R}^{m \times r}$5, Theorem 1 proves that any estimator in the non-shared LoRA family is bottlenecked by a rate $A \in \mathbb{R}^{m \times r}$6, leading to exponential sample complexity. Theorem 2 shows that shared-hypernetwork HoRA admits a rate $A \in \mathbb{R}^{m \times r}$7, corresponding to polynomial sample complexity.

The crucial proof elements include:

• Viewing cross-head adaptation as conditional density estimation in HMoE,
• Demonstrating flat Fisher directions with non-shared LoRA,
• Showing shared parameterization (hypernetwork) abolishes those directions,
• Using empirical process theory and covering numbers to bound generalization.

1.4 Empirical Results and Ablations

Experiments span both vision and language domains:

• Vision: Fine-tuning ViT-B/16 on VTAB-1K and FGVC datasets (1,000 labeled examples each) shows HoRA increases average accuracy from 72.2% (LoRA) to 74.4% on VTAB-1K, and from 84.8% to 89.96% on FGVC.
• Language: On LLaMA-7B, HoRA attains 76.64% (vs. LoRA’s 74.09%), and on LLaMA-13B, 80.82% (vs. 80.18%) across eight reasoning benchmarks.

Ablations establish:

• Modest increases in rank $A \in \mathbb{R}^{m \times r}$8 and $A \in \mathbb{R}^{m \times r}$9 yield accuracy gains;
• Normalization and sigmoid activations stabilize convergence;
• Few-shot regimes (1% of language data) reveal >20% absolute improvement over LoRA;
• HoRA increases per-layer trainable parameters by only ~0.08% over LoRA.

The joint hypernetwork regularizes learning across heads, reducing overfitting and redundancy.

2. Hora in Ancient Indian Astronomical Calculus

2.1 Jyotiṣa and the Shad Bala System

In classical India, Jyotiṣa synthesized astronomy (Ganita), predictive astrology (Hora Sastra), and encyclopedic knowledge (Samhita). The foundational Hora texts (Parasara Hora, Brhat Jātaka, Horasāra) defined six forces (Shad Bala), among which Naisargika Bala ("natural force") and Chesta Bala ("dynamic force") prefigured ideas foundational to gravitational physics and differential calculus.

Naisargika Bala is specified as directly proportional to apparent planetary diameter $B \in \mathbb{R}^{r \times n}$0 and inversely proportional to geocentric distance $B \in \mathbb{R}^{r \times n}$1, expressed as $B \in \mathbb{R}^{r \times n}$2. It is an intrinsic planetary property, constant in time, and used to adjudicate planetary conjunctions (graha-yuddha). If two bodies are equidistant, the ratio of their forces is $B \in \mathbb{R}^{r \times n}$3, echoing the structure of gravitational interaction, albeit without the cubic dependence characteristic of Newtonian gravity.

2.2 Proto-Calculus: Instantaneous Velocity and Retrograde States

Hora treatises, observing planetary retrograde motion, introduced eight dynamical states (e.g., anuvakra, vikala, mandatāra, etc.), each associated with a Chesta Bala value (in shastiāṃśa). The key innovation is the implicit use of the quantity $B \in \mathbb{R}^{r \times n}$4—the instantaneous angular velocity of apparent planetary motion.

Stationary points ("vikala" phase) correspond to $B \in \mathbb{R}^{r \times n}$5, as determined by linear interpolation over epicyclic tables. This operationalizes first-order difference quotients and primitive differential calculus as practiced by Brahmagupta in the 6th century CE and refined by the Kerala school.

2.3 Historical Timeline and Influence

The Naisargika Bala force law, and derivative concepts rooted in planetary conjunction and retrograde analysis, precede the formal emergence of Newtonian gravity (1687 CE) and differential calculus (Newton–Leibniz). Indian contributions evolved within a framework focused on time- and context-sensitive planetary strength and brightness, rather than abstract geometric formalism.

The Indian approach—distinguishing ucca ("high") and nīccha ("low") points and computing interpolated force values—supplied an empirical, physically motivated path toward calculus-like ideas several centuries before analogous European developments (Girish et al., 2011).

3. Rapid Motor Adaptation for In-Hand Object Rotation

3.1 Problem Setup and System Design

HoRA also refers to a rapid motor adaptation (RMA) technique for in-hand object rotation with multi-fingered robot hands, targeting the challenge of robust, dexterous manipulation across object shapes, sizes, and physical parameters. The controller is trained solely in simulation with cylindrical objects and transferred to physical robot hardware with no vision or tactile sensing.

The system's inputs consist of robot joint positions $B \in \mathbb{R}^{r \times n}$6 (4 fingers, 4 joints each), with velocities and accelerations computed over a sliding window. The controller's action is a vector of target joint positions passed through a 300 Hz PD loop to generate torques.

3.2 Architecture: Policy and Adaptation Module

The controller comprises:

• A reinforcement-learned base policy $B \in \mathbb{R}^{r \times n}$7, where $B \in \mathbb{R}^{r \times n}$8 includes recent joint positions/actions and $B \in \mathbb{R}^{r \times n}$9 is a privileged "extrinsics" embedding reflecting object physics (mass, size, friction, etc.).
• An adaptation module $r \ll \min(m, n)$0, trained by supervised regression to infer $r \ll \min(m, n)$1 from a short proprioceptive-action window ($r \ll \min(m, n)$2), enabling online estimation $r \ll \min(m, n)$3 during deployment.
• The overall output action: $r \ll \min(m, n)$4.

3.3 Training and Evaluation

A reward signal in PPO incorporates rotational velocity ($r \ll \min(m, n)$5), pose error, object velocity, mechanical work, and torque penalties. The adaptation module is trained via $r \ll \min(m, n)$6 regression between inferred and true extrinsics. Large-scale training (16,384 environments in IsaacGym, 100,000 gradient steps) utilizes domain randomization over object and environment parameters.

Emergent behaviors include natural, energy-efficient finger gaits without explicit programming of contact switches or hand postures.

3.4 Sim-to-Real Transfer and Performance

HoRA demonstrates robust zero-shot transfer:

• On 30+ real objects (masses 5–200g, diameters 4.5–7.5 cm), the RMA controller achieves an average of 23.96±3.16 rad rotation and 0.98±0.08 normalized time-to-failure (TTF) with mean torque 1.84±0.24 for heavy objects—far outperforming baselines (domain randomization alone; system identification; no adaptation).
• On irregular/novel objects, the controller sustains rotation significantly longer and more stably than any non-adaptive baseline.
• The adaptation module infers physically meaningful object codes—the 8-dimensional embedding $r \ll \min(m, n)$7 clusters by object mass and size, and commanded torques scale proportionally to inferred mass (Qi et al., 2022).

4. Comparative Table of HoRA Contexts

HoRA Context	Domain	Core Mechanism
Cross-Head Low-Rank Adaptation (2025) (Diep et al., 5 Oct 2025)	Neural Networks	Joint hypernetwork for cross-head low-rank
Ancient Jyotiṣa Hora (c. 1st mil. BCE–14th c. CE) (Girish et al., 2011)	Astronomy/Astrology	"Shad Bala" force and proto-calculus ideas
Rapid Motor Adaptation for In-Hand Robotics (Qi et al., 2022)	Robotics	Online adaptation via proprioceptive history

Each instance of "HoRA" innovates by integrating structural or historical synergies: cross-head statistical coupling in neural adapters, synthesis of planetary phenomena in Indian science, or rapid proprioception-driven motor adaptation, all yielding empirical or conceptual advances within their disciplines.

5. Broader Implications and Connections

The cross-head hypernetwork perspective in neural adaptation highlights the relevance of mixture-of-experts and statistical estimation theory for modern PEFT techniques. The ancient Hora tradition illustrates how astronomical prediction seeded foundational physical-mathematical principles, especially under practical calendrical and astrological pressures. In robotics, HoRA (RMA) demonstrates that proprioception-only, history-based adaptation is sufficient for generalized, contact-rich dexterous manipulation, reducing reliance on exhaustive domain randomization or system identification.

These diverse instantiations of HoRA converge in leveraging implicit or explicit structure—whether inter-head statistical dependencies, force-brightness couplings in epicyclic astronomy, or latent physics estimation from proprioception—to mitigate sample inefficiency, redundancy, and the bottlenecks of naive independent parameterizations. Each affirms the utility of disciplined, principled parameter sharing and online adaptation in complex, underdetermined systems.