Library Data Flywheel: Continuous Model Adaptation

• Library Data Flywheel is an iterative, closed-loop paradigm that adapts foundation models using deployment-collected data.
• Its physical instantiation, like Scanford, uses robotics to scan library shelves, label images with catalog metadata, and fine-tune models efficiently.
• Arena Learning, a virtual instantiation, leverages simulated chatbot battles and reinforcement learning to harvest challenging examples and boost LLM performance.

The Library Data Flywheel (LDF) is a closed-loop architectural and algorithmic paradigm for continual, targeted adaptation of foundation models (FMs), where a deployment agent or system both applies and automatically generates new domain-relevant data to enhance its own capabilities. LDF instantiations exist in both physical and virtual domains, such as vision-language robots (“Scanford” in the East Asia Library) and LLM post-training pipelines (“Arena Learning”). LDF enables efficient, scalable model improvement without manual annotation by systematically surfacing and harvesting underrepresented or challenging real-world data at deployment (Grannen et al., 24 Nov 2025, Luo et al., 15 Jul 2024).

1. Definition and High-Level Architecture

The Library Data Flywheel is an automated, iterative loop wherein an FM-equipped agent observes, collects, curates, and integrates new training data while performing tasks that advance domain-specific and domain-adjacent generalization. In the Scanford instantiation, a mobile manipulator robot equipped with a vision-LLM (VLM) traverses library aisles, collects shelf images, labels them using catalog metadata, and fine-tunes the FM on the curated dataset. Conversely, Arena Learning for LLMs leverages simulated chatbot battles to algorithmically uncover model weaknesses, harvest high-value training examples, and apply both supervised and reinforcement learning for continual improvement.

The canonical LDF architecture comprises:

• Deployed Agent: Embodied robot (Scanford) or a software system for virtual arenas.
• Initial Foundation Model: Pretrained VLM (Qwen2.5-VL) or LLM.
• Data Collection Mechanism: Sensors (RGB-D, LiDAR) or simulated battle logs.
• Automatic Labeling/Curation: Integration with external databases (library catalog) or AI-driven annotators.
• Model Adaptation Loop: Fine-tuning or reinforcement learning on aggregated, self-curated data, then redeployment.

Formally, the iterative process for Scanford is: $$\begin{aligned} D^{\mathrm{raw}}_t &\gets \mathrm{RobotDeploy}(\text{FM}_{t-1}, T) \ D_t &\gets \mathrm{Curate}(D^{\mathrm{raw}}_t) \ \mathcal{D}_t &= \bigcup_{k=1}^t D_k, \quad \mathcal{D}_0 = \emptyset \ \text{FM}_t &\gets \mathrm{FineTune}(\text{FM}_0, \mathcal{D}_t) \end{aligned}$$ with analogous loops for Arena Learning.

2. Physical and Virtual LDF Instantiations

A. Library Robotics: Scanford

Scanford is a physical instantiation of LDF using a Franka FR3 manipulator on a TidyBot++ base, equipped with a RealSense D435 RGB-D camera and Unitree L2 LiDAR. The robot operates autonomously in a library environment, executing precise scanning routines and leveraging the library catalog for self-supervised labeling. Drift correction is performed via shelf-plane fitting from LiDAR point clouds. Shelf-image scans capture a range of viewpoints, and sequence-prompting with catalog constraints mitigates occlusion and multilingual labeling challenges.

B. LLM Post-Training: Arena Learning

In Arena Learning, LDF is instantiated as a simulated battle system in which multiple LLMs compete over curated datasets. The WizardArena pipeline generates offline testsets using k-means clustering and GPT-4-based difficulty ranking to maximize coverage and challenge. A strong LLM (Llama3-70B-Chat) adjudicates response quality across facets (coherence, factuality, relevance), yielding continuous Elo ratings. The data flywheel iteratively sources the most informative data—where the target model fails relative to SOTA competitors—for subsequent supervised finetuning, preference optimization (DPO), and reinforcement learning (PPO) (Luo et al., 15 Jul 2024).

3. Automatic Data Curation and Labeling

Key to LDF efficiency is minimizing the need for manual annotation. In Scanford:

• FM predictions are aligned with call-number–indexed catalog records using string-similarity (Ratcliff–Obershelp) and order constraints.
• Prompts restrict candidate titles to shelf-local ranges to reduce ambiguity.
• Predictions below similarity thresholds or with order mismatches are discarded.
• Example curation function:

$$\mathrm{Curate}(D_t^\mathrm{raw}) = \left\{ (I, \hat L) \;\Big|\; \mathrm{sim}(\hat L, L^*) > \tau \wedge \mathrm{order}(\hat L, L^*) \right\}$$

where $L^*$ is the catalog sequence, $\tau$ is the similarity threshold.

In Arena Learning, curated training data is formed by extracting cases where a competitor model outperforms the target model, using those examples as SFT targets, and constructing <chosen, rejected> pairs for DPO and PPO loss.

4. Model Adaptation Algorithms

LDF operationalizes continual FM adaptation through the following algorithmic steps:

• Supervised Fine-Tuning (SFT): Applies curated data for direct cross-entropy minimization. For Scanford:

$$\mathcal{L}(\theta) = -\sum_{n=1}^N \;\sum_{i=1}^{K_n} \log p_\theta(l_i \mid I^{(n)}, l_{<i})$$

• Direct Preference Optimization (DPO): In Arena Learning, maximizes reward margins between preferred and rejected responses:

$$\mathcal{L}_\mathrm{DPO} = -\sum \log \sigma(\beta[r(y^+; x) - r(y^-; x)])$$

• Policy Optimization (PPO): Trains RL policies using clipped policy-gradient losses and KL penalties, as in

$$\mathcal{L}_\mathrm{PPO}(\theta) = -\mathbb{E}_t \left[ \min(\rho_t(\theta) \hat{A}_t, \mathrm{clip}(\rho_t(\theta), 1 - \epsilon, 1 + \epsilon) \hat{A}_t) \right] + c\,\mathrm{KL}[\pi_\mathrm{old} \| \pi_\theta]$$

• Loop Control: Each LDF iteration deploys the newly adapted model, collects new data (from real-world scans or simulated battles), and re-enters the curation-training pipeline.

5. Empirical Results and Data Efficiency

LDF achieves substantial, data-efficient improvement in FM performance:

Scanford Deployment

Fine-tune Data	# Images	Book-ID Accuracy
0 h (0 images)	0	32.4%
1.5 h (1352)	1352	68.5%
6 h (5019)	5019	71.8%

Language	Pre-trained Acc	5,019 Post-LDF
English	24.8%	46.6%
Chinese	30.8%	38.0%

Performance as a function of image volume $m$ follows a saturating exponential: $$\mathrm{Acc}(m) \approx A_{\infty} - \Delta A e^{-m/\tau}$$, with $\tau \approx 1000$ images, indicating diminishing returns at higher volumes (Grannen et al., 24 Nov 2025).

Arena Learning

WizardLM-$\beta$ models trained with LDF loop exhibit Elo improvements on WizardArena-Mix from 871 → 1274 (7B) and 889 → 1349 (8x22B), with concurrent MT-Bench score and win-rate increases against GPT-4o from 8% to 34%. SFT dataset size decreases (30k → 7.8k) as average example difficulty increases, demonstrating high data efficiency (Luo et al., 15 Jul 2024).

6. Operational Efficiency and Human Effort

LDF directly reduces human annotation and supervision:

• Scanford scanned 2,103 shelves in 40 h, saving ≈18.7 h of manual effort (0.47 h saved per deployed hour); only 26 interventions (<5 min each) were required.
• As model quality increases, the frequency of human intervention and error correction decreases, further amplifying operational efficiency.
• Arena Learning achieves full automation by exclusively leveraging LLM-based judging and curation, eliminating the need for human-labeled battle outcomes (Grannen et al., 24 Nov 2025, Luo et al., 15 Jul 2024).

7. Limitations, Generalization, and Prospects

Notable limitations of LDF include significant engineering overhead for robotic deployment (hardware integration, physical scan trajectories, drift correction) and dependency on heuristic curation thresholds. Fine-tuning alone does not guarantee complete accuracy, particularly on corner cases or heavily occluded data. In Arena Learning, the automated judge's bias and coverage remain intrinsic risks. Both LDF instantiations rely on the principle that selecting tasks within the FM’s “Zone of Proximal Development” is critical for the dual objectives of real-world utility and effective self-improvement.

Generalization prospects include transfer to other domains (grocery, healthcare records) and modalities (vision-language-action). Future directions involve incorporating robot-acquired transcripts and trajectories into multimodal FM pretraining, automating curation threshold selection through meta-learning or active human-in-the-loop, and packaging the data flywheel as a modular toolkit for a diversity of robotic or virtual agents (Grannen et al., 24 Nov 2025). The LDF paradigm enables both reduction of operational costs and robust, domain-targeted FM adaptation, scaling foundation model capabilities to the complexity and heterogeneity of real-world environments.