CARLoS: Retrieval via Concise Assessment Representation of LoRAs at Scale

Abstract: The rapid proliferation of generative components, such as LoRAs, has created a vast but unstructured ecosystem. Existing discovery methods depend on unreliable user descriptions or biased popularity metrics, hindering usability. We present CARLoS, a large-scale framework for characterizing LoRAs without requiring additional metadata. Analyzing over 650 LoRAs, we employ them in image generation over a variety of prompts and seeds, as a credible way to assess their behavior. Using CLIP embeddings and their difference to a base-model generation, we concisely define a three-part representation: Directions, defining semantic shift; Strength, quantifying the significance of the effect; and Consistency, quantifying how stable the effect is. Using these representations, we develop an efficient retrieval framework that semantically matches textual queries to relevant LoRAs while filtering overly strong or unstable ones, outperforming textual baselines in automated and human evaluations. While retrieval is our primary focus, the same representation also supports analyses linking Strength and Consistency to legal notions of substantiality and volition, key considerations in copyright, positioning CARLoS as a practical system with broader relevance for LoRA analysis.

• The paper introduces CARLoS, a novel framework that characterizes LoRAs using CLIP embeddings through semantic direction, strength, and consistency.
• The paper details a retrieval pipeline that leverages cosine similarity across diverse prompts to match query signatures with LoRA representations.
• The paper validates its approach with comprehensive evaluations, including user studies and legal analysis, underscoring its impact on generative AI management.

CARLoS: Retrieval via Concise Assessment Representation of LoRAs at Scale

Introduction and Motivation

CARLoS introduces a prompt-independent, generation-based framework to characterize and retrieve Low Rank Adapters (LoRAs) for text-to-image diffusion models, squarely addressing the inefficiency and unpredictability of discovery in large, community-driven LoRA pools. By systematically analyzing the semantic and stylistic shifts LoRAs induce over a standardized set of prompts and seeds via CLIP space embeddings, CARLoS bypasses unreliable metadata and creator descriptions, establishing a principled vector representation for each adapter. This enables robust, semantically relevant retrieval and offers a foundation for legal and attribution analysis in generative AI ecosystems.

Methodology: Representation and Retrieval Pipeline

The core innovation in CARLoS is the concise CLIP-diff-based tripartite representation of each LoRA:

• Semantic Direction: The mean of CLIP-space difference vectors between LoRA-modified and vanilla model generations across prompts and seeds, encoding the central semantic shift.
• Strength: The mean L2 norm of these difference vectors, quantifying the magnitude of the generative effect.
• Consistency: The average pairwise cosine similarity among a LoRA's difference vectors, measuring effect stability.

For retrieval, CARLoS computes the CLIP-space differential vector induced by appending a text query to a diverse prompt set, matching this query signature against LoRAs' semantic directions via cosine similarity. Critical filtering on Strength and Consistency ensures retrieved LoRAs adhere to the prompt without overriding it or behaving sporadically.

Qualitative Retrieval Comparison

The qualitative efficacy of CARLoS is substantiated in extended retrieval galleries, which juxtapose the top-5 LoRAs selected by CARLoS and Qwen3 textual baselines on a fixed prompt paradigm (e.g., 'Cat sitting on a rock'). CARLoS consistently produces generations that reflect both the semantic essence and visual style encoded in complex queries, outperforming text-based retrieval especially for abstract or nuanced concepts.

Figure 1: Top-5 LoRAs retrieved by CARLoS (top) and Qwen3 baseline (bottom) for diverse queries, demonstrating superior stylistic and semantic alignment in CARLoS results, particularly for abstract styles.

This indicates that behavioral representations, as utilized by CARLoS, more reliably expose latent style, thematic, or content features than text-matching.

Retrieval Diversity Analysis

CARLoS retrieval is not biased towards a handful of popular LoRAs; instead, its top-k selections span a broad subset of the corpus, as demonstrated by the distributional analysis of LoRA retrieval frequency. Most adapters in the database are surfaced at least once, with a long-tailed retrieval distribution, confirming that semantic matching in CLIP space can access the full breadth of stylistic and content diversity present in the LoRA zoo.

Figure 2: Frequency of Top-3 retrievals for all LoRAs, showing broad coverage and non-reliance on a small, popular set; majority of the corpus participates in retrieval.

Quantitative analysis using normalized entropy, Gini coefficient, and effective LoRA count demonstrates CARLoS’s competitiveness in both accuracy and retrieval diversity metrics, outperforming text-based baselines in non-bias and effective component activation.

Human User Study: Preference and Relevance

Robust subjective validation via double-blind human user studies corroborates automated evaluations. Participants systematically preferred LoRA sets retrieved by CARLoS over those surfaced by textual baselines on the axes of image quality, relevance to query, and overall preference. The interface presented image sets for direct A/B comparison against the base prompt (e.g., "pencil sketch"), emphasizing both aesthetic and semantic fidelity.

Figure 3: Screenshot of the double-blind user study interface used to compare image sets generated by CARLoS- and baseline-retrieved LoRAs across multiple criteria.

Aggregated results confirm CARLoS’s tangible advantages in practical use cases for digital artists, LoRA database curators, and onboarding workflows.

Theoretical and Legal Implications

The tripartite CARLoS metrics underpin a framework for both technical and legal attribution analysis. Strength and Consistency can serve as interpretable proxies for substantiality and volition in copyright law, helping platforms to screen for adapters liable to reproduce protected works. Weak and inconsistent LoRAs are unlikely to cross legal thresholds, whereas strong-consistent adapters must be carefully managed with respect to derivative work attribution and platform liability. These metrics thus extend CARLoS’s utility beyond retrieval, positioning it as essential infrastructure for regulatory compliance and ecosystem transparency.

Limitations and Future Directions

CARLoS’s reliance on CLIP space inherits weaknesses in spatial composition and subtle texture modeling; future work should investigate integration with more advanced VLMs. The exhaustive one-time LoRA indexing is computationally expensive (∼7 GPU-hr per adapter); scalable surrogate pipelines or distillation would improve throughput. Extensions to broader backbone models (SD3, FLUX) and adapter types (ControlNets, IP-adapters) will further generalize behavioral representation methodology. Additionally, more granular analysis of LoRA scale dependence on Strength warrants deep empirical investigation.

Conclusion

CARLoS provides a unified, rigorous framework for prompt-independent LoRA characterization, retrieval, and legal assessment, supporting both high-precision semantic search and principled, automated moderation in expanding generative AI ecosystems. The approach fosters standardized, interpretable management of modular adapters and sets the stage for scalable best practices in both creative and legal domains.