AquiLLM: Private Tacit Knowledge Retrieval

• AquiLLM is an open-source, lightweight RAG system that aggregates informal scholarly data like emails, notes, and meeting transcripts.
• It employs a six-stage pipeline—document ingestion, chunking, embedding, hybrid retrieval, and LLM tool-calling—to ensure precise, multi-modal search.
• Designed for private, self-hosted academic environments, it enhances onboarding and archival conflict resolution while balancing privacy with retrieval performance.

Searching arXiv for the specified paper to ground the article in the published record. AquiLLM is an open-source, lightweight Retrieval-Augmented Generation (RAG) system designed specifically to help scholarly research groups capture, store, and retrieve the informal “tacit” knowledge that normally lives in emails, meeting notes, lab notebooks, training materials, code comments, and other private resources. It is intended for settings in which much of a group’s collective knowledge remains informal, fragmented, or undocumented, even when structured data for analysis and publication is well managed. The system supports varied document types and configurable privacy settings, with an emphasis on self-hosting, modularity, and access to both formal and informal knowledge within scholarly groups (Campbell et al., 25 Jul 2025).

1. Motivation and problem setting

AquiLLM is motivated by the observation that research groups accumulate large bodies of informal artifacts—lab notebooks, meeting minutes, email threads, slide decks, and ad-hoc documentation—that embody the reasoning behind methodological choices, the context around failed experiments, and the “why” behind published results. These materials constitute tacit knowledge: informal, experience-based expertise that underlies much of a group’s work.

The system is positioned against the limitations of traditional search workflows such as Ctrl+F, grep, and BM25. Those methods require exact keyword matches and fail when vocabulary varies across decades of documents or when information is scattered across multiple formats and storage silos. The problem is compounded for new group members, who often lack the insider context needed to know where to search. The accumulation of conflicting versions of protocols or interpretations introduces an additional archival problem: reconciliation becomes laborious, and institutional memory may be lost when people leave.

AquiLLM is therefore framed as a dedicated RAG tool for private scholarly archives rather than a general literature assistant. Most off-the-shelf RAG systems are described as focusing on public or published literature and as not adequately addressing the privacy, format diversity, self-hosting requirements, and minimal-administration constraints of academic labs. The stated niche is single-tenant deployment behind institutional firewalls or VPNs, integration with local NAS and JupyterHub rather than cloud drives, and avoidance of heavy DevOps or vendor lock-in. This suggests that AquiLLM’s central design target is not merely semantic retrieval, but operational compatibility with the governance and infrastructure norms of research groups.

2. System architecture and data pipeline

AquiLLM’s pipeline is presented as a sequence of six stages: document ingestion; chunking and embedding; indexing in Postgres/pgvector; user query to vector retrieval with hybrid reranking; LLM tool-calling to perform search and compose responses; and response assembly and delivery in a chat UI (Campbell et al., 25 Jul 2025).

The document ingestion layer accepts arbitrary files, including PDFs, TeX, plain text, audio transcripts, and images, and also supports direct import from arXiv and Zotero. Each uploaded document is represented by a Document subclass such as PDFDocument or TeXDocument. The overridden save() method automatically splits the document into contiguous TextChunks, exemplified as 500-word windows with overlap. This chunk-centric design is the basis for downstream embedding, retrieval, and citation.

On chunk creation, the system computes a vector embedding for each chunk according to

$e_n = f_{enc}(chunk_n).$

Embeddings are stored in PostgreSQL through the pgvector extension, while object storage such as MinIO or AWS S3 stores the original files. At query time, the user’s question $q$ is encoded with the same encoder,

$e_q = f_{enc}(q),$

which enables chunk-query similarity search within a shared representation space.

The architecture also includes a custom LLM abstraction layer with three core classes: Conversation, LLMInterface for provider adapters, and LLMTool. Retrieval is exposed as an LLMTool, allowing the model to invoke

search(collection_id, query_string)

to fetch relevant chunks on demand. This tool-calling arrangement is explicitly contrasted with a strategy in which all chunks are appended at once; the model instead plans when and what to search. Once sufficient context has been gathered, the model emits a final ANSWER message grounded in retrieved snippets, and the chat interface displays the answer with citations including collection names, document titles, and chunk locations.

3. Retrieval methodology and prompting model

AquiLLM uses a dense-vector retriever with hybrid re-ranking. Query-time retrieval combines vector similarity with a trigram-based lexical signal intended to approximate BM25-style matching. The vector component is defined by cosine similarity:

$$score_{vec}(q,d) = \frac{e_q \cdot e_d}{\|e_q\| \times \|e_d\|}.$$

In the more detailed retrieval description, the same cosine form appears as

$$score_{cos}(q,d) = \frac{e_q \cdot e_d}{\|e_q\| \times \|e_d\|}.$$

The retrieval procedure is described in six steps: embed all text chunks via encoder $f_{enc}$; compute $e_q = f_{enc}(q)$ for a query; perform indexed nearest-neighbor search in pgvector for top $N$ by cosine similarity; apply a lightweight trigram lookup to boost exact or near-lexical matches; rerank the union of vector- and trigram-selected chunks by a weighted combination of the two scores; and finally filter duplicates while enforcing per-document snippet caps to avoid biasing long documents (Campbell et al., 25 Jul 2025). The use of per-document caps is significant because it targets a common failure mode in chunk-based retrieval systems, namely overrepresentation of long documents in the returned evidence set.

Prompt construction is explicitly grounded in retrieved context. Snippets are numbered and prefixed with metadata such as title, date, and author. The prompt template is illustrated as:

You are an AI assistant for the X research group. Use only the following context to answer. Context 1 (Doc A, p. 12): ‘…’ Context 2 (Doc B, p. 7): ‘…’ Question: q Answer:

An example trigger template further clarifies the tool-calling loop. The system message states, “You are a helpful assistant that cites private group documents.” A user then asks, “When did we decide to switch instrument calibration from method α to β?” The LLM decides to call the tool search('CalibrationNotes', 'switch from α to β'), receives a snippet, and responds with a citation-grounded answer referring to meeting notes dated 2022-05-10. This example indicates that AquiLLM is designed not only to retrieve relevant passages but to operationalize provenance in the final answer.

4. Privacy, permissions, and deployment model

Privacy and access control are core design dimensions rather than ancillary features. Documents are organized into Collections, each of which is private to its creator by default. Owners can grant read or edit permissions to specific users or groups. Collections may also nest, forming a hierarchy analogous to file directories, so that teams can separate group-wide knowledge stores from project-specific ones.

Enforcement relies on both authentication and application-level authorization. Authentication and single-sign-on are implemented through django-allauth, with support for Google, Microsoft, GitHub, ORCID, Globus, and related identity providers. Authorization is enforced through Django’s built-in permission system and ORM constraints, ensuring that only authorized users may read documents, execute searches, or run LLM tools on a given collection (Campbell et al., 25 Jul 2025). This is a notable distinction from many public-document RAG systems, where retrieval is often modeled primarily as an information access problem rather than a permissions-sensitive workflow.

The deployment model is explicitly flexible with respect to data sovereignty. An entirely on-premises configuration can host Django, Postgres, MinIO, and Ollama for local LLM inference behind a VPN. A hybrid configuration can use a public cloud instance with an LLM API such as OpenAI or Anthropic when privacy requirements are less stringent. The system states that there is no third-party access to private documents unless the administrator explicitly configures external API calls. A plausible implication is that AquiLLM treats inference topology as part of the privacy model, not merely as an implementation detail.

A common misconception would be to regard AquiLLM as primarily a cloud chatbot over private files. The described system instead emphasizes self-hosting on university hardware, a robust permission model, and single-tenant operation, indicating that archival control and institutional governance are first-order requirements.

5. Use cases, deployments, and reported performance

Preliminary deployments are reported in at least two research settings. In the UCLA Astronomy Group, a Jetstream2 instance hosts AquiLLM with collections of papers, meeting transcripts, and code documentation. A new PhD student used it to “catch up” on why pipeline decisions were made and reported faster onboarding. In a Wildlife Camera-Trap Team, the system was used for ingesting training videos and meeting recordings in order to retrieve best practices for image annotation and model pipeline choices.

The reported benefits are operational rather than benchmark-centered. The system is said to improve search precision for informal notes, with precision@5 improving from approximately ~20% with grep to approximately ~70% with AquiLLM retrieval. It also reduces the time needed to answer complex, multi-document questions from hours to minutes and offers historical conflict reconciliation by surfacing temporally ordered snippets (Campbell et al., 25 Jul 2025). These claims are framed as outcomes from preliminary deployments rather than formal large-scale evaluation.

Response latency is reported as approximately ~1–2 s additional per RAG cycle, defined as tool call plus LLM. Precision@k is described as having been measured qualitatively by lab users, and beta feedback is summarized by the statement, “I no longer have to hunt through a dozen directories for that one email.” Taken together, these details indicate that AquiLLM’s initial evidence base is practice-oriented: user-perceived retrieval quality, answer time reduction, and onboarding support are foregrounded over standardized benchmark comparisons.

6. Limitations, misconceptions, and future directions

The current system is tuned for single-tenant groups with hundreds, rather than tens of thousands, of users. The combination of a Django monolith with Postgres/pgvector is noted as beginning to lag at very large scale. This constrains the system’s intended operating regime and differentiates it from enterprise-scale document platforms. It would therefore be misleading to interpret AquiLLM as an internet-scale or institution-wide search appliance.

A second limitation is the privacy-versus-inference-performance trade-off. Local models accessed through Ollama are described as slower and lower-quality than cloud-hosted GPT-4. The system also has limited multimodal support: audio transcripts and images are stored, but are rarely searched semantically. In addition, there is no native conflict detection beyond date metadata; the LLM still makes final judgments on contradictory sources (Campbell et al., 25 Jul 2025). This is important because tacit knowledge archives often contain divergent accounts, superseded protocols, and unresolved interpretations.

Future work is defined along several axes. The paper proposes expanded multimodal retrieval through vector search over audio embeddings and image captions; a lightweight conflict-resolution module that flags temporally contradictory passages for user review; formal benchmarking across more domains with metrics including recall@k, MRR, and hallucination rate in RAG answers; domain adaptation through fine-tuning of the retrieval encoder on group-specific vocabulary and concepts; and UX enhancements such as a visual timeline of decisions and threaded Q&A that can spawn new collections automatically. These directions suggest a transition from a pragmatic group-memory assistant toward a more formally evaluated and modality-aware scholarly knowledge system, while retaining the original emphasis on privacy-aware, self-hosted retrieval.