TAMUSA-Chat: Institutional LLM Framework

• TAMUSA-Chat is a domain-adapted conversational system designed for institutional use, focusing on tailoring LLM outputs to university-specific policies and terminology.
• The framework integrates supervised fine-tuning with retrieval-augmented generation to ensure responses are accurate, current, and grounded in institutional context.
• Its modular architecture supports reproducible experimentation and responsible deployment, emphasizing structured data acquisition, clear evaluation metrics, and governance compliance.

Searching arXiv for TAMUSA-Chat and closely related university chatbot papers to ground the article. TAMUSA-Chat is a research-oriented framework for building domain-adapted LLM conversational systems in the institutional context of Texas A&M University–San Antonio. It is presented not as a generic open-domain chatbot, but as a modular system for adapting foundation models to university-facing use through supervised fine-tuning, retrieval-augmented generation, and systematic evaluation methodologies. The framework emphasizes contextual grounding in institutional sources, reproducible experimentation with training configurations and evaluation protocols, and responsible deployment practices oriented toward transparency, governance compliance, and public-sector constraints (Alsmadi et al., 14 Feb 2026).

1. Definition, scope, and institutional motivation

TAMUSA-Chat is explicitly framed as a domain-adapted conversational system for institutional use rather than a general-purpose assistant. Its motivating problem is that general-purpose LLMs may hallucinate policies, invent programs, miss local terminology, or answer with outdated information when asked about admissions, degree requirements, student services, or policy. In the university setting described, such failures are treated not merely as quality defects but as governance, privacy, and liability risks. The framework therefore adopts institutional domain adaptation as a practical middle ground between prompt engineering alone, which is characterized as too brittle, and training a model from scratch, which is described as infeasible for most institutions (Alsmadi et al., 14 Feb 2026).

The architecture is organized into five layers: data acquisition, data processing and embedding, model fine-tuning, retrieval and inference, and utilities/configuration. This decomposition is central to its role as a research framework because it allows each component to be varied independently for reproducible experimentation. The system is therefore described as both an application-oriented conversational assistant and a platform for studying how academic institutions can build, evaluate, and responsibly deploy LLM systems grounded in their own public content (Alsmadi et al., 14 Feb 2026).

A defining design premise is that institutional question answering depends on local context. Questions about admissions, prerequisites, financial aid, program names, academic calendars, or policy details may depend on catalog year, policy updates, or department-specific rules. TAMUSA-Chat responds to that premise by combining supervised fine-tuning, which teaches the model institutional style and recurring question-answer patterns, with retrieval augmentation, which supplies up-to-date source-grounded context at inference time (Alsmadi et al., 14 Feb 2026).

2. Data acquisition and corpus construction

The data acquisition layer uses Playwright as the primary crawler because it can render modern JavaScript-heavy pages and interact with dynamic site content better than static HTTP crawlers. Crawling begins from seed URLs spanning academic programs, admissions, student services, research, and other major sections of the university website. Separate modules handle general website crawling, document extraction, and orchestration of the crawl pipeline. The document crawler targets PDFs, Office files, and other downloadable institutional materials such as catalogs, handbooks, and policy documents. Crawling is described as respecting robots.txt and using polite delays to avoid overloading institutional servers (Alsmadi et al., 14 Feb 2026).

Raw material from web pages and documents is normalized through a preprocessing pipeline. HTML is parsed to isolate semantic content such as headings, paragraphs, lists, and tables while discarding navigation boilerplate, ads, and template fragments. Text is cleaned by normalizing whitespace, punctuation, and encodings; duplicate content is removed; low-quality pages are filtered out; and metadata such as page title, publication date, authorship, and content type are preserved. For PDF and Office documents, the framework uses PyPDF2, pdfplumber, python-docx, and openpyxl, with an effort to retain structural cues such as sections and table layouts (Alsmadi et al., 14 Feb 2026).

The first release reports the following corpus statistics.

Artifact	Reported value
Web pages crawled	3,847
Documents	412
Cleaned tokens	2.4M
Unique instruction-response pairs	8,932
Average response length	127 tokens
Median response length	89 tokens

These figures indicate a corpus that is modest in absolute scale but institutionally specific. The framework also attaches governance safeguards to acquisition: only public content is collected, personally identifiable information is screened, subject matter experts review content for sensitivity and appropriateness, and periodic recrawling refreshes the corpus while retaining historical snapshots for change tracking (Alsmadi et al., 14 Feb 2026).

3. Preprocessing artifacts, embeddings, and supervised fine-tuning

A key preprocessing artifact is a standardized JSON representation of each document, with fields such as source_url, title, content, metadata, and a sections array. This structure preserves hierarchical organization and improves traceability, because retrieved chunks can be linked back to their original sources. For supervised fine-tuning, the pipeline converts institutional text into instruction-response pairs through direct question generation from content, FAQ extraction from pages that already contain Q&A, synthetic question generation using LLMs for less structured text, manual validation and curation, and reformulation and augmentation to diversify question phrasing (Alsmadi et al., 14 Feb 2026).

For retrieval, document sections are chunked semantically rather than flattened into monolithic text. The chunks are typically 256–512 tokens with overlap to preserve context across boundaries. Each chunk is encoded with a sentence transformer model such as all-MiniLM-L6-v2, and the resulting dense vectors are indexed in FAISS using HNSW for approximate nearest-neighbor search. The stored artifacts are described as tamusa_embeddings.pkl and tamusa_faiss.index, which link vectors back to documents and metadata for provenance and citations (Alsmadi et al., 14 Feb 2026).

The retrieval objective is described conceptually as nearest-neighbor search in embedding space:

$$\operatorname*{arg\,topk}_{i} \; \mathrm{sim}(q, d_i)$$

where $q$ is the query embedding and $d_i$ are document chunk embeddings. The paper also mentions hybrid retrieval in which dense search is combined with BM25 sparse matching and then reranked using a cross-encoder (Alsmadi et al., 14 Feb 2026).

The model fine-tuning layer uses supervised fine-tuning on the institution-specific instruction-response corpus. The framework evaluates open-source foundation models such as LLaMA-2, Mistral, and Falcon, while primarily using SmolLM-135M-Instruct for the reported implementation because it is compact and computationally efficient. The configuration system exposes standard training hyperparameters, including learning rate typically $10^{-5}$ to $5 \times 10^{-5}$ for full fine-tuning, batch size often in the 64–256 range, 3–5 epochs, maximum sequence length around 2048 tokens, and LoRA rank, alpha, and target modules when parameter-efficient tuning is used (Alsmadi et al., 14 Feb 2026).

The framework discusses both full fine-tuning and LoRA-style parameter-efficient fine-tuning. For LoRA, the update is given in the standard form:

$W' = W + \Delta W,\qquad \Delta W = BA$

with $A \in \mathbb{R}^{r \times k}$, $B \in \mathbb{R}^{d \times r}$, and $r \ll \min(d,k)$. Training runs are instrumented with logging of loss curves, gradient norms, learning rate schedules, and memory usage in TensorBoard, reinforcing the framework’s reproducibility and debugging posture (Alsmadi et al., 14 Feb 2026).

4. Retrieval-augmented inference and deployment modes

TAMUSA-Chat uses a hybrid inference strategy. The user query is normalized and embedded using the same sentence transformer used for document indexing. The FAISS index retrieves the top $k$ relevant chunks, typically $q$0 to $q$1, with the observation that top three chunks often work best in practice because too many passages can overwhelm the model. The retrieved content, the user query, and a system prompt are then passed to the fine-tuned model (Alsmadi et al., 14 Feb 2026).

A simplified inference prompt template is reported as: “You are a helpful assistant for Texas A&M University-San Antonio. Use the following information to answer the user’s question accurately and completely. If you cannot answer based on the provided information, say so clearly.” Controlled decoding parameters are used to balance factuality and fluency, with default settings of temperature $q$2, top-$q$3, and repetition penalty $q$4. The generation process is summarized as conditional decoding over the user query and retrieved context:

$q$5

where $q$6 denotes the retrieved context (Alsmadi et al., 14 Feb 2026).

The retrieval pipeline also includes query reformulation, dense plus sparse retrieval, and reranking. This produces a three-stage relevance pipeline: reformulate the query for better recall, retrieve candidates using dense embeddings and BM25, and rerank candidates with a cross-encoder. This is presented as particularly suitable for institutional search because exact program names or policy terms can matter as much as semantic similarity (Alsmadi et al., 14 Feb 2026).

Deployment is intentionally flexible. The framework describes a FastAPI-based REST service for integration into institutional systems, interactive web UIs via Gradio or Streamlit, Docker containerization for portability, cloud GPU deployment for scalable serving, and local API deployment for secure or on-prem institutional use. YAML-based configuration and structured JSON logging are used to support versioned, reproducible deployments across development, staging, and production contexts (Alsmadi et al., 14 Feb 2026).

5. Evaluation posture, compute trade-offs, and responsible deployment

The evaluation methodology is described as combining automated metrics with human assessment, although the reported benchmark results in the summarized version are limited. The framework references BLEU, ROUGE, and METEOR as standard text-generation metrics for summarization and QA-like outputs, while also treating factual correctness, groundedness, and user experience as necessary dimensions of evaluation. A central empirical observation is that smaller models can adapt effectively after relatively short supervised fine-tuning runs, which supports the use of compact models such as SmolLM-135M-Instruct for institutional experimentation (Alsmadi et al., 14 Feb 2026).

The resource discussion is explicit. Smaller models and parameter-efficient methods can run on a single high-end GPU, whereas larger models require distributed training using PyTorch DistributedDataParallel. The training system is designed to integrate with institutional HPC infrastructure, including SLURM or PBS schedulers on the CAMSA cluster. The framework treats model size, memory footprint, training time, latency, and infrastructure complexity as part of a quality-cost trade-off rather than as isolated engineering concerns (Alsmadi et al., 14 Feb 2026).

A practical implication of the design is that retrieval acts as both a quality mechanism and a maintenance mechanism. By grounding responses in retrieved institutional sources, the system reduces hallucination risk and improves informational freshness. By storing knowledge externally in the retrieval corpus, it also reduces the need to retrain the model whenever policies change (Alsmadi et al., 14 Feb 2026).

Reproducibility and governance are major themes. The codebase is public at https://github.com/alsmadi/TAMUSA_LLM_Based_Chat_app, and the framework emphasizes version-controlled code, configuration files, structured logging, transparency in training data provenance, mitigation of hallucinations through grounding, bias monitoring, clear boundaries about authoritative information, PII screening and human review, public-only data collection, compliance awareness including FERPA in the references, and accountability through logging and escalation. The system is designed so that users are informed that the chatbot should not be treated as an unquestionable authority and that important decisions should be verified against official sources (Alsmadi et al., 14 Feb 2026).

6. Position within university chatbot research

Within adjacent university-chatbot literature, TAMUSA-Chat belongs to a class of systems that ground answers in institutional sources rather than treating campus support as open-domain dialogue. Comparable systems differ in scope, modality, and evaluation design. The following examples illustrate that broader landscape.

System	Core grounding strategy	Distinctive emphasis
KUET multimodal assistant	RAG over the official KUET university handbook in PDF form	Text and image queries; hallucination reduced from 31.7% to 6.6% (Shaikh et al., 1 Jul 2026)
Sabiá	RAG over official public institutional PDFs from UTFPR	Model comparison across quality, latency, and LLM-as-a-Judge (Rodrigues et al., 13 Nov 2025)
SAMCares	RAG over a fixed database of SHSU course materials	Adaptive study support with optional upload of additional study materials (Faruqui et al., 2024)
VirtualTA	Syllabus-driven knowledge discovery and QA	Platform-independent, curriculum-oriented assistant with web, Discord, and Google Assistant delivery (Sajja et al., 2023)
ChatISA	Prompt-engineered, multi-model departmental chatbot	Four modules: Coding Companion, Project Coach, Exam Ally, and Interview Mentor (Megahed et al., 2024)

This comparison suggests that TAMUSA-Chat is distinguished less by a single end-user feature than by its framing as a research-oriented framework for institutional LLM deployment. Its emphasis on modular data acquisition, supervised fine-tuning, retrieval, deployment portability, and governance-aware reproducibility places it close to the infrastructure end of the university-chatbot spectrum (Alsmadi et al., 14 Feb 2026).

The literature summarized under related names also shows why the term “chat” can be misleading. In one direction, it can refer to a simple live client-server chat room built with HTML, CSS, JavaScript, AJAX, PHP, and MySQL, with public and private messaging and chat history recording (Thakur et al., 2021). In another direction, it can denote multilingual communication support through chat translation error detection using multilingual BERT and the BPersona-chat corpus (Li et al., 2023). TAMUSA-Chat belongs to neither of those categories. It is instead described as a domain-adapted LLM conversational system for institutional research and responsible deployment (Alsmadi et al., 14 Feb 2026).