ragR: Retrieval-Augmented Generation and RAG Assessment in R

Abstract: Retrieval-augmented generation (RAG) combines document retrieval with LLMs to produce responses grounded in external evidence. While several R packages support core components of RAG workflows, integrated evaluation of RAG systems in R remains limited and is often conducted through Python-based tools, most notably the RAG assessment (RAGAS) framework. To address this gap, we introduce ragR, an R package that unifies document ingestion, embedding and vector storage, similarity-based retrieval, grounded generation, structured question-answer logging, and RAGAS-style evaluation within a single R-native workflow. The current implementation provides LLM-based scoring for four core RAGAS metrics: context precision, context recall, faithfulness, and answer relevance. Validation experiments under controlled settings show that ragR captures similar metric behavior to the reference Python RAGAS workflow across multiple use cases. By integrating RAG construction and evaluation within a reproducible workflow in R, ragR provides a practical framework for research, teaching, and moderate-scale experimentation on RAG systems entirely within the R ecosystem.

• The paper's main contribution is the design of ragR, an integrated R-native framework for constructing and evaluating RAG pipelines with persistent QA logging.
• The methodology encompasses a three-stage architecture that includes document ingestion with variable chunking, vector-based retrieval, and LLM-driven answer generation.
• Empirical results show high Pearson correlations with Python RAGAS metrics, confirming ragR’s effectiveness for reproducible research and teaching in R.

Integrated R-Native Pipeline for Retrieval-Augmented Generation: An Analysis of "ragR: Retrieval-Augmented Generation and RAG Assessment in R" (2604.23515)

Overview and Motivation

"ragR: Retrieval-Augmented Generation and RAG Assessment in R" (2604.23515) presents an R package, ragR, designed as a unified, fully native framework to both construct and evaluate retrieval-augmented generation (RAG) pipelines within the R ecosystem. The principal contribution lies in fusing document ingestion, vector-based retrieval, LLM-based answer generation, persistent QA logging, and RAG assessment using RAGAS-style metrics, enabling fully reproducible end-to-end RAG experiments without reliance on Python-based components. The package is positioned as a response to the absence of an integrated R-native solution, given that prior RAG R packages (notably ragnar and RAGFlowChainR) did not include fully integrated RAG assessment, and researchers largely depended on Python tools such as the original RAGAS framework.

Architecture and Workflow

ragR establishes a modular, three-stage architecture:

1. Document Ingestion: Input PDFs or plain-text files undergo cleaning, chunking (with variable, user-controlled chunk size/overlap), and embedding via OpenAI models. All processed data are stored for retrieval in an RDS-backed vector store, with metadata ensuring traceability and supporting multi-collection operation for domain separation.
2. Retrieval-Augmented Generation: User queries are embedded and used to retrieve top-$k$ relevant chunks from the specified collection via similarity search. These context chunks are then integrated into the prompt for LLM-based answer generation. The entire interaction is structurally logged, including all relevant retrieval, prompt, and model-output details.
3. RAGAS-Style Assessment: QA logs form the basis for automatic metric-based evaluation. ragR implements four RAGAS-inspired metrics—context precision, context recall, faithfulness, and answer relevance—via LLM-based judgment closely emulating the Python RAGAS methodology. Each metric targets a distinct facet: retrieval quality, evidence completeness, correctness/groundedness, and user-query fit.

A defining design choice is the persistent, structured QA log: this not only closes the loop between generation and evaluation but also directly facilitates reproducibility across varying experimental settings.

Empirical Validation and Results

Empirical validation centers on alignment between ragR and the reference Python RAGAS framework across controlled, multi-domain use cases. The suite of experiments systematically varied chunk sizes during ingestion and assessed QA performance on three distinct collections: graduate course syllabi, USMLE anatomy corpus, and a policy brief document.

The principal findings are:

• Metric Parity: Across all cases, ragR achieves high Pearson correlation ($>0.97$ in most metrics) with Python RAGAS scores across chunk sizes, with aggregate (RAGAS Overall) correlations near unity. This confirms that the R-native LLM judgment emulates Python RAGAS scoring with high fidelity.
• Predictable Behavior Across Chunking Regimes: Both implementations display consistent trends: larger chunk sizes generally yield increased context precision, recall, and faithfulness, plateauing after certain thresholds, thereby validating both the pipeline and metric logic.
• Persistent QA Logging: The fixed QA logs enable direct diagnosis of how ingestion parameters (e.g., chunk size, overlap) and retrieval settings in RAG pipelines impact final evaluation outcomes, facilitating robust, granular analysis.

Theoretical and Practical Implications

By bridging RAG pipeline construction and assessment within a consistent, idiomatic R environment, ragR substantially lowers barriers for R-centric research and teaching on retrieval-augmented LLMs. The RDS-backed vector store and modular pipeline enable rapid, reproducible experimentation at moderate scale directly in R, emphasizing accessibility, traceability, and diagnostic transparency.

The RAGAS-based metrics advance the field’s movement toward principled, automated evaluation of RAG pipelines, fostering replicable measurement of both retrieval and generation components within hybrid systems. The persistence and structure of QA logs present clear benefits for experiment auditing and computational reproducibility.

Several limitations are acknowledged:

• Scalability Constraints: The RDS vector store targets research-scale applications; high-throughput or low-latency production needs would require external, specialized vector DBs.
• Document Format Support: Currently restricted to PDF and plain text; extending to richer formats (e.g., Word) is a clear direction.
• Retrieval and Metric Coverage: Retrieval is limited to collection selection, lacking fine-grained, metadata-aware filtering; metric coverage is focused on four RAGAS primitives, omitting advanced or emerging assessment targets relevant to some use cases.
• Non-conversational Logging: The design is tailored for single-turn Q&A rather than multi-turn or interactive dialogue.

Implications and Prospects for Future Development

ragR occupies a distinct niche within the toolchain landscape, serving as a research-and-teaching-first framework for R users. Notably, the consistent metric behavior with Python RAGAS will encourage greater cross-platform comparability and catalyze evaluation research in non-Python environments.

Potential directions for extension include:

• Local Model Integration: Enabling pluggability of local embedding and LLM models for privacy, cost, or customization.
• Enhanced Retrieval: Allowing metadata filtering and complex retrieval strategies.
• Conversational RAG: Supporting dialogic pipelines with conversational memory/state.
• Expanded Evaluation Suite: Adoption of broader RAGAS metrics or other task-relevant automated assessments.

These developments could further the use of R in applied LLM research, especially in education and reproducibility-focused study designs.

Conclusion

ragR (2604.23515) offers an R-native, unified approach for constructing, executing, and evaluating retrieval-augmented generation pipelines, closing a long-standing ecosystem gap. Through architectural modularity, persistent QA logging, and a faithful reimplementation of RAGAS-style metrics, ragR empowers R users to conduct robust research on document-grounded LLM workflows and their evaluation without external dependencies. The convergence of empirical results with the Python reference framework cements its utility for experimentation and replication. While not optimized for production at scale, ragR is poised to be foundational for R-based research, pedagogy, and methodological studies on RAG systems and their evaluation.