SRWToolkit for Social Robotic Avatars
- SRWToolkit is an open-source, web-based Wizard-of-Oz toolkit for creating and testing screen-based social robotic avatars integrating local LLM inference.
- It enables rapid prototyping through multimodal interactions, real-time operator control, and a modular three-layer architecture separating frontend, backend, and LLM services.
- A preliminary user study demonstrated positive usability, trust, and user experience, highlighting its potential for role-specific conversational agent design.
SRWToolkit is an open-source, web-based Wizard-of-Oz toolkit for rapidly creating and testing social robotic avatars powered by local LLMs. It is designed to reduce the technical effort required to prototype conversational social robots for human-robot interaction research and design by combining WoZ-style rapid prototyping, multimodal interaction, real-time operator configuration, and local LLM inference. In the paper’s terminology, the resulting artifacts are “social robotic avatars,” meaning virtual representations of physical social robots, typically displayed on screens rather than physically embodied platforms (Nilgar et al., 4 Sep 2025).
1. Scope, purpose, and conceptual position
SRWToolkit addresses a familiar systems problem in social robotics: even basic interactive robot prototypes typically require expertise across robotics, AI, dialogue systems, interface design, and infrastructure. The toolkit is intended to make early-stage experimentation faster and more accessible by allowing a system to appear autonomous while hidden control or partial automation drives behavior. The paper positions this within the Wizard-of-Oz tradition, while emphasizing that many prior WoZ systems either focus mainly on manual operator control, do not integrate automated conversational intelligence well, or depend on cloud-based LLM services (Nilgar et al., 4 Sep 2025).
The toolkit’s stated contributions are a modular open-source WoZ toolkit for customizable social robotic avatars, a browser-based interface for both operator control and end-user interaction, multimodal interaction support, integration of local LLMs via Ollama, real-time configuration of avatar and interaction settings, and a preliminary user study reporting positive ratings for usability, trust, and user experience. The paper contrasts SRWToolkit with WoZ4U, LLM Wizards, WebWOZ, and GPT-enabled robot platforms such as QTrobot. The distinction it emphasizes is not merely interface design, but the combination of local LLM deployment, role/persona customization, and a browser-mediated WoZ workflow (Nilgar et al., 4 Sep 2025).
A recurrent misconception is to treat SRWToolkit as a physical robot platform. The paper does not do so. It explicitly frames the system as screen-based avatars rather than physically embodied robots. This makes the toolkit especially relevant where embodiment is unnecessary or impractical, while still preserving conversational behavior, role, voice, and expressive presentation. A plausible implication is that the platform is best understood as a prototyping environment for robot character design and dialogue behavior rather than as a full robotics middleware stack.
2. System architecture and implementation stack
SRWToolkit is organized as a three-layer architecture consisting of a frontend, a backend, and an LLM layer (Nilgar et al., 4 Sep 2025).
| Layer | Implementation |
|---|---|
| Frontend | JavaScript, node.js v20.15.0, yarn v1.22.22, React.js, Redux |
| Backend | Python, FastAPI, Uvicorn, Poetry |
| LLM layer | Ollama with local LLM serving |
The frontend is a browser-based application in JavaScript. It exposes two main interfaces: the Control Panel and the Social Robot Screen. The Control Panel is the operator or administrator interface; the Social Robot Screen is the user-facing interaction surface. Redux is used to manage dynamic avatar state, including animations, blinking, and response indicators. Inputs are transmitted to the backend through persistent WebSocket connections, and audio responses returned by the backend are decoded and played in the browser (Nilgar et al., 4 Sep 2025).
The backend is implemented in Python using FastAPI, served with Uvicorn, with dependencies managed by Poetry. It handles input processing, configuration management, data storage, communication with the LLM runtime, and communication with frontend interfaces. The backend maintains WebSocket connections to both the control panel and the social robot screen, parses incoming data, and routes requests appropriately. MongoDB is used for storage and is containerized with Docker. The paper names two collections: communications, which stores session metadata and configuration, and chat_messages, which stores user input, custom prompts, and LLM response logs (Nilgar et al., 4 Sep 2025).
This architecture is explicitly modular. The paper argues that modularity allows components to be replaced or extended, including future migration from cloud speech services to on-device speech modules. That claim is significant because the system is not fully local end to end: local inference applies to the LLM layer, whereas speech recognition and speech synthesis currently depend on Google Cloud services.
3. Interaction modalities and real-time operator control
SRWToolkit supports three interaction methods: text input, button-activated speech input, and wake-word-triggered voice interaction (Nilgar et al., 4 Sep 2025).
Text input is entered directly on the social robot screen and sent to the backend through WebSocket. Button-activated speech uses MediaRecorder in the browser to capture audio, encode it, and transmit it to the backend. Wake-word interaction uses the browser’s SpeechRecognition API to detect the phrase “Hey Bot” in one section of the paper and “Hey bot” in the workflow description. After wake-word detection, the system continues listening for 5 seconds. The presence of both “Hey Bot” and “Hey bot” in the paper is a small terminological inconsistency rather than a substantive architectural difference (Nilgar et al., 4 Sep 2025).
The Control Panel enables real-time configuration of session and avatar parameters. The operator can initiate sessions, create human-readable communication IDs, configure avatar visuals, select communication language, enable or disable interaction modes, choose LLM models, write custom prompts to define the robot’s role or character, and define voice gender. These configurations are sent to the backend in real time, stored in MongoDB, and monitored via WebSocket status updates every five seconds, including robot state and connectivity (Nilgar et al., 4 Sep 2025).
The paper also explicitly lists configurable avatar attributes as appearance or visuals, behavior, language, voice, voice gender, role or character, and custom prompts. At the same time, it states that it does not provide a more granular schema of avatar customization. For example, it does not specify exact visual parameters, animation settings beyond blinking and animations, or a structured persona-template editor. This suggests that SRWToolkit offers high-level runtime control rather than a formally specified avatar-description language.
4. Dialogue pipeline, local LLM integration, and data handling
Local language-model inference is provided through Ollama, which the paper describes as a lightweight framework for serving LLMs through an API on a local device. The backend connects to Ollama through a one-step port setup linking the toolkit to the local device’s GPU environment. The deployment reported in the paper uses macOS, an M2 Pro, Sequoia 15.0.1, and 32 GB RAM (Nilgar et al., 4 Sep 2025).
The toolkit includes Llama 3.2, Gemma 2.0, Phi 3.5, Qwen 2.5, and Nemotron-mini 1.0, all pulled and served through Ollama’s API. The backend sends HTTP POST requests containing the user query and conversation context, and the LLM returns JSON-formatted responses. The response-generation context includes user text or transcribed speech, conversation history, and the administrator-defined custom prompt. The paper describes this as producing personalized, context-aware responses, but it does not provide exact prompt templates, prompt formatting, token-window strategies, retrieval methods, memory summarization schemes, or response decoding settings (Nilgar et al., 4 Sep 2025).
The dialogue and speech pipeline proceeds as follows. The user interacts on the Social Robot Screen using text or voice. If proactive mode is enabled, browser speech recognition listens for the wake phrase. Text input goes directly to the backend; voice input is recorded in the browser and sent to the backend. The backend transcribes audio through Google Cloud Speech-to-Text, combines the user’s text, conversation history, and the administrator’s custom prompt, and sends this combined context to the selected local LLM via Ollama. The backend then converts the LLM response to speech via Google Cloud Text-to-Speech, and the frontend receives the audio and plays it back. The control panel can change settings in real time during operation (Nilgar et al., 4 Sep 2025).
The paper emphasizes privacy, data control, latency, reliability in local environments, and reproducibility as motivations for local LLM deployment. It also states that the deployment is intentionally restricted to a local network environment for security, and that the LLM server does not store audio or conversational content, only anonymized metadata. However, the system is not fully offline because speech-to-text and text-to-speech still use Google Cloud APIs. This is one of the central trade-offs identified by the paper.
5. Avatar creation, roles, and prototyping workflow
SRWToolkit is designed to let operators create robot characters or roles quickly by changing prompt, model, voice, language, and appearance. The paper’s operator workflow is straightforward: open the control panel, start a session with a communication ID, choose or adjust avatar appearance, select language, enable desired interaction modes, choose a local LLM model, write a custom prompt defining the character or role, select voice gender, interact with the robot through the social robot screen, and iterate settings in real time (Nilgar et al., 4 Sep 2025).
The user study centered on three example roles: hospital receptionist, mathematics teacher, and driving assistant. These roles were chosen to represent diverse contexts and to test whether the toolkit supports flexible persona creation. The paper implies that persona and dialogue style are shaped mainly through custom prompts, LLM selection, language setting, and voice gender. Embodiment remains screen-based, and the paper does not provide concrete example prompts or dialogue transcripts for the three roles (Nilgar et al., 4 Sep 2025).
This role-centric design is important for interpreting the toolkit’s intended use. It is not presented as a framework for full autonomy, robot control, manipulation, or navigation. Rather, it is a rapid prototyping environment for role-specific conversational behavior and interface-level experimentation. A plausible implication is that it can function as a pre-embodiment testbed before deployment to a physical robot, which the paper explicitly mentions as a future direction through systems similar to QTrobot.
6. Evaluation, limitations, and distinctions from similarly named toolkits
The paper reports a preliminary small-scale user study with participants. The sample consisted of 9 male and 2 female participants, with an age range of 24 to 33, mean age , and standard deviation . All had at least an undergraduate degree; 6 reported daily use of AI chatbots, 4 weekly use, and 1 use only a few times per year; 6 had prior experience with conversational robots. Participants were recruited from university students and colleagues. Each participant designed and interacted with three robot roles—hospital receptionist, mathematics teacher, and driving assistant—spending about 30 minutes total, or roughly 10 minutes per role design and interaction task. Afterward, they completed a questionnaire via LimeSurvey using a 7-point Likert scale, following recommendations from Schrum et al. for HRI perception measures (Nilgar et al., 4 Sep 2025).
| Measure | Reported result |
|---|---|
| SUS | 72.87 |
| Trust rating | |
| UEQ Attractiveness | |
| UEQ Perspicuity | |
| UEQ Efficiency | |
| UEQ Dependability | |
| UEQ Stimulation | |
| UEQ Novelty |
The authors interpret these results as indicating a good level of perceived usability, above-average trustworthiness, and generally positive user experience. At the same time, the paper explicitly limits the evidential scope of the study: the sample size is small, participants were largely familiar with chatbot technologies, demographic composition is narrow, and no inferential statistics, hypothesis tests, or condition comparisons are reported. The study is therefore described as an initial formative evaluation rather than strong evidence of generalizable performance (Nilgar et al., 4 Sep 2025).
The paper also identifies several system limitations. SRWToolkit is not fully offline because speech services still rely on Google Cloud APIs. The current system uses screen avatars rather than physical robots, so conclusions may not transfer directly to embodied HRI scenarios. The paper does not report latency benchmarks, model comparison results, failure-rate analysis, speech accuracy metrics, or which exact model was used for each study condition. Proposed future work includes replacing Google Cloud speech components with open-source on-device alternatives, improving and optimizing real-time performance, investigating the effects of role type, LLM choice, voice characteristics, gender presentation, and avatar appearance, and conducting studies with physically embodied robots (Nilgar et al., 4 Sep 2025).
The name SRWToolkit can be confused with unrelated arXiv resources. The dataset includes SRZoo, an integrated repository for deep-learning-based image super-resolution (Choi et al., 2020); rSRD, an R package for the Sum of Ranking Differences statistical procedure (Sziklai et al., 5 Feb 2025); SaFeR-ToolKit, a multimodal safety framework for structured reasoning via virtual tool calling (Xu et al., 3 Mar 2026); and the Savile Row Manual, which describes a solver-independent constraint modelling tool (Nightingale, 2021). These works occupy distinct technical domains. The paper explicitly titled “SRWToolkit” concerns open-source, web-based Wizard-of-Oz prototyping for social robotic avatars powered by local LLMs (Nilgar et al., 4 Sep 2025).