Non-Display Smart Glasses Overview
- Non-display smart glasses are wearable devices that rely on sensors and non-visual outputs to provide context-aware, hands-free interaction.
- They integrate modalities such as voice, haptics, and biosensing to support assistive navigation, communication, and real-time decision-making.
- Key challenges include ensuring sensor accuracy, managing power efficiency, addressing privacy concerns, and achieving effective multimodal fusion.
Searching arXiv for papers on non-display smart glasses, interaction, and on-device sensing. Non-display smart glasses are wearable computing systems in which the primary interface is not an optical AR display but a combination of sensing, on-device or edge inference, and non-visual output such as speech, spatial audio, or haptics. Recent work spans voice-only assistive glasses, haptic navigation systems, sensing-first eyewear for event cameras or biopotentials, and boundary cases in which active display hardware is moved off the head and the glasses themselves become passive optics (Ding et al., 8 Apr 2026, Bonazzi et al., 5 Jun 2026, Tokmurziyev et al., 4 Mar 2025, Akşit et al., 2022). This diversity suggests that the category is best understood functionally: such glasses are heads-up, hands-free devices that use the frame as a site for microphones, accelerometers, cameras, electrodes, or speakers, while relocating interaction away from conventional screens.
1. Conceptual scope and architectural forms
A narrow definition appears in systems such as CollabLens, whose authors explicitly use “smart glasses” to mean “voice-only interaction glasses with multimodal input ability, without augmented reality displays” (Ding et al., 8 Apr 2026). A broader systems definition appears in Egocentric Co-Pilot, where the glasses are treated as a sensing and communication hub—camera, microphone, network—while the effective interface is conversational and context-aware rather than visual (Yang et al., 1 Mar 2026). OpenGlass pushes the category further toward sensing-first eyewear, describing an open-source platform optimized for onboard perception and on-device ML rather than display driving (Bonazzi et al., 5 Jun 2026). A complementary capture-first interpretation appears in the practical stereo depth system for smart glasses, where the glasses primarily acquire synchronized egocentric imagery and the phone performs rectification, depth estimation, and rendering (Wang et al., 2022).
A useful architectural summary is:
| Archetype | Representative systems | Primary interaction/output |
|---|---|---|
| Voice-only or audio-first glasses | CollabLens, Egocentric Co-Pilot, “ASR-LLMs-Smart Glasses” | Spoken queries, TTS or audio feedback |
| Haptic or non-visual control glasses | LLM-Glasses, STEALTHsense | Temple haptics or silent command triggers |
| Sensing-first on-device eyewear | OpenGlass, GAPSES, VergeIO, ElectraSight | Gesture, eye, EEG/EOG, or context inference |
| Off-head active-display boundary case | HoloBeam | Passive eyewear; external projector |
An important boundary case is HoloBeam. The system is described as reducing the glasses to an almost paper-thin passive optical film while moving active display hardware to an external holographic projector 0.3–2 m away; its prototypes report submillimeter thickness, 24 cycles per degree, and a 70 degrees-wide field of view (Akşit et al., 2022). This broadens the topic from “glasses with no display” to “glasses with no active on-head display subsystem.”
2. Interaction channels and input modalities
The interaction vocabulary of non-display smart glasses extends the earlier taxonomy of touch, on-body, hands-free, and freehand interaction surveyed for smart glasses more generally (Lee et al., 2017). In recent non-display systems, however, the dominant tendency is toward subtle, low-visibility, always-available channels embedded directly in the frame.
One class uses oral or cranial micro-motions. STEALTHsense detects subtle teeth clicks through accelerometers embedded in the nose pads, using a 2-channel sensor stream sampled at 48 kHz, notch and band-pass filtering, 41 features per frame, and a temporal broadcasting neural network with about 88K trainable parameters and 7.14M MACs per second of input (Mohapatra et al., 2024). It recognizes a single click, a double click, and a broad “no pattern” class, and reports balanced accuracy of 0.93 on clean test data and 0.91 under noisy conditions. Because the sensing point is in physical contact with the nose, the approach is relatively insensitive to environmental acoustic noise and is explicitly positioned as suitable when there is no or minimal visual display.
A second class uses eye activity. VergeIO frames vergence as a depth-aware eye gesture channel for glasses, with near, mid, and far fixation distances of 30 cm, 70 cm, and 200 cm, and a temple-to-nose EOG layout that improves average SNR from about 1.6 dB in a JINS-like layout to about 8.9 dB (Zhang et al., 2 Jul 2025). Personalized models distinguish four and six depth-based gestures with accuracies spanning roughly 83–98%, and the system adds both motion-artifact filtering and a brow-raise preamble to reduce false positives. ElectraSight pursues a related but distinct design point: a hybrid contact and contactless EOG system that operates fully onboard within 79 kB of memory, reports 81% accuracy for 10 classes and 92% for 6 classes, achieves 90% movement detection within 60 ms, and supports over 3 days of continuous operation on a 175 mAh battery (Schärer et al., 2024).
A third class uses broader biopotential sensing. GAPSES integrates 8 EEG and 3 EOG channels into a 40 g glasses form factor, with fully dry soft electrodes, a GAP9 edge processor, and total system power of 16.28 mW for continuous EOG acquisition and inference (Frey et al., 2024). In its EOG eye-movement classification task it reaches 96.68% accuracy on 11 classes, corresponding to an information transfer rate of 94.78 bit/min, and the paper further reports that ITR can be increased to 161.43 bit/min by reducing accuracy to 81.43%. The same platform also reports EEG-based biometric recognition with sensitivity 98.87% and specificity 99.86%.
A fourth class uses egocentric vision rather than direct body sensing. OpenGlass evaluates event-based hand-gesture recognition on glasses using a Prophesee GENX320 camera and reports 83.94% cross-subject accuracy with macro F1 of 0.781 for an R(2+1)D model under leave-two-subjects-out validation (Bonazzi et al., 5 Jun 2026). This suggests that non-display interaction need not be voice- or bio-signal-centric; it can also be mediated by low-power visual sensing in front of the user.
3. Audio-centric pipelines, conversational loops, and acoustic mediation
A major branch of non-display smart glasses research treats the device as an audio-first interface to LLMs. The “ASR-LLMs-Smart Glasses” loop decomposes the system into ASR, LLM, and a smart-glasses output module, with ASR using 20–30 ms overlapping frames, FFT-based feature extraction, MFCC or PLP coefficients, acoustic modeling, and language modeling before text is passed to a transformer pipeline with BPE tokenization and standard decoder-style generation (Wang, 2024). Although that paper explicitly assumes text display on smart glasses, it also states that the output stage is fundamentally modality-agnostic and can be reinterpreted as audio or haptic rendering for non-display devices. Its central ASR accuracy metric is the usual word error rate,
which remains directly relevant when spoken output replaces visual inspection.
Egocentric Co-Pilot extends this audio-first model into a web-native assistive agent. Its reasoning core combines Temporal Chain-of-Thought with Hierarchical Context Compression over a unified event log of egocentric video narrations and ASR transcripts, and it reports 40.9% accuracy on Egolife QA and 46.2% on HD-EPIC QA, outperforming several monolithic multimodal baselines (Yang et al., 1 Mar 2026). The same system deploys a WebRTC pipeline for streaming speech, video, and control messages, and a human-in-the-loop study on smart glasses yields a mean rating of 4.70/5, higher than several commercial baselines.
Audio mediation also includes front-end hearing and ASR enhancement. FoVNet defines a configurable acoustic field of view by partitioning the 360° horizontal plane into 20 blocks of 18° each and conditioning a lightweight network on the selected block interval, with about 50 MMACS, 0.206 M parameters, and 16 ms latency (Xu et al., 2024). It enhances all speech sources inside the selected region rather than a single tracked talker. MMW addresses a different failure mode—side speech in wearer-centric ASR—by combining a Tri-Mamba Mix Block, a Frame Diarization Mamba layer, and Multi-Scale Group Relative Policy Optimization, reducing WER by 4.95% in noisy conditions (Liu et al., 8 Jul 2025).
At the environmental-audio level, smart glasses can also sense room acoustics. “Blind Identification of Binaural Room Impulse Responses from Smart Glasses” uses an 8-microphone array in the glasses frame, GWPE dereverberation, MVDR beamforming, and a multichannel Wiener filter to estimate multichannel RIRs from a few seconds of speech, then derives BRIRs via eMagLS rendering (Deppisch et al., 2024). The resulting room-acoustic parameter estimates outperform ACE Challenge baselines for reverberation time and direct-to-reverberant energy ratio, and a listening study shows that the estimated BRIRs are often perceptually more convincing than measured BRIRs taken from other rooms of similar size. This suggests that non-display glasses can act not only as conversational devices but also as adaptive audio-AR probes.
4. Embedded compute, power budgets, and hardware–software co-design
A recurrent feature of non-display smart glasses is that the absence of a display does not simplify the engineering problem; it reallocates the budget toward sensors, inference, and power management. OpenGlass exemplifies this shift with a two-domain architecture: an always-on efficiency domain built around an nRF5340 and low-power sensors, and a performance domain centered on a GAP9 RISC-V SoC, PSRAM, flash, and cameras (Bonazzi et al., 5 Jun 2026). Its prototype fits in a Ray-Ban Stories–like 40 g frame, runs a pipelined event-based gesture recognizer with 33.9 ms end-to-end latency, and achieves up to 11.8 hours of continuous on-device ML from a 200 mAh battery.
GAPSES follows a similar logic for biopotentials. It places dry EEG/EOG electrodes and a GAP9-based BioGAP platform into an eyeglasses form factor, reporting 24 µJ per inference for deployed EOG classification, 121 µJ per inference for EEG-based biometric recognition, and more than 12 h of continuous operation from a 75 mAh battery (Frey et al., 2024). ElectraSight compresses this design pattern further by distributing roles between an nRF5340 main board, a GAP9 tinyML coprocessor, and a dedicated QVar sensing board, while keeping continuous acquisition at 7.75 mW and inference time at 301 µs (Schärer et al., 2024).
Low-footprint ML is equally central in non-biopotential interfaces. STEALTHsense reaches robust teeth-click recognition with 88K trainable parameters; its strongest baseline comparisons show that very small models can outperform both classical ML and larger alternatives under the right feature design and augmentation strategy (Mohapatra et al., 2024). A similar compute-aware division of labor appears in the practical stereo depth system for smart glasses, where two front-facing cameras on the glasses stream images to a paired phone, and the full productionized pipeline still runs in less than 1 s on a six-year-old Samsung Galaxy S8 CPU (Wang et al., 2022). This paired-device pattern remains important where the glasses must stay lightweight or battery-constrained.
5. Applications, accessibility, and everyday use
The best-developed applications of non-display smart glasses are assistive, context-aware, and socially situated. LLM-Glasses is a navigation aid for visually impaired users that combines an ESP32-CAM, YOLO-World, GPT-4o reasoning, and temple-mounted five-bar haptic actuators (Tokmurziyev et al., 4 Mar 2025). In three studies it reports an 81.3% average recognition rate across 13 haptic patterns, successful waypoint following in a VICON-based open-space navigation study, and LLM-guided video-evaluation accuracies of 91.8% in open scenarios, 84.6% with static obstacles, and 81.5% with dynamic obstacles. Its design is notable because all guidance is tactile rather than visual or continuously auditory.
CollabLens shifts the emphasis from individual navigation to mixed-vision collaboration. Built on RayNeo X2 hardware but used as a voice-only system with camera and microphone input, it streams 320×240 JPEG-compressed frames at 2 FPS and 24 kHz audio to a local server, which forwards multimodal input to Qwen-Omni-Turbo-Realtime and returns spoken responses through pyttsx3 (Ding et al., 8 Apr 2026). In workshop studies with Blind and Low Vision participants and sighted peers, the glasses expanded BLV participants’ “assistive networks” by letting them read cards, inspect objects, and query the scene more independently, while also creating uncertainty for sighted collaborators about when to intervene. A plausible implication is that non-display glasses reshape social helping relations rather than merely replacing them.
Egocentric Co-Pilot points in a similar direction but with a broader web-assistance scope: nutrition labels, reminders, over-the-board game tutoring, and long-horizon questions about daily activities, all primarily through audio dialogue rather than visual UI (Yang et al., 1 Mar 2026). The month-long collaborative autoethnography of Meta Ray-Ban AI Glasses adds a complementary everyday perspective. It identifies three recurrent success patterns—instant referential problem-solving, understanding unfamiliar knowledge, and decision-making support—and four recurring breakdown patterns: referential incoherence, conflict with human perception, social embarrassment, and voice-only interaction limitations (Zhu et al., 25 Feb 2026). These findings matter because they show that non-display glasses succeed most clearly when visual grounding makes short, situated queries effortless, but they also fail conspicuously when the promise of “shared perception” breaks.
6. Limitations, controversies, and research directions
Across the literature, several limitations recur. First, many interaction channels are user-specific or anatomy-sensitive. STEALTHsense notes strong person-specific spectral signatures in teeth clicks and limits the vocabulary to two patterns to protect robustness (Mohapatra et al., 2024). VergeIO shows that far-involving depth gestures are much more separable than the full six-gesture set, and that comfort degrades when vertical electrodes are added (Zhang et al., 2 Jul 2025). OpenGlass reports that cross-validation gives a more conservative subject-generalized estimate of about 75% than the headline 83.94%, with some subjects substantially harder than others (Bonazzi et al., 5 Jun 2026).
Second, cloud dependence and contextual brittleness remain substantial. Egocentric Co-Pilot explicitly states that on-device models around 0.5B parameters were prohibitive for real-time interaction and that current deployment assumes high-bandwidth environments, while CollabLens documents latency, verbosity, and hallucination as practical barriers during social activity (Yang et al., 1 Mar 2026, Ding et al., 8 Apr 2026). The autoethnography of everyday Ray-Ban use similarly shows that confident but wrong visually grounded answers are more damaging than generic VA errors because they undermine the premise that the system “sees what I see” (Zhu et al., 25 Feb 2026).
Third, privacy and social acceptability are unresolved. Audio-first operation avoids some display-related concerns, but persistent cameras and microphones raise bystander and recording-status issues, and speaking to the device in public remains embarrassing for some users (Ding et al., 8 Apr 2026, Zhu et al., 25 Feb 2026). This has motivated design proposals for private or hybrid interaction alternatives, including gesture-based, gaze-based, or haptic input, and more explicit uncertainty handling.
Fourth, the category itself has open boundaries. HoloBeam demonstrates that active display hardware can be removed from the glasses while preserving high-quality visual output via external projection, but it also leaves major challenges in eyebox, head mobility, alignment, and full-color HOE engineering (Akşit et al., 2022). This suggests that “non-display” should not be treated as a purely negative descriptor. It marks a reallocation of system resources and interaction logic toward sensing, embodied control, and environmental adaptation.
The dominant future directions are therefore convergent rather than singular: personalization for anatomy-dependent inputs, multimodal fusion across microphones, IMUs, cameras, and electrodes, event-driven power management, privacy-first hybrid architectures, and interfaces that can negotiate uncertainty and turn-taking more gracefully (Mohapatra et al., 2024, Bonazzi et al., 5 Jun 2026, Yang et al., 1 Mar 2026). Non-display smart glasses are consequently emerging not as display-less AR leftovers, but as a distinct wearable systems class centered on non-visual interaction, context sensing, and embedded intelligence.