Papers

We present Mix3R, a training-free 3D scene reconstruction framework that enables immersive Mixed Reality (MR) interaction from a single real-world image. Despite recent progress in single-image 3D scene generation, existing methods often struggle with background recovery under occlusion and lack evaluations regarding natural interaction as well as immersion in MR environments. We address these challenges through a structured integration of large-scale model priors, enabling geometrically consistent 3D scene reconstruction without additional training. The framework combines vision–language-guided foreground–background separation with depth alignment from back-projected point clouds to achieve reliable background completion while preserving detailed object geometry. Experiments on the 3D-FRONT dataset show that Mix3R improves reconstruction quality, reducing Chamfer Distance by over 23% and increasing object-level F-score by more than 6%, with additional gains in scene-level perceptual metrics. User studies on in-the-wild images further demonstrate higher user preference for the proposed method across diverse real-world scenes. These results indicate that our approach enables practical reconstruction of interactive MR spaces from an arbitrary single image. Our project page is https://anonymous-17482.github.io/Mix3R/

We introduce MR-Compare, a spatially grounded mixed reality framework that registers heterogeneous 3D reconstructions to the live video see-through (VST) view for visual comparison. Implemented on a Meta Quest 3, it combines a two-stage registration pipeline with a 3D Slider to enable seamless cross-media comparison between the reconstructed past and the live present. Through a real-world benchmarking and exploratory user study involving 30 participants, we evaluated five representative workflows spanning both mesh-based and 3D Gaussian Splatting (3DGS)-based reconstructions across desktop pipelines (Nerfstudio 3DGS, 3DGS-MCMC, and RealityScan mesh) and mobile capture pipelines (Polycam mesh and Scaniverse 3DGS). The results showed that MR-Compare achieved centimetre-level registration in all cases. Desktop 3DGS workflows outperformed the other methods overall, with 3DGS-MCMC showing the lowest registration error (0.91 cm and 0.86 degrees) and the highest visual consistency with the VST. These subjective findings were strongly corroborated by our objective evaluations. MR-Compare also demonstrated high overall usability and low cognitive workload across all tested workflows. To further optimise registration precision for 3DGS workflows and complement our two-room user study, we propose the anisotropy filter, a zero-shot module that leverages Gaussian anisotropies to extract geometric surface proxies. Evaluations on the Replica dataset show that moderate pruning improves robustness and reduces alignment errors. For standard 3DGS, the filter reduced translation and rotation errors by 0.35 cm (37.6%) and 0.08 degrees (38.7%), respectively. For 3DGS-MCMC, it reduced translation and rotation errors by 0.12 cm (16.9%) and 0.04 degrees (29.1%), respectively.

Estimating the physical properties of objects is as important as reconstructing their geometry for achieving physically coherent interactions in mixed reality. Prior work typically analyzes object dynamics observed in video or object semantics inferred from a single image, relying on vision-language models (VLM) for contextual reasoning. These methods are either computationally costly or lack geometric grounding, and consequently fail to capture spatial and contact relationships across objects. To address these challenges, we present SnapPhysics, a framework that reconstructs 3D objects with their physical properties, such as mass, friction or center of gravity, from a single image through instance-level 3D object reconstruction, spatial alignment, and reasoning based on a scene graph. The physics-aware scene graph is central to our approach. It is provided to a VLM along with per-object geometry to infer physical properties. Our experiments on the 3D-FRONT dataset demonstrate that SnapPhysics improves the F-Score on a scene level by 18.6% over previous approaches and achieves competitive performance in physical property estimation, reducing mean absolute log difference error (mALDE) by up to 20.5% and improving linear correlation (r2 ls) by up to 19.6% compared to VLM-only estimation. SnapPhysics enables physically interactive MR experiences from an image without any manual parameter tuning.

Controlling devices connected to the Internet of Things often requires juggling multiple smartphone apps or physical remote controls, creating a fragmented user experience. Augmented Reality (AR) can afford superior control by automatically presenting virtual user interfaces that are spatially aligned with networked devices. However, before such user interfaces can be delivered, physical devices must be localized in the environment. This paper introduces LORIOT (LOcalization-Ready Internet Of Things), a novel end-to-end system that uses a semantic scene graph and a large language model to map the identities of the networked devices to physical objects, given a pre-filtered set of IoT-device candidate nodes. A declarative UI specification enables automatic generation of device control panels for AR and non-AR clients. We evaluate the mapping component on a controlled synthetic-room benchmark of 100 randomly generated rooms. Using device network metadata alone, we achieve a baseline macro-averaged F1 score of 0.80 for digital → physical associations. When device metadata is enriched with physical attributes (mounting location, materials, color, and size), performance improves to 0.88. Moreover, we evaluate the benefit of spatially registered AR control in a within-subject user study (N=20), comparing in-situ AR panels against conventional non-AR control with smartphone apps or physical remote controls. AR yields significantly faster task completion, lower mental demand, and higher usability.

Passive haptic retargeting effectively addresses real-world limitations in virtual reality by repurposing physical proxies. However, existing methods typically rely on single proxies, 2D planar arrangements, or fixed 3D proxies, failing to provide authentic tactile feedback for complex 3D interaction tasks. This limitation stems from the challenges of managing mobile proxies in volumetric space while ensuring directional consistency between virtual and physical contacts. To address these problems, this paper proposes a direction-aware 3D haptic retargeting method. We first design a magnetic physical proxy system that enables stable 3D stacking and robust omnidirectional pose tracking. Central to our approach is the introduction of a Haptic-Direction Score, which quantitatively evaluates non-occupied proxies and available placement positions with Local Exposure, Global Haptic Direction Quality, and Non-occupied Proxy Availability. We also propose a haptic retargeting method based on Haptic-Direction Score, which dynamically maps virtual targets to optimal physical targets during the retargeting process. A user study comparing our approach with state-of-the-art methods demonstrates that our method significantly reduces the system reset rate, while improving task completion efficiency, the consistency of haptic perception, and overall user experience in complex 3D environments.

Redirected Walking (RDW) is a locomotion technique where users can explore large virtual environments while physically moving within a limited real-world space. Existing haptic redirection techniques often rely on infrastructure-heavy setups that limit deployability in real-world XR systems. In this work we investigate whether lightweight, ankle-based vibrotactile feedback can modulate perceptual sensitivity to RDW curvature manipulations. We present a wearable, wireless, gait-synchronized ankle-tendon vibrotactile system that delivers unilateral or bilateral cues during the swing phase of walking. In a within-subject study (N=28) spanning 14 experimental conditions, we systematically evaluate how stimulation side, temporal profile (constant vs. gait-velocity modulated), and curvature direction affect curvature detection thresholds. Results show that RDW effectiveness is strongly state-dependent: slow walking consistently permits larger curvature gains, and spatially asymmetric (unilateral) ankle stimulation produces greater perceptual bias than bilateral cues. Movement-contingent stimulation further improves perceived naturalness, presence, and user acceptance. Together, these findings provide the first systematic investigation of ankle-based vibrotactile augmentation for RDW and demonstrate that portable lower-limb haptics can meaningfully extend redirection capabilities without environmental instrumentation. We derive design guidelines highlighting the importance of locomotor-state adaptation, spatial asymmetry, and gait synchronized feedback for scalable redirected walking in XR.

Avatar motion remapping enables users to experience walking on virtual stairs while physically walking on a flat surface. In stair walking, the downward vertical field of view (FOV) plays an important role in ground recognition and foot placement; however, its perceptual and behavioral effects remain poorly understood, largely due to the FOV constraints of conventional head-mounted displays (HMDs). An extended downward FOV may enhance postural naturalness by supplying peripheral visual cues, yet it could also heighten awareness of the visuo-proprioceptive mismatch inherent in avatar remapping. To address this gap, we investigated the effects of downward FOV on posture control, height perception, and applicability of virtual stair walking using an HMD with an extended downward viewing range. Our results reveal that, without an extended downward FOV, participants actively compensated by tilting their heads downward to acquire necessary visual information, yet this behavior yielded no significant differences in subjective height perception or awareness of the spatial manipulation. In contrast, providing an extended downward FOV significantly suppressed this compensatory head tilt, promoting a natural, forward-facing posture more consistent with real-world stair walking. Subjective evaluations further indicated enhanced experienced realism during ascent and greater general presence during descent. Together, these findings demonstrate that an extended downward FOV effectively fosters natural locomotion behavior and deepens immersion in virtual stair walking, highlighting downward FOV as a key design parameter for locomotion interfaces in VR.

In this work, we explore the integration of Orientation Selection into Steering interfaces, aiming to preserve the seamless sensation of real-world movement while mitigating the risk of inducing cybersickness. Our implementation encounters conflicts in standard input mappings, prompting us to adopt bimanual interaction as a solution. Recognizing the complexity of interaction that may arise from this step, we also develop unimanual alternatives, e.g., utilizing a Human-Joystick, commonly referred to as a Leaning interface. The outcomes of an empirical study centered around a primed search task yield unexpected findings. We observed a sample of users spanning multiple levels of gaming experience and a balanced gender distribution exhibit no significant difficulties with the bimanual, asymmetric interfaces. Remarkably, the performance of Orientation Selection is, as in prior work, at least on par with Snap Rotation. Moreover, through a subsequent exploratory analysis, we uncover indications that Pointing-Directed Steering outperforms embodied interfaces in usability and task load in the given setting.

Avatars for social VR are increasingly driven by imperfect tracking or AI-generated motion, where mismatches between appearance fidelity and behavioral realism can harm user perception. We investigate whether a simulated point-cloud rendering (PCR) can improve social perception and increase tolerance to motion artifacts compared to standard shaded mesh rendering. We present a real-time VR avatar system that supports (i) a streaming conversational loop, (ii) human-controlled embodiment via hybrid tracking using a head-mounted display for upper body and an external camera for lower body and root motion, and (iii) AI-controlled embodiment via context-aware selection from an offline-generated animation library. We evaluate a 2x2x2 within-subjects study crossing rendering style (PCR vs. mesh) and control source (human vs. AI) across two sequential scenarios: a conversation-focused desk interaction and a movement-focused guided exercise. We measure social presence, human-likeness and affinity, and appearance–behavior plausibility, and collect post-study rendering-style preference rankings. Our results show that PCR significantly improves perceived human-likeness and plausibility, with particularly strong benefits for AI-controlled avatars. Crucially, PCR successfully closed the plausibility gap between human and AI avatars during high-movement interactions, acting as a perceptual buffer against motion artifacts. While social presence was driven by the interaction scenario rather than rendering style, post-study rankings revealed a trade-off between perceptual coherence and visual comfort.

Embodied photorealistic avatars are increasingly used in emotionally rich and diverse virtual reality (VR) applications due to their perceptual benefits. However, it is unclear how specific avatar perceptual features (APFs) and individual differences may relate to user physiology across different emotional virtual environments (VEs). In a within-participants study, we investigated the relationship between user neuroticism, APFs and physiological metrics, measured across four emotion VEs systematically manipulating emotional valence and arousal (N=51). Users embodied photorealistic personalised avatars (i.e., mirroring their appearance) and generic gender-matched avatars. We demonstrate that individual APFs have distinct relationships with user physiology, which are generally multiplied and reversed by higher user neuroticism. Moreover, APFs such as familiarity, appeal, appearance similarity and perceived realism can support emotional regulation by increasing heart rate variability (HRV) for higher neuroticism individuals. Overall, our findings highlight how avatar design can be strategically leveraged to optimise affective outcomes across diverse VR contexts.

Personalized, doppelgänger-like virtual agents may increase social realism, yet it remains unclear how users calibrate their perceptions and responses when an agent both resembles them and conveys distinct persona traits. In a 2 x 2 within-group study (N = 24), we examined how multimodal self-similarity and persona valence jointly affect study participants during open-ended conversations with an embodied virtual agent. After each condition, participants reported measures of perceived identity and persona, relational and social experiences, and affective responses; we also extracted conversational behavior from logged data, including word and turn counts and awareness-probing questions. We found that self-similarity increased perceived self-similarity, rapport, and emotional reactivity, while trust showed a valence x similarity interaction. Persona valence produced stronger overall shifts. Positive valence increased perceived agent persona, trust, co-presence, rapport, and willingness for future interaction, whereas negative valence increased emotional reactivity and elicited more words, turns, and awareness-probing questions. These findings clarify how resemblance and persona valence jointly shape user perceptions and interactions with embodied conversational agents and inform the design of self-similar virtual agents.

While virtual reality allows users to embody diverse non-human avatars, continuous body transformation such as growing new limbs or reshaping the body remains largely unexplored. We present a shape-shifter avatar based on metaball based implicit surfaces and examine how control mapping design and prior character knowledge influence embodiment, usability, cybersickness, preference, and plausibility. In a study with 24 participants, we compared three control mappings, Button, Task-Oriented, and Full-Control, across two tasks: extending a tentacle to reach objects and changing body height to search for keys. Prior knowledge was manipulated between participants using animations in which the character either transformed its body or remained in its default form. The Full-Control condition achieved the highest embodiment, plausibility, and preference. The Task-Oriented condition produced higher embodiment than button input by using task relevant body movements as triggers. Prior knowledge increased plausibility related to appearance and world fit, but did not affect evaluations of control correspondence or embodiment. Based on these findings, we discuss interaction design for shape-shifter avatars.

Extended reality (XR) interfaces for assembly specification typically require users to specify the 6DoF pose of every part. This works well when exact poses are necessary, but can impose unneeded constraints when not. We present a collaborative XR constraint-authoring interface that supports a spectrum of specification authority. In our interface, an author can specify groups of parts and transfer them to a human or robot executor that carries out the author's instructions in the physical world. In prescribed mode, the author specifies the 6DoF pose of a part or group. In delegated mode, the author entrusts the executor to determine the 6DoF pose of a part or group. Both modes can be mixed within a single task. We conducted a within-subjects study comparing prescribed-only authoring against dual-mode authoring. Results show that dual-mode authoring significantly reduced authoring task completion time and perceived workload. Most participants significantly preferred dual-mode, with delegation rate scaling with the complexity of authored structures. We also evaluated both conditions offline with a robot executor, feeding authored constraints through a workspace optimizer and motion planner. Delegated constraints achieved significantly higher inter-group clearance than prescribed-only specifications.

We present object-driven event retrieval and an assembled object interface for Mixed Reality (MR) assembly playback, enabling users to select parts on the completed assembly and automatically retrieve and play back the corresponding joining process. When users want to review how a specific part was assembled, time-based navigation requires repeatedly scrubbing through the timeline and existing object-based approaches are limited to single-object queries that cannot address assembly scenarios where inter-part joining relationships are central. The proposed approach extracts inter-part joining events from a scene-graph-based recording and returns the relevant event segment by traversing the shortest path on the assembly graph, while presenting the completed assembly as a scaled-down 3D model so that part selection directly serves as a retrieval query. In a within-subjects experiment with 24 participants across drone, chair, and bicycle assembly scenarios, the proposed Object Interface(OB) significantly reduced navigation time compared to time-based comparison conditions, with the advantage most pronounced in scenarios with many parts or wide spatial extent, and the majority of participants selected OB as the most suitable interface for part-specific lookup. Based on these results, we expect object-based retrieval to support part-specific lookup that is difficult to achieve with time-based navigation alone, particularly in large-scale industrial assembly or maintenance environments with expansive workspaces.

We present ZCAR, a zero-annotation CAD-driven assembly recognition framework for Mixed Reality (MR) guidance that eliminates the need for manual training labels. Unlike generic object recognition, assembly-state tracking must align idealized 3D CAD assets with complex real-world observations, resulting in a substantial sim-to-real domain gap. ZCAR addresses this problem by using the LDraw CAD standard as a supervision source and rendering automatically labeled synthetic training images directly from 3D models, thereby bypassing the conventional data collection and annotation pipeline. The deployed system extracts visual embeddings with a frozen MobileCLIP2-S0 encoder and learns a lightweight projection head (0.33M parameters in the MobileCLIP2-S0 setting) using Supervised Contrastive Learning (SupCon). This design targets fine-grained state discrimination, enabling the model to capture subtle geometric differences between consecutive assembly steps. For resource-constrained spatial computing hardware, we further introduce a K-means-based embedding compression scheme that reduces memory usage while suppressing transient viewpoint noise. We also formulate a human-in-the-loop active perception strategy to resolve viewpoint-dependent ambiguities in which distinct assembly states appear similar from particular angles. In a controlled stress-test replay, the multi-frame Active Vision procedure improves average accuracy from 62.7% to 75.0% under ambiguous viewpoints. By favoring low-latency inference over heavyweight foundation models, ZCAR demonstrates the feasibility of real-time deployment on passthrough MR platforms such as the Meta Quest3.

Debugging Cyber-Physical Systems like Digital Twins requires developers to mentally bridge the gap between two-dimensional source code and three-dimensional physical behavior. While Augmented Reality (AR) offers potential for immersive debugging by spatially registering information, Virtual Reality (VR) might serve as a suitable substitute when physical entities are not (yet) available or are inaccessible. In this paper, we investigate the impact of additional situated visualizations, placed next to the physical entity, versus additional stationary visualizations, placed next to the IDE. Furthermore, we evaluate the degree of virtuality by comparing these situated visualizations across AR and VR environments to assess VR's suitability as a substitute. We conducted a within-subject laboratory study (N=36) using a distributed robotic scenario to evaluate debugging performance and experience across three conditions: Stationary AR, Situated AR, and Situated VR. Results indicate that situated visualizations significantly enhance usability and user experience compared to stationary visualizations, even when objective performance metrics were comparable. Qualitative analysis reveals a trade-off between the spatial awareness of AR and the abstracted focus of VR, as well as a preference for the clear spatial assignment provided by situated panels. Our findings suggest that while situated visualizations do not necessarily speed up the resolution of bugs, they provide a more intuitive and satisfying workflow for identifying and fixing them.

This paper presents HaptiCraft, a modular handheld haptic controller designed to replicate the appearance, mass distribution, and multimodal haptic properties of virtual objects. HaptiCraft consists of many modules that provide distinct functions for assembly and multimodal haptic feedback, supporting five types: vibration, impact, thermal, variable inertia, and variable stiffness. Users can readily assemble these modules to create a controller tailored to their needs. To simplify the design process, especially for novice users, we also provide a graphical authoring tool and assess its usability. Finally, we evaluate the system’s effectiveness through various virtual reality (VR) scenarios, demonstrating that HaptiCraft significantly enhances user experience. HaptiCraft makes a substantial contribution to the field of VR handheld controllers by offering comprehensive support for shape-changing, variable inertia, and rich multimodal haptic feedback, coupled with an effective authoring method.

While forearm-worn interfaces are effective for tactile navigation in VR, the high degrees of freedom in arm posture-exacerbated by handheld controller use-introduce distortions in directional perception. This paper presents a posture-adaptive compensation model designed to deliver consistent 2D azimuthal cues regardless of arm posture, elbow flexion, and wrist rotation during handheld interaction. In our first experiment, we quantified how different arm postures shift perceived azimuth, establishing a foundation for our computational model. Notably, the perceptual shift remained substantially smaller than the physical change in elbow angle, indicating partial compensation between body-centered and device-centered coordinates. Based on these findings, we developed a posture-adaptive compensation algorithm combining representative-posture modeling with lightweight user adaptation. Subsequent VR navigation experiments showed that our model significantly outperforms methods that do not consider posture. This work contributes a methodology for eyes-free navigation, providing reliable spatial guidance in virtual environments by decoupling directional communication from user posture.

Grounded haptic experience is essential for realistic virtual reality (VR) experiences, but existing solutions face a fundamental trade-off. Complex grounded devices and exoskeletons provide convincing force feedback, but limit user mobility and can be challenging to wear. In contrast, handheld and hand-wearable devices can maintain user mobility but do not provide sufficient grounded resistance. We identify a subset of scenarios common to many VR applications in which the user stands or sits in a fixed location (e.g., in a cockpit). In such situations, useful interactions with grounded objects can be realistically rendered by controlling their arm extension or flexion. We introduce GroundedReach, an elbow-wearable device that controls arm extension and flexion with a single-joint architecture. The system combines brake-based Dynamic Passive Haptic Feedback (DPHF) with a servo-controlled ratchet to render both continuous resistance and rigid stopping forces for interactions such as opening heavy doors and contacting hard surfaces. Our evaluation shows that GroundedReach can effectively simulate grounded feedback without applying force to the palm or shoulder. This design enables a compact, lightweight, and easily donnable peripheral that can be readily shared across users.

Haptic feedback is widely employed to enhance immersion in Virtual Reality (VR) environments. However, designing haptic stimuli that cover diverse interaction conditions remains a significant scalability challenge. Data-driven haptic generation has emerged as a promising approach, yet existing models face an inherent trade-off between waveform expressiveness and inference responsiveness, which becomes increasingly critical as training data grow in scale and diversity. To address this challenge, we propose HaptoFlow, a vibrotactile generative model based on Flow Matching, designed for interactive real-time haptic rendering in VR. Flow Matching learns a continuous vector field that transforms a base distribution into the target data distribution, enabling efficient representation of complex haptic data distributions and thereby facilitating both high-quality generation and computational efficiency. We train HaptoFlow conditioned on material labels and interaction parameters (stroking velocity and applied force), and integrate it into a VR system. Technical evaluation demonstrates that HaptoFlow outperforms all baseline methods in both waveform reproduction accuracy and inference latency. Furthermore, user studies confirm that the system latency falls well within the perceptual threshold of visual-haptic delay, and statistically significant improvements in perceived haptic quality are observed for a subset of materials. These findings establish a practical foundation for scalable, data-driven haptic content creation in VR, and provide latency benchmarks that inform the design of future real-time haptic rendering systems.

Embodying non-human body parts in VR is a prevalent practice among certain subcultures, and is a personally important creative outlet to many individuals. However, the discrepant morphology between real and virtual bodies can decrease Sense of Embodiment (SoE). Haptic feedback can compensate by increasing SoE felt towards non-human body parts, but there is a literature gap in comparing the effects of different haptic modalities, and their combinations, on SoE. Through an online survey sent out to social VR communities (n=63), we determined that animal ears are a commonly embodied and ecologically valid non-human body part to study. We then ran a 2x2 within-subjects user study (n=28) with two independent variables: active haptics, delivered through vibrotactile gloves, and passive haptics, delivered through a physical headband, for when participants reach up to touch virtual dog ears appended to their avatar in VR. Our findings show that (1) passive haptics outperform active haptics, (2) combining the two modalities produces an antagonistic effect on SoE, and (3) SoE towards non-human body parts positively correlates with SoE towards the entire avatar. We discuss implications of our findings in various domains, and on embodiment literature.

Avatars whose appearance deviates from that of the physical body are widely used in virtual reality to enable experiences that go beyond biological limitations. To extend prior avatar research beyond subjective and performance-based measures, we systematically examined how variations in hand avatar appearance influence the internal representation of hand geometry. Using a structured set of 26 hand avatars, participants adapted to each avatar and subsequently reported their hand geometry. Results show that internal hand geometry does not change in direct correspondence with avatar geometry and exhibits a higher-dimensional structure than the avatar designs themselves. Furthermore, changes in hand geometry were organized along multiple interpretable dimensions rather than reflecting unsystematic variability. These findings suggest that avatar appearance is reinterpreted through internal bodily constraints. The proposed framework provides a basis for understanding avatar appearance in relation to body representation, supporting more informed avatar design beyond simple geometric mapping.

Combining, otherwise known as `stacking', auditory and haptic cues with visual guidance is a common strategy for improving online error correction in VR training. However, under sparse 6-DoF tracking with inverse kinematics (IK), multimodal benefits may be limited by solver-induced kinematic ambiguity and may not generalize across users. We investigated this question in a VR motor-imitation study ($N=104$) with four feedback conditions: visual only (V), visual+audio (VA), visual+haptic (VH), and visual+audio+haptic (VAH). Results show that multimodal efficacy is constrained by IK topology and learning-stage trade-offs. Along the kinematic chain, feedback effects were absent at solver-driven joints (elbows), exhibited trend-level divergence at sensor-driven joints (wrists), and were strongest at hierarchy-driven distal segments (fingers). Visuo-haptic guidance (VH) accelerated online adaptation relative to the visual baseline, but this benefit did not carry over to the full stack (VAH). Instead, VAH was associated with lower offline cued recall, indicating a trade-off between execution support and short-term retention. We also observed a distal Condition$\times$Gender interaction --- haptic inclusion improved performance in the male group but degraded performance in the female group under the same mapping, consistent with a confirmation-versus-interference pattern. Based on these findings, we derive three design guidelines for IK-mediated VR training --- Topology-Aware Masking, Interference Detection, and Scaffold-and-Wean Scheduling.

Running virtual reality (VR) user studies with older participants is challenging due to mobility and cognitive limitations prevalent in this age group. Therefore, most VR studies opt to bypass this participant demographic, with the consequence that little is known in terms of whether, which, and under what conditions VR interventions can benefit the older members of our society. This paper presents a first step towards improving age representation in VR, not by actually enrolling older participants in the study, but rather by attempting to modify the cognitive and motor behavior of younger participants for it to resemble that of older participants. The behavior modification relies on the Proteus effect, a well-documented change in the behavior of a user of a VR application when provided with an incongruent self-avatar. A user study asked N = 30 participants in the 18 to 35 age group to play a memory game in VR, twice: once with virtual hands with a young appearance, in line with the actual participant age, and once with virtual hands with an old appearance, in contradiction with the actual participant age. The game board is a torus divided into four buttons of different colors. The buttons light up one at the time in random order and the participant has to recall the sequence and play it back by pressing the buttons in the same order. The sequence grows longer each time it is played back correctly. The results show that when presented with older looking hands participants recalled shorter sequences and their hands did not reach as far as when presented with age-congruent hands. The study brings evidence of simultaneous cognitive and motor manipulation of VR user behavior. Specifically, the study shows that both the cognitive and motor behavior of young VR users can be shifted towards behavior commonly associated with older participants. This opens the door to collecting initial data on possible older participant responses to a VR application using only younger participants, by manipulating their avatar.

Spatial audio is crucial for immersive 360-degree video experiences, yet most 360-degree videos lack it due to the difficulty of capturing spatial audio during recording. Automatically generating spatial audio such as first-order ambisonics (FOA) from video therefore remains an important but challenging problem. In complex scenes, sound perception depends not only on sound source locations but also on scene geometry, materials, and dynamic interactions with the environment. However, existing approaches only rely on visual cues and fail to model dynamic sources and acoustic effects such as occlusion, reflections, and reverberation. To address these challenges, we propose DynFOA, a generative framework that synthesizes FOA from 360-degree videos by integrating dynamic scene reconstruction with conditional diffusion modeling. DynFOA analyzes the input video to detect and localize dynamic sound sources, estimate depth and semantics, and reconstruct scene geometry and materials using 3D Gaussian Splatting (3DGS). The reconstructed scene representation provides physically grounded features that capture acoustic interactions between sources, environment, and listener viewpoint. Conditioned on these features, a diffusion model generates spatial audio consistent with the scene dynamics and acoustic context. We introduce M2G-360, a dataset of 600 real-world clips divided into MoveSources, Multi-Source, and Geometry subsets for evaluating robustness under diverse conditions. Experiments show that DynFOA consistently outperforms existing methods in spatial accuracy, acoustic fidelity, distribution matching, and perceived immersive experience.

Augmented reality (AR) telemeetings, also referred to as holographic calls, promise a stronger sense of presence and a more natural experience than regular video calls. Using point-cloud transmission and see-through AR displays, visual rendering aims to create the impression that a remote interlocutor is present in the user's room. Room-adapted, dynamic spatial audio rendering should provide the corresponding auditory impression. However, it is still unclear whether room-adapted spatial audio enhances perceived quality and social presence in AR telemeetings, and how its effectiveness is influenced by practical constraints, such as suboptimal room-acoustic match, increased audio capture distance, and lossy audio compression. In our experiment, pairs of participants engaged in conversations on pre-defined topics using an AR holographic calling system under eight spatial audio rendering conditions, which varied in room-acoustic rendering (no room, matched room, too dry, too wet), audio capture distance (close vs. far), and compression (lossless vs. low bitrate). After each condition, participants completed an eight-item questionnaire, rating audiovisual coherence, audio quality, overall experience, and social presence. The results show that room-adapted spatial audio can significantly enhance overall quality and social presence in AR telemeetings compared to no room acoustic rendering. Surprisingly, we also found improved experiences with relatively large mismatches between rendered and physical room acoustics. Furthermore, increasing capture distance or applying lossy audio compression degraded the experience and eliminated the benefits of room acoustic rendering. These findings provide design guidelines for spatial audio rendering in future AR telemeeting systems: to maximize presence, room-acoustic rendering should be added to high-quality audio from close-microphone capture, even when moderate room acoustic mismatches are expected.

Augmented/mixed reality (AR/MR) wearable glasses now permit information interaction anywhere, but visual displays can be inappropriate when real-world awareness is essential. We propose \textit{Switched Reading}, a novel interaction framework for reading text in AR/MR that supports switching between visual and auditory modalities as needed. Specifically, we explore two key interaction techniques within this framework: (1) gaze-based voice playback and (2) a correspondence-aware transition effect. We implemented them on an MR headset through a parameter-tuning user test. Next, we conducted a user study (N=16) to investigate the impact of the two techniques on reading performance and overall user experience with simulated modality switching in virtual reality. The results show that the condition combining both techniques was the most preferred among four conditions. Moreover, we found that the gaze-based voice playback reduced gaze offsets when switching modalities and improved reading speed over the baseline condition using scroll position. Finally, we implemented a Switched Reading application for reading while walking and collected user feedback, yielding further design implications for practical use.

Large-scale immersive virtual reality (VR) systems often rely on physical loudspeakers for spatial audio. However, practical constraints such as venue geometry, hardware placement, and multi-user safety requirements frequently hinder the precise co-location of physical sound sources and virtual visual targets. This leads to inherent audiovisual (AV) directional discrepancies. Although AV spatial alignment is extensively studied, the perceptual limits of such discrepancies in real acoustic environments remain poorly understood. In this work, we investigate how physical sound source distance affects the detection threshold (DT) for AV directional offsets. We conducted a within-subject psychophysical experiment using the method of constant stimuli. Horizontal azimuth deviations were systematically manipulated across multiple source distances in a physical acoustic environment. Participants heard the audio directly from physical loudspeakers while simultaneously visually viewing a virtual avatar. Our results indicate that DTs increase significantly with source distance. This trend is accompanied by broader perceptual tolerance and flatter psychometric functions, reflecting a distance-dependent decline in directional localization sensitivity. These findings offer empirical insights into the perceptual limits of spatial audio. Furthermore, they provide actionable guidelines for loudspeaker deployment and AV calibration in large-scale VR systems.

Affective computing research in social Extended Reality (XR) has largely relied on passive emotion elicitation paradigms and single-perspective annotations, leaving open how emotional states are transmitted and perceived across different observational perspectives in avatar-mediated interaction. We introduce Remote Embodied Improvisation (REI), a dataset collected through active dyadic emotion elicitation in a full-body avatar XR environment, annotated simultaneously from three perspectives (self, interaction partner, and third-party observer) along the Valence, Arousal, and Dominance (VAD) dimensions of emotion. Analysis of inter-perspective agreement reveals a systematic dimension-dependent pattern: Valence and Arousal are moderately to well preserved across perspectives, while Dominance exhibits markedly poor agreement and a consistent observer overestimation bias. Mixed-effects modeling of behavioral cues further demonstrates that the association between each behavioral signal and inter-perspective misalignment varies across affective dimensions: mouth dynamics reduced disagreement in Arousal but increased disagreement in Dominance and Valence, whereas upper-body motion was uniquely associated with disagreement in Dominance. These findings challenge the assumption that affective annotations from any single perspective constitute reliable ground truth, and suggest that Dominance is difficult to transmit through avatar mediation due to its dependence on relational context unavailable to external observers.

User behavior in virtual reality (VR) is influenced by the self-avatar's appearance through the Proteus effect, which can be modulated by social comparisons between one's own and others' avatars. However, it remains unclear how these effects are moderated in large-scale groups. Grounded in Social Identity Theory, this study investigated how appearance commonality between the self-avatar and surrounding avatars, combined with group size, influences user behavior. 32 participants performed a VR Taiko drumming task in a mixed design of the following factors: embodying either a festival attire (Happi) or a business suit avatar, surrounding avatars' appearances either matching the participant's attire or wearing diverse casual clothing, and group sizes of 3, 25, and 100 avatars. Results revealed that when surrounded by matching avatars, participants embodying Happi avatars exhibited significantly greater strike intensity, and participants embodying Suit avatars exhibited marginally significantly greater movement synchrony. Furthermore, larger group sizes decreased movement synchrony but increased subjective unity, while matching avatar appearance facilitated perceived group membership over diverse clothing. These findings offer design insights for social VR experiences: matching avatar appearance can enhance group identity and amplify task-relevant behavior when the appearance is contextually congruent with the activity, while increasing group size facilitates felt togetherness and decreasing it boosts behavioral coordination.

This study investigates how partner avatar design affects learning and memory when an avatar serves as a lecturer. Based on earlier research on the environmental context dependency of memory, we hypothesize that the use of diverse partner avatars results in a slower learning rate but better memory retention than that of a constant partner avatar. Accordingly, participants were tasked with memorizing Tagalog-Japanese word pairs. On the first day of the experiment, they repeatedly learned the pairs over six sessions from a partner avatar in an immersive virtual environment. One week later, on the second day of the experiment, they underwent a recall test in a real environment. We employed a between-participants design to compare the following conditions: the varied avatar condition, in which each repetition used a different avatar, and the constant avatar condition, in which the same avatar was used throughout the experiment. Results showed that participants in the varied avatar condition recalled significantly worse during the learning trials on the first day. However, we found no significant difference between conditions in the delayed recall test on the second day. We discuss these effects in relation to the social presence of the partner avatar. This study opens up a novel approach to optimizing the effectiveness of instructor avatars in immersive virtual environments.

Immersive social VR enables fine-grained manipulation of anxiety-inducing encounters---crowds, proxemic intrusions, and confrontational agents---yet how physiological, behavioral, and environmental signals co-vary with and could decode moment-to-moment anxiety in these settings remains largely unexplored. To address this, here we present a study of N=108 participants navigating five self-paced, multi-agent social VR scenes, under synchronized physiological (EDA, PPG, RSP, eye-tracking), locomotion (speed, head rotation), and proxemics (agent distance, count) recording, with continuous anxiety rated retrospectively. Four converging validity analyses validate this continuous retrospective self-report as a sensitive measure of state social anxiety, with consistent physiological co-variation confirming genuine affective responses. Importantly, we systematically evaluate three settings for anxiety prediction: within-person decoding, cross-participant generalization, and causal prospective prediction. Within individuals, deep sequence modeling (BiLSTM) achieves excellent prediction performance. In the cross-participant setting, however, performance drops substantially. In both settings, combining all modalities yields the best overall performance. In the most challenging prospective prediction setting, we chart how predictability varies across time provided for training and different classifiers. Our results establish a quantitative baseline for continuous anxiety decoding in ecological social VR and highlight cross-participant transfer as a key open challenge.

State-of-the-art head-mounted displays (HMDs) enable gaze-based selection in virtual environments. Yet, these HMDs suffer from the vergence–accommodation conflict (VAC), which is known to affect interaction performance. The VAC might influence gaze-based selection performance as it directly affects eye movement behavior. Thus, in this paper, we investigate how the VAC influences \gaze-based 3D target selection across varying depth conditions. Our results show that as the (visual) depth increases, user performance significantly decreases with gaze-based selection. Moreover, a previously-suggested ``\textit{Variation in Diopter}'' Fitts’ law model captured this performance change better relative to a linear model. These findings provide evidence that gaze-based pointing is negatively affected by the VAC and highlight the importance of accounting for depth-dependent factors when designing gaze-based interaction in 3D environments.

While dwell-based selection enables interaction with little physical effort, it is prone to (eye) gaze jitter. Target expansion methods mitigate this problem by increasing the effective selectable target area in 2D environments. Yet depth-related factors may limit their effectiveness in 3D virtual environments due to stereo-deficiencies. In this paper, we investigate how depth affects target expansion methods for dwell-based gaze selection, and propose Depth-based expansion, a method that adapts target expansion based on depth. We conducted two user studies to evaluate our proposed method with an ISO 9241:411 multidirectional selection study and a 3D selection study in a complex environment. Our results show that while the effectiveness of 2D target expansion methods decreases significantly with increasing depth, our proposed method consistently improved user performance and experience. Our findings provide design insights for effective dwell-based selection of expanding targets in 3D virtual environments.

The Vergence–Accommodation Conflict (VAC) is well-studied for 3D interaction performance in state-of-the art head-mounted displays. These interaction studies examined VAC in scenarios where the selected target remains at a fixed depth during acquisition. Yet, many VR applications involve targets moving continuously toward or away from the user, causing vergence demand to change over time while accommodation remains fixed at the display focal plane. We study this scenario using an ISO 9241-411 multidirectional target selection task with: noVac (targets at the display focal plane), Constant VAC (targets further from the focal plane), Varying VAC (targets appear at both NoVAC and ConstantVAC depths across selections), and Dynamic VAC (target depth changes continuously during acquisition) conditions. In a within-subject study with 24 participants, Dynamic VAC produced slower selections, higher error rates, lower accuracy, and reduced throughput compared to the noVAC condition. These findings suggest that continuously changing depth during target acquisition introduces additional performance costs beyond static VAC scenarios.

Eye tracking is a key enabler of natural and intuitive interaction. In virtual reality (VR), gaze depth estimation through vergence offers promise for precise interaction in real 3D space, not limited to the image plane. We propose a set of gaze-based techniques that utilize vergence in various ways for interaction tasks within visually cluttered environments. We conducted a series of experiments to assess accuracy, robustness, and user comfort. We report results from the evaluation of vergence reliability, selection pointing, and selection confirmation methods. Our findings show that, while gaze direction estimation is highly stable, depth precision decreases with distance, requiring adaptive error tolerance. These findings advance the design of gaze interaction systems toward seamless, controller-free experiences in immersive environments.

Semantic mapping is a core service that enables grounded interactions in emerging Extended Reality (XR) applications such as AI assistants and spatial object search. Deploying this capability on mobile XR devices requires a system that is open-vocabulary, real-time, and low-power. Existing approaches are compute-intensive and assume server-class resources. Cloud offloading offers a practical path, but no existing system implements distributed semantic mapping, and current approaches do not address how to manage communication, execution, and memory footprint across a device-cloud boundary. We present SemanticXR, the first device-cloud system for real-time, open-vocabulary semantic mapping and querying under XR power, bandwidth, and memory constraints. Our key insight is to elevate semantically identifiable objects to first-class units of distributed system design, governing how the system communicates, executes, and manages memory across the device and the server. On the server, object-level parallelism and geometry downsampling improve mapping latency, while object-level depth-mapping co-design reduces upstream bandwidth. On the device, an object-level sparse local map with incremental updates and update prioritization enables network-robust querying with bounded memory and downstream bandwidth. Object-level configurable resource usage vs. quality trade-offs allow applications to adapt the system to their needs. Evaluation against a distributed baseline using the same perception models shows that object-level system organization improves server-side mapping latency by 2.3×at equivalent semantic quality. Object-level depth-mapping co-design maintains upstream bandwidth under 2.5 Mbps. On the device, SemanticXR enables sub-100 ms query latency even under network drop while supporting thousands of objects within 500 MB, and scales downstream bandwidth with map changes rather than total scene size. The system adds only 2% device power during normal operation.

Particle-based physics simulations, including fluids, smoke, and granular media, are fundamental to visual realism in immersive VR and AR. With the growing adoption of social VR and digital twins, demand is increasing for shared experiences in which multiple users interact with the same simulation in real time. Realizing such experiences requires low-latency streaming of large-scale particle data from a server to each client, yet existing point cloud compression methods such as G-PCC (TMC13) and Draco assume static geometric structures; when applied to dynamic particle streaming, their encoding latency exceeds the frame period, failing to meet real-time delivery requirements. We propose DELUGE, a streaming compression architecture that exploits the temporal coherence and velocity predictability inherent in physics simulation particles, achieving sub-frame-latency encoding and decoding through three complementary techniques. Evaluation on dynamic point cloud datasets demonstrates that DELUGE achieves 19 times faster decoding than G-PCC (TMC13) and 6 times faster encoding than Draco, when all codecs are measured in-process via FFI. We further build end-to-end client implementations for both web browsers and Apple Vision Pro, and confirm through a within-participants perceptual quality evaluation and a two-person collaborative task study on Vision Pro that the proposed method supports real-time collaborative experiences with hand-tracked fluid interaction.

Testing autonomous robots in hazardous scenarios presents a fundamental dilemma: pure simulation cannot capture real-world physical dynamics such as surface friction and center-of-gravity effects, while physical test environments are dangerous, expensive, and slow to reconfigure. We present IMRSIVE, an inverse mixed reality framework that enables physical robots to navigate through virtual hazards while preserving real-world physics. The system provides dual synchronized perspectives: a first-person augmented reality view rendered on an AR headset for the autonomous agent, and an extended third-person spectator view composited from a live camera feed for external observers. To achieve correct visual compositing in the spectator view, we introduce a stencil buffer inverse occlusion pipeline that renders motion capture-tracked physical objects behind virtual geometry in real time. To our knowledge, this is the first framework to unify first-person AR and third-person composited perspectives for autonomous system evaluation. We evaluate the system through sustained rendering performance analysis, end-to-end latency measurement, occlusion quality assessment, and a parametric physical fidelity experiment in which an 11.43 cm center-of-gravity shift under identical control inputs produces trajectory deviations up to 79.6 cm, amplifying the input perturbation by a factor of seven. These results demonstrate that the framework captures physical phenomena that conventional simulation cannot predict, while maintaining real-time performance suitable for interactive mixed reality.

Cross Reality (CR) is a new emerging field based on the current developments in Mixed Reality hardware, especially supported by the broad market penetration of video-based see-through Head-Mounted Displays. It refers to applications that span across different stages (real, Augmented Reality, Augmented Virtuality, Virtual Reality) of the reality-virtuality continuum, where users are interconnected between different stages and/or are able to transition between these stages. This publication follows the concept of other grand challenges publications and reflects the discussion of various researchers invested in CR. After an initial discussion at the 1st Joint Workshop on Cross Reality at IEEE ISMAR 2023, six topic groups have been identified, leading to 22 challenges, which were discussed in groups over the period of multiple months. The discussion of these challenges should act as a road map for future research in the area of CR.

Light field AR glasses can provide better visual comfort than conventional AR glasses; however, studies on user performance comparison between them are notably scarce. In this article, we present a systematic method employing a serial visual search task without confounding factors to quantify and compare the user performance and experience between these two types of AR glasses at two different viewing distances, 30 cm and 60 cm, and in two modes, purely virtual VR mode and virtual-real integration AR mode. The results show that the light field AR glasses led to a significantly faster reaction speed and higher accuracy than the conventional AR glasses at 30 cm in the AR mode. The participant feedback also shows that the former led to better virtual-real integration. User performance and experience of the light field AR glasses remained consistent across different viewing distances. Although the conventional AR glasses had a better search efficiency than the light field AR glasses at 60 cm in both AR and VR modes, it had more negative feedback from the participants. Overall, the design of this experiment successfully allows us to quantify the effect of VAC and underscores the strength of the evaluation method.

Several augmented reality (AR) applications (e.g., AR-assisted surgery) require users to judge the depth of AR objects relative to real objects at near-field distances. Although near-field AR depth perception tasks demand high accuracy and precision, the perceived depth of AR objects is not veridical when compared to real objects. In addition, the vergence–accommodation conflict (VAC) can potentially lead to distorted visual perception, increased visual fatigue, eye strain, and other adverse effects when using AR devices. In this paper, we present the first systematic study examining how varying magnitudes and directions of VAC affect the accuracy and precision of near-field AR depth perception, using a custom-built AR Haploscope. Two focal planes were examined: 2D and 2.25D. Five levels of VAC magnitude and direction were tested: -1.0D, - 0.5D, 0D, +0.5D, and +1.0D. A perceptual depth-matching task was carried out with 24 participants. Participants subjectively rated various visual discomfort symptoms related to perceived depth. Our results showed that as the value of VAC magnitude increased, the accuracy and precision of near-field AR depth perception declined. In line with the theory of human perception under VAC, we observed that the perceived depth of the AR object was biased toward the focal plane, was underestimated when positioned behind the focal plane, and was overestimated when positioned in front of the focal plane. Our findings indicated that both the accuracy and precision of depth perception, as well as the degree of visual discomfort with near-field AR objects, were influenced by the focal plane position. Overall, our research will advance our understanding of near-field AR depth perception and expand the currently limited empirical evidence on how VAC influences near-field AR depth perception.

In Video-See-Through (VST) Head-Mounted Displays (HMDs) the cameras are fixed at a static distance which causes discrepancy with user's inter-pupillary distance (IPD). This discrepancy could differentially affect users with varying IPDs. To examine how this discrepancy affects size perception in near-field XR, a mixed-factorial study was designed, with Virtual Reality (VR), VST - Augment Reality (AR) and VST-Real viewing conditions manipulated between-subjects. Participants, with IPDs ranging from 58 mm to 70 mm, visually observed and provided size estimates of 9 different target discs of graspable sizes. Data from a large study involving 150 participants was gathered, providing us with 6750 data points for analysis of perceptual accuracy. The results revealed that compared to VR, in VST-AR, larger discrepancy involving wider IPDs resulted in more pronounced underestimation, especially for larger target objects. Also, when the interplay between IPD and the IPD-ICD discrepancy was considered, augmented targets were underestimated more than the physical targets. Over the course of trials, participants' size estimations tended to improve despite the variation in IPDs, suggesting perceptual calibration over time. These shed novel insights on the impact of IPD and IPD-ICD difference, and have the potential to guide the future research and design of hardware and software to facilitate improved size perception.

The rapid expansion of Augmented Reality (AR) technologies has increased the need to ensure visual comfort during extended, real-world use. In optical see-through AR systems (OST-AR), users perceive the real environment without mediation, whereas inaccuracies in the rendered virtual content can lead to perceptual conflicts with the real-world view. One such issue is Binocular Vertical Misalignment (BVM), which adds vertical disparity to augmented content while leaving the real-world view unaffected. Existing research on BVM relies on restricted stereoscopic displays and static stimuli, limiting ecological validity for real-world AR applications. In this study we investigate the effects of BVM directly on a see-through AR device, during prolonged (30-minute), interactive usage sessions. Our methodology incorporates individually calibrated BVM thresholds and evaluates discomfort across two distinct scenarios: a search and navigation task involving ego-motion and a stationary manipulation task. Visual discomfort was assessed using questionnaires and in-task Likert-scale ratings collected throughout the session. Experimental results indicate that perceived discomfort scales with a user’s individual tolerance threshold rather than the absolute magnitude of misalignment. Notably, the navigation task significantly exacerbated discomfort compared to the manipulation task, highlighting the critical role of ego-motion in BVM perception. These findings suggest that visual comfort in AR is a multi-faceted interaction between hardware-induced disparity, user tolerance, and specific task demands. This work provides foundational insights for designing more comfortable, long-term wearable AR experiences and informs calibration requirements for OST-AR displays.

Modern home appliances pose significant accessibility barriers for individuals with visual impairments, driven by increasing functional complexity and minimalist design trends. Drawing on formative interviews with blind and low vision (BLV) participants, we propose Poros, a novel system providing perception-enabled interactive assistance to facilitate seamless appliance interaction. Our approach features an automated instruction generator that models appliance operation logic directly from user manuals to synthesize executable guidance for arbitrary tasks. Integrated into an Augmented Reality (AR) wearable, Poros provides real-time auditory feedback synchronized with the user's hand gestures and the appliance's display state. A user study with quantitative analysis and qualitative insights shows that our approach enhances user independence, enabling the successful execution of complex, multi-step operations that were previously considered inaccessible.

Virtual Reality (VR) offers immersive experiences through embodied interactions like head tracking, going beyond traditional controls like joysticks or keyboards. Prior research has studied the impact of VR affordances on people’s experience and performance, but exclusively through sighted people's perspectives. While blind people may also leverage the physical nature of VR interactions, there is no knowledge on how VR compares with traditional controls in nonvisual tasks. We conducted a study where 26 blind participants performed navigation tasks with two conditions: VR and Game Controller. Additionally, as virtual environments can offer various degrees of complexity, we observed how participants performed under different cognitive demands. Our findings showed that VR improved navigation performance, whereas adding a simultaneous cognitive task decreased performance in both conditions. In VR, participants benefited from direct rotation, but with game controllers, the added complexity of rotating often led to reliance on translational 2D movements (e.g., moving sideways).

Robust navigational guidance is an important XR application for both sighted and non-sighted populations. In this paper, we mainly focus on blind pedestrians, who continue to face “last-mile” challenges such as locating entrances and navigating cluttered spaces. While smartglasses and wearables are maturing, a foundational design question remains underexplored: where on the body should cameras be placed to best support navigation? We present a mixed-methods investigation that focuses on the question of optimal sensor placement for generating spatial data supporting ego-centric navigation. A survey of 10 blind cane users surfaced practices for last-mile navigation and perceptions of body-mounted XR devices. A controlled case study with a blind co-author compared head- and cane-mounted cameras using synchronized Project Aria glasses while traversing five real-world environments. Using Simultaneous Localization and Mapping (SLAM) and Neural Radiance Fields (NeRFs) as diagnostic probes, we find that head-mounted cameras offer stability, cane-mounted cameras capture complementary ground-level detail, and fusing both yields increased robustness in scene reconstruction. We synthesize these findings into design guidance for hybrid XR systems that extend the cane without interfering with tactile and auditory cues.

Despite the rapid growth of Virtual Reality (VR), many applications remain inaccessible to individuals with disabilities. This study addresses this gap by identifying accessibility barriers and proposing inclusive design practices through the evaluation of three interaction modalities: controllers, hand tracking, and passive 360-degree video. Using a Meta Quest 3 headset, 37 participants, including 19 individuals with motor, visual, auditory, or neurological impairments, tested the First Steps with Hand Tracking application. A mixed-methods approach was employed, combining usability evaluation with the SUS, NASA-TLX, a custom accessibility questionnaire, and qualitative feedback. The results revealed significant accessibility barriers across the tested modalities. Findings show that hand tracking presents more barriers than controllers. Furthermore, qualitative data highlighted critical issues such as sensory overload, rigid interface heights, insufficient haptics for deaf users, and camera instability affecting wheelchair users' balance. The research demonstrates that even introductory VR tutorials contain accessibility limitations, concluding that comprehensive personalization of interfaces and input methods is essential for true inclusivity.

Projector-based adversarial attack aims to physically manipulate real-world scenes by projecting adversarial patterns, thereby causing deep image classifiers to produce incorrect predictions. However, existing stealthy projector-based adversarial attack methods model the project-and-capture process in a single and static view, limiting their view-invariant capability and practical applicability in dynamic environments. In this paper, we introduce View-Invariant Projector-Based Adversarial Attack (VIPA), a novel method designed to overcome these limitations by achieving both true view-invariant and classifier-agnostic adversarial attacks in the physical world. We first leverage a view-invariant projector-camera system simulation method to model the physical interactions between projected patterns and real-world surfaces. To ensure robustness and stealthiness of the attack across different views, VIPA optimizes adversarial projections by aggregating simulated attack losses from multiple views. This joint optimization across diverse views helps maintain robustness regardless of the viewpoints. Finally, the optimized patterns are physically projected into real-world scenes, where they successfully fool various classifiers from different viewpoints, thereby enabling robust and practical view-invariant adversarial attacks. Our experiments in both targeted and untargeted attacks demonstrate that VIPA consistently achieves higher attack success rates than existing methods, while also enhancing stealthiness and ensuring minimal perceptual degradation across all tested views.

Dynamic Projection Mapping (DPM) has revolutionized cosmetic consulting by transcending 2D screen overlays to be realized in the real world. However, existing DPM systems often rely on rigid interfaces and predefined textures, limiting creative expression. Therefore, we explore natural language as an intuitive interaction modality to maintain user focus on the projected results. Nevertheless, applying free language input remains challenging, particularly in achieving precise localized makeup generation and appropriately rendering the generated makeup on the face through projection. To address these limitations, we present Text2Makeup-DPM, the first fully text-driven DPM makeup system transforming free-form descriptions into facial makeup projections. Our system features a two-scale makeup generative pipeline and efficient radiometric compensation. First, we conduct text parsing to align free-format text input into the global style and local demand of makeup. The global style module captures the overall makeup style and atmosphere through contrastive learning. A local component module optimizes latent direction compositions to enable precise, disentangled editing of specific regions (e.g., eyeshadow, lipstick). Subsequently, an asynchronous Hadamard-division compensation ensures high-fidelity projection without compromising temporal interactivity. Quantitative evaluations demonstrate that our two-scale pipeline aligns more accurately with natural language descriptions than baseline models. Furthermore, a user study (N=12) confirms the system's effectiveness in enhancing the immersion of interaction flow, proving its contribution to intuitive DPM authoring.

Dynamic projection mapping (DPM) enables spatially aligned projection onto moving objects, but requires low-latency rendering and accurate appearance reproduction. Meanwhile, 3D Gaussian Splatting (3DGS) has recently attracted considerable attention as an image-based approach for reconstructing 3D scenes solely from images and rendering them from arbitrary viewpoints. This paper proposes a framework for realizing DPM using 3DGS. Compared with conventional DPM approaches based on physically based rendering (PBR), the proposed 3DGS-based framework enables simpler, high-fidelity reproduction of real object appearance by eliminating the need for complex prior measurement of material parameters. However, the low-latency rendering required for DPM necessitates the use of fast 3DGS variants, which introduce rendering noise that leads to visible projection artifacts. To address this issue, we introduce blue-noise sampling into stochastic transparency for 3DGS to reduce perceptually noticeable noise. We further reduce noise by using high-frame-rate projection that leverages persistence of vision and adapting perceptual-quality-preserving efficient rendering techniques from the PBR literature to 3DGS. Furthermore, assuming a rigid target object, we exploit the equivalence between viewpoint motion in 3DGS and object motion in DPM to obtain viewpoints consistent with the pose of the projection target, enabling the direct use of existing 3DGS renderers. Experiments on synthetic scenes and real-world projections demonstrate that the proposed method maintains rendering speeds comparable to conventional fast variants while achieving LPIPS reductions of 57.7 % and 45.2 %, respectively, effectively suppressing visual artifacts.

Everyday-wearable Augmented Reality (AR) glasses must meet strict power limits, making displays a key target for optimization. We cast display power optimization as a power-constrained tone-mapping problem and propose a human-vision--grounded, learning-based framework that maximizes perceptual quality under a given power budget. We introduce an optimization-friendly tone-mapping operator (TMO) parameterization along with a progressive optimization strategy to effectively navigate the quality-vs-power landscape. We distill the iterative optimization into a lightweight feed-forward neural network for real-time deployment. Subjective experiments show that our method yields better perceptual quality than prior work at the same power budget.

Most software-based cybersickness mitigation techniques, such as field-of-view (FOV) restrictors and rest frames, alter visual content and peripheral optical flow, often negatively affecting various locomotion quality factors. Path integration, a critical component of spatial updating, relies on optical flow during virtual locomotion and may therefore be particularly affected by these techniques. Despite a substantial body of work evaluating cybersickness mitigation methods, relatively few studies have examined their effects on spatial updating or path integration. Moreover, prior studies on spatial updating have not yielded conclusive results, despite theoretical predictions from the perception literature. In this work, we conducted a preregistered experiment with 48 participants to compare the effects of two mitigation techniques, the dynamic FOV restrictor and peripheral teleportation, on path integration, relative to an unrestricted control condition. By minimizing potential confounding factors identified in previous work, we found that participants made significantly larger homing errors with the FOV restrictor than in the control condition, whereas performance with peripheral teleportation did not differ significantly from either of the other conditions. These findings provide direct evidence that FOV restriction impairs path integration, while the impact of peripheral teleportation remains inconclusive. Our results further highlight the trade-off between mitigation effectiveness and spatial updating, providing VR designers with deeper context for selecting mitigation techniques.

As spatial computing offers expanded design spaces and brings digital information seamlessly into a physical environment, it redefines how we perceive space and navigate our surroundings. This study investigates enhancing human navigation and spatial knowledge acquisition in large space environments, focusing on the integration of spatial visual cue display in 3D design space. Three cues were designed (world-fixed cue, body-fixed world-in-miniature map, and a combination of both) and their impact on spatial learning, eye movements, cognitive load, and user experience was assessed. The findings reveal that displaying cues across frames of reference (combining the world- and body-fixed cues) significantly enhances spatial information processing and learning without the cost of cognitive load. The study also sheds light on how individuals' eye movement patterns differ with cue design and while learning, which advances our understanding of the relationship between gaze behavior and spatial learning outcomes and informs cognition-aware adaptive 3D interface design.

Augmented reality (AR) often presents navigational cues directly within the user’s field of view. While centrally placed cues provide explicit direction, they may compete with task-relevant perception and increase visual clutter during multitasking. Peripheral cues are often considered a less visually intrusive alternative by leveraging the human visual system’s sensitivity to motion and orientation, but their effectiveness for guidance remains insufficiently understood. This paper investigates how cue placement within the visual field (foveal 3D Arrow vs. peripheral segmented halo) influences navigation performance, attention management, and user experience in AR environments involving divided attention and visually complex surroundings. In a controlled study (N=22), participants navigated between targets while performing concurrent reading tasks. Results show that foveal cues enable significantly faster navigation and spatial localization across the majority of target destinations. In contrast, peripheral cues led to a significant 2.12-second delay in secondary task response times, indicating increased cognitive processing demands during multitasking. Interestingly, despite this performance cost, peripheral cues were perceived by some participants as visually comfortable, and user preference was divided (55% foveal cues vs. 45% peripheral cues). These findings highlight a critical trade-off between visual presentation and cognitive efficiency: while peripheral cues were at times experienced as less visually intrusive, their abstract representation requires additional interpretation effort. This challenges the assumption that relocating guidance cues to the periphery inherently improves attentional resource allocation in AR systems.

Navigation aids are central to immersive virtual reality (VR) experiences that involve physical locomotion. Their effectiveness depends not only on how much spatial information they provide, but also on how directly that information supports movement decisions. We compared three common guidance techniques for immersive VR wayfinding: a directional arrow, a minimap, and a compass. In a controlled room-scale VR study with 42 participants completing 1008 trials, participants navigated to target landmarks in a time-pressured maze with reduced visibility and forced route replanning. Across behavioral and eye-tracking measures, arrow guidance produced the strongest navigation performance, minimap guidance yielded intermediate performance, and compass cues performed worst, suggesting that during immersive locomotion users benefit from guidance that can be interpreted rapidly while moving. These results suggest that in demanding immersive locomotion tasks, interfaces that translate spatial information directly into actionable movement cues can outperform richer but more interpretive spatial representations. Our findings highlight the importance of designing XR navigation interfaces that minimize the cognitive translation between spatial information and movement decisions.

Throwing is a natural and embodied interaction for selecting distant targets in virtual reality (VR). Unlike conventional pointing, throwing is ballistic: once released, the trajectory can no longer be corrected. Despite its practical importance, throwing-based target selection has received relatively little attention in human behavior and performance modeling, making systematic design and evaluation still difficult. In this paper, we investigate throwing-based target selection in VR and propose predictive models for its behavior and performance. Through a controlled user study manipulating target width ($W$) and movement amplitude ($A$), we measured planning time ($PT$) and endpoint distribution. Our results showed that $PT$ increased systematically with task difficulty, following a pattern similar to movement time in Fitts' law. Throwing endpoints were also generally well approximated by a bivariate Gaussian distribution, and target parameters significantly affected both endpoint bias and variability. Based on these findings, we derived a probabilistic accuracy model that predicts target acquisition performance. The proposed models showed high goodness-of-fit, and the accuracy model was robust under leave-one-condition-out cross-validation. Our findings provide a quantitative understanding of throwing-based target selection and support the design of throwing interactions in immersive environments.

Portal-based interaction enables users to access and manipulate distant content without physical navigation. However, effective remote manipulation remains challenging due to mismatches between object and interaction scale, reducing control precision and increasing coordination effort. In this study, we investigate how different scale-management strategies in portal-based interaction affect task performance and user experience in virtual reality (VR). We designed four portal techniques that vary in how interaction scale is controlled during manipulation: fixed-scale, manually scalable, overview+detail, and transfer-based configurations. We conducted a within-subject VR experiment (N = 31) using a remote 6DoF docking task with objects of varying sizes. Results show that scale-management strategies significantly influence manipulation performance, particularly for smaller objects. Multi-portal configurations improved efficiency by providing task-appropriate interaction scales, whereas single-portal approaches offered simpler workflows but required additional user effort. These findings highlight the importance of structuring interaction scale across task phases and provide design implications for portal-based manipulation in VR.

As a common action in our daily lives, rubbing has been a viable technique for various devices, including touchscreens, digital pens, and skin surfaces. However, there has been little research on the design of rubbing in virtual reality (VR), particularly with VR controllers. This study explores the design space of rubbing with two-handed VR controllers. We started with a one-on-one interview with 10 participants to understand four components of rubbing with VR controllers: hand use, rubbing direction, device type and rubbing contact. In Experiment 2, we proposed a method to classify rubbing with three hand uses and six rubbing directions. Results showed that the overall classification accuracy reached 96%. Experiment 3 revealed that our method was able to successfully reject all seven tested non-rubbing actions. Results from Experiment 4 reflected user preference for using our technique to perform five typical VR tasks. Our study contributes to VR interaction design with controllers.

A major challenge in radiological diagnoses is the ability to correctly and quickly identify abnormalities from Computed Tomography (CT) images. Although some research exists on the volumetric rendering and visualization of CT scans for training, interaction with and testing of 3D visualizations for diagnosis and educational training in radiological imaging is still limited. To address this gap, we present a 2D + 3D interactive system that allows users to locate potential fractures simultaneously on 2D and 3D CT visualizations and to annotate them using a controller-based encapsulation technique. Using algorithms we built to automatically filter 2D CT images and display a bounding box on the 2D images from the 3D model's equivalent location, participants can simultaneously view both 2D and 3D anomalies to learn to recognize fractures. In addition to this technical framework, we conducted an experiment (N=55) to compare 2D, 2D3D, and 3D viewing conditions to evaluate the effects of visualization methods on abnormality detection using VR-based annotation. Our findings show that interactive methods comprised of 2D images in a 3D space lead to a higher detection sensitivity and reaction time when compared to 3D only. These results suggest that embedding 2D diagnostic imagery within an interactive 3D spatial context may improve perceptual training in novices, offering a promising direction for using virtual reality to improve radiological training in fracture identification.

With the rapid development of virtual reality technology, 360◦ panoramic video has become increasingly widespread. To ensure high visual quality across the wide field of view, the resolution of such videos continues to increase, which brings significant challenges for video storage, processing, and especially trans- mission. Previous research has proposed foveated image transformation techniques to reduce the resolution in non-foveal regions, thereby lowering transmission bandwidth. However, these approaches operate on ERP-projected panoramic video frames without accounting for the severe geometric distortions introduced by ERP projection, particularly in high-latitude regions. This paper proposes a spherical Gaussian-guided multi-resolution sampling (SGMRS) method. Specifically, we use a normalized spherical Gaussian distribution to generate a Fibonacci-based sampling point set that conforms to the spherical visual acuity model, and perform foveated multi-resolution sampling directly in the spherical coordinate domain. We then precompute the spherical sampling map for a canonical gaze direction and exploit the rotational symmetry of the spherical Gaussian distribution to achieve fast recovery of 360◦ foveated video frames. This enables real-time foveated 360◦ video streaming. We conduct quantitative evaluations on existing 360◦ video datasets and compare our method against state-of-the- art techniques. The results demonstrate that our method substantially improves visual consistency across different gaze directions. Moreover, under comparable transmission bandwidth, our method significantly improves both the perceptual quality of the foveated video and its temporal stability.

Foveated sparse volume rendering is a key direction for enabling real-time immersive volume visualization on resource-constrained standalone headsets. However, existing methods typically ignore the human visual system’s sensitivity to salient content in peripheral vision and often suffer from severe temporal flickering artifacts under sparse sampling. In this paper, we present Saliency-aware Foveated Sparse Volume Rendering, a lightweight pipeline that enables real-time immersive raycasting on standalone headsets. The key idea is to jointly model gaze-dependent acuity falloff and content-driven visual saliency, and to use the resulting screen-space saliency map to guide sparse raycasting. We further introduce an artifact-aware reconstruction strategy that leverages two inexpensive, shading-simplified auxiliary renderings as guidance, effectively mitigating flickering artifacts induced by sparse sampling. We evaluate our method on multiple biomedical volumes; both objective metrics and a user study demonstrate improved perceptual quality over representative foveated sparse volume rendering methods under real-time constraints.

360-degree video telepresence offers strong immersive potential but remains constrained by the limited resolution of current capture and display hardware. Many telepresence installations feature fixed viewpoints and largely static scenes, yet optimization strategies tailored to such setups have received limited attention. We present a multi-layer, ultra-high-resolution system for static 360-degree telepresence that combines an 8K panoramic camera with a rotatable 4K pan-tilt-zoom (PTZ) camera. Our approach builds a three-layer representation: (1) a tile-based ultra-high-resolution panoramic background, generated by offline stitching high-detail 4K PTZ scans onto the base 8K panorama to achieve effective resolution beyond native capture, and represented as a set of spatial tiles; (2) a dynamic update layer that composites foreground motions from the 8K stream via real-time high-resolution background matting; and (3) a region-of-interest 4K layer that streams a real-time PTZ view of the selected region and additionally updates the corresponding background tiles over time. We evaluate the proposed system through comparisons with representative video super-resolution approaches and a user study assessing perceived detail and immersive experience. Our results indicate that tile-based background refinement, together with user-guided updates, provides a practical way to balance panoramic fidelity and interactivity in static 360-degree telepresence.

Panoramic video has emerged as a core medium for virtual reality (VR) and immersive media, yet its audio typically remains stereo, lacking spatially coherent soundfield reproduction consistent with the visual content and severely compromising immersion. Although multi-loudspeaker spatial audio systems can reconstruct realistic soundfields in physical spaces, existing solutions remain heavily reliant on manual production, and mainstream algorithms such as Vector Base Amplitude Panning (VBAP) handle only directional information while inadequately modeling distance perception. This paper proposes the Vision–Acoustic Coupled Spatialization (VACS) framework, which automatically extracts three-dimensional (3D) motion trajectories of sounding objects from panoramic videos containing only stereo audio tracks and generates spatially consistent multichannel audio end-to-end. The core innovation is the Depth-Aware Spatial Audio Rendering (DASAR) method, which leverages monocular depth estimation to integrate sound pressure level (SPL) attenuation, spectral adjustment, and reverberation control within a unified framework, overcoming the angle-only limitation of conventional VBAP. Experiments show 62.6% localization accuracy within 10° tolerance, with Pearson correlations of 0.877 and 0.843 (p < 0.001) for azimuth and elevation. Subjectively, compared with VBAP, spatial depth and immersion improve by 13.2% (p < 0.05) and 19.5% (p < 0.01), respectively. Comparison with professional manual production shows comparable quality across all dimensions (p > 0.05) while substantially lowering the production barrier.

Cybersickness remains a critical barrier to the widespread adoption of virtual reality (VR), making accurate prediction of its onset an important research priority for enabling comfortable VR use. However, cybersickness varies across individuals, and models trained on population-level data often fail to generalize to new users. Most existing approaches also focus on detecting or classifying cybersickness after symptoms appear rather than forecasting how discomfort will evolve, which limits timely mitigation. To address these challenges, we propose an asymmetric few-shot meta-learning approach that uses asymmetric temporal encoding to forecast cybersickness severity 30 and 60 seconds ahead using multimodal data, including heart rate, eye tracking, and head tracking, with only 30 seconds of user calibration. We validate the approach on two public datasets: a real walking VR task (Maze, 36 participants) and seated VR tasks (Simulation21, 30 participants). The model is evaluated for both in-domain performance and cross-domain generalization. Compared to state-of-the-art methods, it achieves a 53.7% reduction in RMSE for in-domain prediction on the Maze dataset and a 49.1% reduction under cross-domain transfer at the 60-second forecast horizon. Memory and computational analysis show an inference latency of 2.5 milliseconds and a memory footprint below 1.0 MB, indicating suitability for standalone VR deployment. Overall, the results provide a practical foundation for personalized cybersickness forecasting to support proactive mitigation and improve user comfort in VR environments.

Predicting the onset of cybersickness enables proactive mitigation strategies that minimize user discomfort, which is essential for the widespread adoption of virtual reality (VR) in education, healthcare, training, and entertainment. Current prediction models leverage multimodal data such as visual scene parameters and physiological signals, but often overlook individual differences that strongly influence cybersickness susceptibility, limiting generalization across diverse users. To address this limitation, we propose a personalized cybersickness prediction framework based on Bayesian Networks (BN) that incorporates individual factors, including age, gender, and prior VR experience. We introduce three personalization strategies: direct integration of user attributes, a two-stage pipeline that learns a latent susceptibility score, and stratified models trained on homogeneous user groups. We validate the approach on two publicly available datasets, Maze and Simulation21. On the Maze dataset, the method achieves a 14.27% relative improvement over the non-personalized Bayesian baseline and outperforms DeepTCN and LSTM by 44.59% and 48.07%, respectively. It also improves accuracy by 37.83% over the transfer learning strategy. Similar trends are observed on Simulation21, where the framework achieves a 13.40% improvement over the non-personalized baseline, outperforms DeepTCN and LSTM by 9.16% and 10.82%, and improves accuracy by 3.89% over transfer learning. Consistent gains are also observed in F1 score, Cohen’s kappa, and Spearman correlation. The method maintains low inference latency, supporting interpretable and efficient probabilistic predictions in VR settings.

Cybersickness remains a persistent barrier to the widespread adoption of virtual reality (VR), often undermining user comfort and immersion. Existing cybersickness mitigation techniques are mostly static and may fail to adapt to diverse user needs and dynamic VR contexts. For instance, aperture size in the dynamic field of view (DFOV) and blur intensity in the dynamic Gaussian blur (DGB) may need to vary across users and applications based on real-time cybersickness severity. To address this, we introduce AdaptVR, a reinforcement learning (RL)–based adaptive cybersickness mitigation framework that predicts, explains, and mitigates cybersickness. Unlike static methods, our RL agent optimizes long-term user comfort by adaptively selecting and adjusting mitigation strategies through continuous interaction with VR environments. We train the RL agent using Proximal Policy Optimization (PPO) in domain-randomized simulations and deploy the learned policy on a consumer-grade VR headset (HTC Vive Pro). At runtime, AdaptVR continuously processes real-time user signals (e.g., eye-tracking, head motion, scene context) to detect cybersickness severity and appropriate mitigation (e.g., DFOV and DGB) with exact intensity. We also design a large language model (LLM)–powered interactive dialogue engine that enables users to engage in natural, voice-based conversations to understand detection outcomes, explore mitigation choices, and provide feedback. This feedback is incorporated into a human-in-the-loop training process, allowing the RL agent to fine-tune its policy for personalized cybersickness detection and mitigation. We evaluate AdaptVR through a user study in a VR maze simulation. Results show that our RL-driven adaptive mitigation significantly reduces cybersickness with minimal impact on immersion. Moreover, 90% of participants reported that the framework is intuitive, effective, and improved their understanding of the mitigation process through the dialogue engine. The open-source implementation of the AdaptVR framework can be found at https://anonymous.4open.science/r/AdaptVR-D676

Cybersickness remains a major barrier to the widespread adoption of virtual reality (VR) systems. Although many prior studies have attempted to predict cybersickness using objective sensor data, most approaches rely on subjective self-reported measures such as the Fast Motion Scale (FMS) or the Simulator Sickness Questionnaire (SSQ) as ground truth labels. The highly subjective nature of these measures introduces variability across participants, making it difficult to reliably relate objective signals to cybersickness levels and limiting the generalizability of prediction models. In this work, we propose an alternative approach that extracts cybersickness-relevant information directly from objective eye-tracking data by detecting anomalies in gaze behavior. Raw eye-tracking signals are first converted into scanpath graphs that represent fixation and saccade patterns, enabling the capture of both structural and temporal characteristics of visual exploration. These graphs are then used to train several anomaly detection models, including Isolation Forest, One-Class SVM, Dense Autoencoder, LSTM Autoencoder, and Spatio-Temporal Graph Autoencoder. The models learn patterns of normal gaze behavior and identify deviations that may indicate progression of cybersickness. We evaluate the proposed framework on three VR datasets: VRWalking, Terrain, and VRNet, comprising 5,357 scanpath graphs derived from $\sim$443 sessions. Using segments with minimal sickness (FMS ≤ 1) as the baseline behavior, anomaly scores are generated to simulate the temporal progression of cybersickness within each session. Across all datasets and participants, our approach achieves an RMSE of 1.188 and an MAE of 0.927 when approximating FMS trajectories. These results suggest that gaze-based anomaly detection can provide an objective signal for cybersickness assessment and enable the development of alternative scoring frameworks that reduce reliance on subjective self-reported measures.

We present a systematic review of spatial user interfaces (SUIs) in mixed reality (MR) environments. SUIs organize interface content and interaction through 3D spatial relationships. However, despite the rapid proliferation of prototypes and system implementations, the existing contributions are largely confined to narrowly scoped application contexts, isolated interface components, or discrete interaction techniques, with limited cross-dimensional synthesis. As a result, we still lack a systematic understanding of how core design dimensions interact and co-evolve within MR environments. This absence of an integrative perspective impedes the identification of recurring design patterns and constrains the field’s capacity to surface emerging innovation opportunities. To address this gap, we conducted a systematic review of 108 publications across four key dimensions: Display Representation, Spatial Organization, Generation and Update Strategy, and Interaction Method. We further analyzed intra- and inter-dimensional relationships, synthesizing recurring cross-dimensional patterns and distilling their broader design implications. Building on our findings, we propose a four-part research agenda to advance the development of next-generation SUIs for real-world contexts and complex task environments. Our review and supplementary materials are available through an interactive online literature platform (https://spatialuserinterface.github.io/) to support continuous updates and community engagement.

Mixed Reality interfaces let users anchor virtual windows to specific, meaningful locations in the physical space. When the association between a window and a location is strong, accessing the window often entails users physically going to that location rather than bringing the virtual window to them. This task, which we call Spatial Window Switching (SWS), consists of two steps: 1) identifying the window of interest from an overview; and 2) reaching that window in physical space. We compare four design variations of SWS techniques, based on two overviews: a 3D world-in-miniature that preserves spatial relationships between windows but is prone to occlusion; and a 2D tiled overview inspired by desktop interfaces that is occlusion-free but breaks spatial relationships between windows. Both overviews are evaluated with and without animated visual guidance. We report on a comparative study across low and high window-density conditions, quantifying the performance improvements of SWS techniques compared to a baseline condition without assistance. Results indicate that 2D overviews perform best in high-density scenarios while 3D overviews better support spatial awareness and one-handed interaction.

Anchoring - the choice of frame of reference for mixed reality (MR) interface elements - is a critical design decision involving trade-offs between accessibility, interaction comfort, and visual interference. Despite its importance, user preferences for anchoring across different mobility contexts and interface properties remain poorly understood, as prior work has largely focused on specific tasks or fixed interface configurations. We address this through a mixed-methods user study in which participants configure anchoring strategies across different mobility conditions and interface types. Combining behavioral analysis with structured qualitative inquiry, we analyze how participants select and reason about anchoring modes. Our results show a clear transition from world-anchored interfaces in stationary contexts to body-anchored interfaces during locomotion. However, no single body anchor consistently dominates, highlighting the personal nature of anchoring strategies. Our qualitative analysis reveals the factors users consider in their anchoring decision, including interface accessibility, stability during interaction, visual clutter, and individual mental models. These findings inform the design of adaptive and controllable MR interfaces and highlight the importance of supporting user customization.

We investigate hybrid user interfaces (HUIs), aiming to establish a cohesive understanding and to adopt consistent terminology for this nascent research area. HUIs combine heterogeneous devices in complementary roles, leveraging the distinct benefits of each. Our work focuses on cross-device interaction between 2D devices and mixed reality environments, which are particularly compelling, leveraging the familiarity of traditional 2D platforms while providing spatial awareness and immersion. Although prior work has prominently explored such HUIs in the context of mixed reality, we still lack a cohesive understanding of the unique design possibilities and challenges of such combinations, resulting in a fragmented research landscape. We conducted a systematic survey and present a taxonomy of HUIs that combine conventional display technology and mixed reality environments. Based on this, we discuss past and current challenges, the evolution of definitions, and prospective opportunities to tie together the past 30 years of research with our vision of future HUIs.

Virtual reality (VR) systems continuously capture multimodal sensor streams, including head motion, gaze direction, and physiological signals, to enable interaction and immersive experiences. However, this multimodal telemetry carries persistent biometric signatures that adversaries can exploit to re-identify users and infer sensitive demographic attributes such as age and gender, even from anonymized datasets. Existing privacy-preserving approaches, including noise injection and anonymization, degrade the temporal structure of the data, which is essential for downstream utility in VR. To address this gap, we present a Progressive Privacy Preserving method designed specifically for multimodal VR telemetry while maintaining downstream utility. Rather than optimizing against all demographic attributes jointly, the method separates physiological and kinematic feature domains and applies adversarial optimization sequentially. This progressive strategy prevents gradient interference between demographic traits that originate from distinct physiological signal sources, enabling targeted suppression of sensitive attributes while preserving behavioral patterns required for application utility. We evaluate the method on three publicly available VR datasets: Mazed and Confused, Simulation21, and WhoIsAlyx. Results show that demographic inference accuracy drops to near random chance, approximately 50% for binary attributes, while downstream utility task performance, including cybersickness severity classification, cognitive load classification, physical load classification, and action intent classification, remains strong, with reductions of at most 14.5% compared to unprotected data. On average, the method modifies approximately 11% of the original signal magnitude, indicating that the temporal structure remains largely intact. These findings demonstrate that sensitive demographic attributes in VR data can be effectively suppressed through progressive adversarial optimization while preserving signal fidelity for downstream utility.

As Augmented Reality (AR) devices become embedded in everyday environments, their outward-facing cameras introduce continuous privacy risks by capturing sensitive information. While intelligent privacy assistants can mitigate these risks, little is known about how varying levels of autonomy reshape user responsibility and engagement in embodied AR contexts. We developed an AR privacy assistant with three autonomy roles and evaluated them in a within-subjects study (N=30) where participants performed a timed search task in a virtual home while managing privacy concerns. We collected objective performance metrics, subjective workload and trust ratings, and qualitative interview data. The Autonomous assistant significantly improved privacy concern detection accuracy without increasing subjective workload or degrading trust. Rather than eliminating human effort, autonomy redistributed cognitive responsibility, from environmental scanning to supervisory verification, prompting participants to adopt an “auditor” role when system judgments conflicted with their privacy expectations. While autonomy heightened situated perceptual awareness, it did not produce measurable gains in global privacy literacy, suggesting AR privacy assistants must balance transparency and alignment to support human agency.

Virtual reality (VR) is increasingly used in security-sensitive domains, but user authentication in immersive environments remains underexplored. Current methods (e.g., passwords) are impractical and more susceptible to observation-based attacks. We present a challenge-response authentication pipeline for VR based on consumer-grade electroencephalography (EEG). Our pipeline combines a behavioral liveness module with an EEG authentication module built on an interactive VR task. Using a four-channel EEG headband, our dataset achieved an Equal Error Rate (EER) of 3.37% in single-session evaluation and 11.48% in cross-run evaluation, and remained competitive with several public datasets under a shallow machine learning benchmark. It also delivered the strongest channel-normalized cost-performance among all datasets considered, offering roughly 8× greater efficiency than 30–32-channel benchmarks. Usability evaluation results further indicate that the combined VR-EEG setup was generally well tolerated in short sessions. These findings support the feasibility of lightweight, deployable EEG-based authentication for VR applications.

The convergence of artificial intelligence (AI) and extended reality (XR) technologies (AI-XR) holds great promise for innovative applications in many domains. However, XR data (e.g., eye-tracking) is highly sensitive, and trained AI models can retain user-specific information that attackers can exploit. Hence, privacy regulations such as the European Union’s General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) grant individuals the \emph{right to be forgotten}, requiring the removal of personal data not only from databases but also from AI models trained on them when requested by a user. This makes post-hoc deletion of XR user data a critical requirement for deployed AI-XR systems. Existing state-of-the-art privacy mechanisms in AI-XR applications (e.g., differential privacy (DP)) primarily provide pre-training or sample-level protection and do not explicitly remove the influence of revoked XR users from already-trained AI models; satisfying deletion by retraining from scratch is often computationally prohibitive in real-world, large-scale AI-XR pipelines. Motivated by this, we propose a novel machine-unlearning-based framework for AI-XR that enables post-training revocation compliance by efficiently removing the influence of revoked XR users from trained AI models. Specifically, we design and evaluate two practical unlearning strategies: SISA (sharded, isolated, sliced, and aggregated training) and knowledge distillation (KD) to update AI-XR classifiers by removing revoked users’ learned contribution from the trained model parameters, producing an updated model that is equivalent to one trained model without those users, while preserving high utility on retained users and minimizing update cost compared to retraining the model from scratch. We evaluate the effectiveness of our proposed method on three SOTA AI-XR models and three different XR datasets (i.e., cybersickness, emotion, and activity classification). Our experimental results show that the proposed method preserves both privacy and utility (i.e., low accuracy on the forgotten-user set and high accuracy on the retained-user set) while substantially reducing model update cost compared to retraining from scratch. For instance, using a Transformer model, SISA reduces forgotten-set accuracy to 18\%, 15\%, and 19\% while maintaining retained-set accuracy of 95\%, 97\%, and 93\% for cybersickness, activity, and emotion classification, respectively, and achieves up to an $\approx 12\times$ speedup in update time over retraining from scratch. Privacy auditing via membership inference attack further shows that our proposed method reduces the attack success rate by up to $38\%$, $36\%$, and $40\%$ for the same classification tasks compared to the no-unlearning baseline. Finally, we validate these findings by constructing a new XR cybersickness dataset collected from 40 participants in a user study, which shows similar trends in privacy and utility, demonstrating that the proposed approach generalizes beyond benchmark datasets to a realistic AI-XR setting.

Cybersickness, a phenomenon of negative symptoms arising from visual motion, remains an obstacle to prolonged use of immersive displays. While cybersickness in virtual reality (VR) has been extensively studied, its mechanisms and mitigation remain underexplored in optical see-through augmented reality (AR). Unlike VR, AR preserves a direct view of the physical world while overlaying virtual content. This stable visual context fundamentally alters the visual–vestibular conflicts that contribute to cybersickness. In this paper, we investigate the causes of cybersickness in optical see-through AR and examine whether mitigation techniques developed for VR transfer to this setting. Specifically, we analyze how attention to virtual and physical content influences cybersickness under visual–vestibular conflict and whether reducing the virtual field of view (FOV) lowers sickness when users do attend to the virtual layer. Our results show that optical see-through AR can induce severe cybersickness when users attend to dynamic virtual content despite the presence of a grounding physical environment. In contrast, sickness levels are attenuated when attention is allocated towards the physical environment. We further find that reducing the global motion cues by a reduced FOV does not reliably mitigate AR cybersickness, suggesting that established cybersickness mitigation strategies from VR may not directly transfer to AR. Together, these findings demonstrate that AR cybersickness is an attention-dependent phenomenon with significant implications for AR interface design.

Architectural design review in Virtual Reality (VR) requires clients to move in the environment to understand building layouts. However, locomotion methods present a trade-off: continuous movement supports spatial awareness but causes cybersickness, while teleportation prevents cybersickness but does not support spatial orientation as much. We investigated whether applying sketch-style non-photorealistic rendering (NPR) during continuous locomotion in guided architectural tours reduces cybersickness while maintaining the information necessary for spatial comprehension of the design. In a within-subjects study (N = 36), participants completed guided tours in three virtual museums on three separate days using full color continuous locomotion (FCL), NPR continuous locomotion, or jump teleportation (JMP). Spatial awareness was measured via different tasks and tests including: point-to-landmark, egocentric distance estimation, map drawing, and object recognition tasks. Pointing accuracy in NPR was comparable to FCL (11.6◦ vs 10.6◦, p = 0.143, and Bayesian equivalence testing BF01 > 1000) confirming statistical equivalence. NPR significantly outperformed JMP (15.1◦, p < 0.001) and reduced post- exposure cybersickness compared to FCL(∆SSQ: FCL M = 28.6 vs NPR M = 21.6, p = 0.018). Real-time Fast Motion Sickness (FMS) scores showed that JMP produced the lowest discomfort (M = 0.88), NPR had an intermediate level (M = 1.19), and FCL showed the highest discomfort (M = 1.48). These findings demonstrate that deploying sketch-style rendering during guided transitions enables comfortable architectural explorations without losing the spatial comprehension for design evaluation.

Cybersickness, a form of motion sickness experienced in virtual reality, remains a significant challenge that limits prolonged user engagement. Researchers have explored various mitigation techniques to reduce user discomfort in VR; however, although several public datasets on cybersickness exist, datasets that include data collected under mitigation techniques are rarely publicly available. To address this gap, we present a user study with 38 participants in a VR roller coaster simulation where they experienced three visual conditions: no blur (NB), static maximum blur (MB), and dynamic blur (DB), where the blur intensity adapts in real time based on users’ self-reported sickness levels. We present a novel multimodal dataset that includes eye tracking, head tracking, and physiological signals such as electrocardiogram and galvanic skin response collected under conditions with and without cybersickness reduction techniques. In addition, the dataset contains subjective measures including fast motion sickness ratings reported every 30 seconds, simulator sickness questionnaire scores, and igroup presence questionnaire responses for evaluating users’ sense of presence in the virtual environment in different visual conditions. In addition to data recorded during the experience, the dataset contains baseline physiological recordings collected before the VR simulation. As an initial use case, we conducted a case study of training several deep regression models to predict cybersickness in which we achieved 1.58 RMSE. This open dataset allows future researchers to explore the relationship between physiological responses, user behavior, and cybersickness in VR. It can also support the development of predictive models and adaptive systems that select the most appropriate cybersickness reduction technique for each user, leading to more comfortable and personalized VR experiences.

Redirected Walking (RDW) enables users to explore large virtual environments within confined physical spaces by leveraging perceptual tolerances in self-motion perception. Previous research has shown that adding vestibular noise through electrical or continuous vibratory stimulation can increase redirection detection thresholds (DTs). However, electrical stimulation may compromise gait stability, while continuous vibration often induces discomfort. Consequently, alternative approaches that balance effectiveness and user comfort are needed. To address this issue, this study investigates whether temporally synchronizing vestibular vibration with gait events can enhance redirection tolerance while preserving comfort and locomotor stability. Thirty-six participants performed a curvature-based RDW task under three conditions: no vibration, continuous vibration, and step-synchronized vibration. Detection thresholds were estimated using a two-alternative forced-choice paradigm combined with psychometric modeling. In addition, subjective discomfort, simulator sickness, and presence were assessed alongside objective gait stability measures obtained through markerless motion analysis. Overall, both vibration conditions significantly increased curvature DTs compared to the no-vibration baseline. Notably, step-synchronized vibration produced curvature DTs comparable to continuous vibration while significantly reducing discomfort. Furthermore, markerless gait analysis revealed no significant differences in trunk sway or vertical body displacement across conditions. Taken together, these findings indicate that gait-synchronized vestibular vibration can effectively enhance redirection tolerance while maintaining user comfort and without introducing observable gait instability during walking-based RDW.

Precise tool manipulation with magnification loupes is fundamental in dentistry, yet the design of virtual magnification in immersive environments remains underexplored. This paper presents two Virtual Magnification Loupes (VMLs) co-designed with two senior dentists over six months: a Dynamic Virtual Loupe (DVL) providing head-coupled stereoscopic magnification and a Static Virtual Loupe (SVL) providing world-fixed magnification anchored near the intra-oral cavity. Both were integrated in a realistic immersive dental training environment featuring a physical dental treatment room with a phantom-based patient replica, a drill tool, and foot pedal-task confirmation to preserve the clinical workflow. We conducted a within-subjects study (N = 25) in which dental students performed high-precision drill positioning tasks under three conditions: DVL, SVL, and a No Loupe (NL) baseline. We measured positional error, rotational error, task completion time, cognitive load (NASA-TLX), and user preference. Results showed no significant differences in completion time or spatial error across conditions. However both VMLs significantly reduced perceived cognitive workload compared to NL. DVL yielded the lowest temporal demand and highest perceived performance, while SVL produced the lowest physical demand. In preference rankings; DVL was preferred by 15 dental students, SVL by nine, and NL by only one. These results indicate that VMLs can achieve equivalent task precision while significantly reducing cognitive load. The complementary strengths of both VML designs suggest that future immersive systems should support switchable loupe modes to accommodate varying task demands.

Ultrasound-guided placement of peripheral venous catheters (PVCs) is a routine medical procedure that involves inserting a needle in a patient’s vein with the visual guidance of an ultrasound scanner to monitor the progression of the needle towards the vein. This procedure is cognitively demanding, as it requires operators to constantly divide their attention between the ultrasound display and the spatial relationship among the probe, the needle, and the vein. As a result of this difficulty, the procedure is associated with a high complication rate for patients. The goal of this paper is to evaluate the potential of augmented reality (AR) to facilitate the procedure for clinicians by displaying ultrasound (US) images in situ. To do so, we conducted a user study that compares two visualization modes: a conventional monitor-based procedure replicating the standard ultrasound display, and an AR-assisted procedure where the ultrasound image is spatially anchored directly under the probe and augmented with real-time distance cues (needle-to-skin and needle-to-ultrasound plane). The study was conducted within a virtual reality (VR) environment to provide a standardized and controlled setting. Our user study involved 30 participants from the healthcare domain, including 9 experts and 21 novices. Each participant performed ten insertions in both conditions. We analyzed procedure time, successful insertion attempts, and needle-beam alignment accuracy. We also assessed perceived workload (NASA-TLX) and system usability (SUS) using standardized questionnaires. Experts completed the procedure in 524.2 ± 154 s with AR, compared to 1125.7±472 s using the traditional monitor-based layout, whereas novices completed the procedure in 746 ± 410 s, compared to 1165 ± 597 s in the traditional setup. Spatial accuracy in novices increased from 50.6 ± 8.2% to 65.9 ± 13.6%, corresponding to a reduction in mean targeting error from 1.26 mm to 0.79 mm. The AR condition also greatly reduced tissue perforations in novices, decreasing from 29.4 ± 45.6 to 3.0 ± 2.7. Participants reported a lower perceived workload with AR, which may indicate that placing the ultrasound image directly under the probe helps reduce attention shifts and supports spatial understanding of the gesture. These findings indicate that AR-based in situ visualization can meaningfully enhance the efficiency, accuracy, and safety of ultrasound-guided procedures.

Percutaneous needle procedures are minimally invasive interventions that involve inserting a needle through the skin to collect tissue samples, deliver medication, or treat specific conditions. Despite the apparent simplicity of the motor action, these tasks are challenging because clinicians cannot directly visualize anatomical targets and must instead rely on ultrasound images, which increase cognitive demand. Augmented reality (AR) has emerged as a valuable resource to provide assistance during the performance of these tasks, allowing for visualization of pertinent information in the clinician’s field of view. However, the use of naïve representations of augmented content frequently ignores meaningful visual cues, complicating depth perception and spatial understanding. In this work, we use and combine a set of visualization techniques designed to provide relevant information during the performance of percutaneous procedures: a localized focus-and-context window that exposes internal structures, a color-based proximity encoding that provides information regarding the distance between a tool and the organs, and an explicit trajectory overlay that provides directional information. These techniques, developed during design sessions with medical experts, were evaluated in a user study involving 26 participants, 7 of them clinical experts, using a prostate-biopsy-inspired phantom. Across objective and subjective measures, cue effects depended on both the surrounding cue configuration and user expertise. For novices, trajectory guidance improved targeting accuracy and reduced retreat behavior, but its effect on completion time depended on cue context and introduced a speed–precision trade-off when presented alone. For experts, the focus-and-context window reduced completion time and retreat events, while color feedback also improved completion time. Subjectively, proximity color cues were often perceived as helpful even when their effects on accuracy were not consistently positive. Overall, these results show that AR guidance cues are not uniformly additive and should be designed with attention to both user expertise and visual context in needle-based interventions.

The discovery and evolution of medical imaging technologies have enabled non-invasive visualization of internal anatomy that has become essential for supporting diagnosis, monitoring, and treatment. However, because medical imaging relies on complex physical processes and contrast mechanisms for image formation, imaging alone is not sufficient to enable humans to fully leverage the resulting information. In addition, traditional methods to visualize the resulting information use two-dimensional displays to present three-dimensional anatomical structures. The introduction of Augmented and Mixed Reality (AR/MR) technologies offers an opportunity to provide valuable paradigms for medical imaging visualization, allowing users to observe, explore, and interact with anatomical information in more spatially intuitive ways. However, naive implementation without careful design considerations can lead to perceptual inconsistencies, potentially compromising utility and effectiveness. In this paper, we present a structured taxonomy of medical AR/MR visualization strategies aimed at providing clearer insight into how visualization design varies across clinical use cases. The taxonomy organizes techniques based on four core design components: image modality, data dimensionality, display technology, and clinical application. In addition, we introduce two critical dimensions that are often overlooked in the literature: visualization anchoring --the spatial relationship between virtual content and the physical world -- and perceptual awareness -- the use of visual cues to support spatial interpretation. Together, these components form a comprehensive taxonomy, offering a detailed framework for selecting appropriate visualization techniques in medical applications.

Video see-through (VST) technology aims to seamlessly blend the virtual and physical worlds by reconstructing reality through cameras. However, while manufacturers promise high perceptual fidelity, it remains unclear how accurately these systems replicate natural human vision, particularly across varying environmental conditions. In this work, we introduce a low-cost testing method to quantify the perceptual gap between the naked eye and three popular VST headsets: Apple Vision Pro, Meta Quest 3, and Meta Quest Pro. Using adapted psychophysical measures, we evaluated participants' visual acuity, contrast sensitivity, and color vision under both normal and low-light conditions. Our results demonstrate that despite recent hardware advancements, all tested VST systems fail to match the dynamic range and adaptability of natural human vision. While high-end devices can approach human performance in ideal lighting, they exhibit significant visual degradation in low-light environments, particularly concerning contrast sensitivity and visual acuity. By mapping these specific perceptual limitations, this work establishes a definitive benchmark for current VST capabilities and highlights the need for experience design compensations to achieve truly indistinguishable mixed reality experiences.

Broad public adoption of video passthrough head-mounted displays remains elusive despite significant market investment, and as a community we lack a precise understanding of why users experience persistent discomfort even as hardware factors such as resolution and latency have dramatically improved. This paper establishes the first mechanistic link between passthrough usage and discomfort, specifically regarding the impact on visual behavior and physiology. We developed a novel protocol to capture synchronized oculomotor, kinematic, and physiological data during a block assembly task requiring complex hand-eye coordination. Using a within-subject design (N=110), we evaluated both natural and passthrough viewing conditions. Our results reveal a four-fold suppression of rotational head velocity (33.4°/s vs. 133.3°/s, p < 0.05) and a fundamental decoupling of head-gaze coordination – which we coin “Passthrough Rigidity”. This phenomenon appears to shift the information-gathering burden to the oculomotor system, resulting in significantly longer fixation durations (304 ms vs. 279 ms, p < 0.05) and restricted visual search patterns. These kinematic shifts directly correlate with poorer task performance (natural accuracy 85.5%, passthrough 82.1%) and measurable physiological cost, evidenced by a significant reduction in blink duration (229ms vs. 261ms, p < 0.05) and increased reports of ocular strain and cognitive load. We conclude that current passthrough implementations force a transition from flexible exploration to motor caution, where task performance is preserved at the cost of user comfort and biomechanical efficiency. These findings provide a critical quantitative framework for evaluating and improving future XR devices, establishing that resolving "comfort" for passthrough requires addressing the deep-seated biomechanical compensations caused by mediated perception. All data is made publicly available at https://removed.for.double.blindness.

For virtual objects to appear spatially anchored in Augmented Reality (AR), users must be able to tell whether they rest on a real surface. Humans judge this primarily through cast shadows and contact edges (the visible boundaries where a virtual object meets the physical surface). However, additive blending in Optical See-Through (OST) displays weakens cast shadows, leaving contact edges as the main available cue. How rendering parameters and environmental factors jointly shape this edge-based judgment remains poorly understood. We investigated this on an OST display with segmented dimming (Magic Leap 2) by manipulating virtual object opacity (20\%--100\%), background spatial frequency (0.5, 1.5, 4.5 cpd), and ambient lighting (500 lx, 1500 lx). A within-subjects study (N = 39) assessed contact judgment accuracy, signal detection sensitivity (d′), confidence ratings, and eye-tracking behaviour. Results show that 60\%–80\% opacity yields the highest accuracy and efficiency, as this range keeps the virtual object visible without occluding the surface texture needed to judge contact. This advantage was most pronounced over mid-frequency backgrounds (1.5 cpd). Notably, users feel most confident at 100\% opacity despite lower objective performance, revealing a disconnect between perceived and actual judgment quality. Eye tracking reveals two distinct failure modes: at 100\% opacity, relatively minimal gaze effort at the contact edge suggests overconfidence-driven underinspection; at 20\% opacity, prolonged fixations indicate that increased effort cannot compensate for insufficient surface texture. Together, these findings show that the most visually solid rendering is not the most perceptually effective, pointing toward contact-state-driven opacity adaptation as a design strategy for OST-AR.

Static registration error can impair user performance in augmented reality, but its behavioural and subjective effects remain insufficiently characterised. While existing works often evaluate acceptable registration quality as a purely quantitative geometric problem, this perspective overlooks how spatial misalignment creates perceptual ambiguity for the user. In this paper, we demonstrate that AR registration requirements might not be defined by absolute offset magnitude alone, but also by the physical layout and the boundaries of cue-target ambiguity. We report a controlled video see-through augmented reality study with 24 participants using a reproducible 3 × 3 button-acquisition task, in which a visual cue was displaced relative to physical targets across six offset magnitudes. User performance remained close to the zero-offset reference at the smallest offsets, but degraded significantly under larger misalignments. Participants also showed short-term adaptation after the switch to misalignment, although larger offsets retained substantial residual costs. Subjective ratings showed parallel increases in rating with performance. These findings provide a basis for setting task-specific registration requirements by identifying offset ranges that remain behaviourally tolerable, rather than those that produce ambiguity-driven breakdown under explicit performance criteria.

Teleportation is widely used in immersive virtual reality, as it enables efficient travel and reduces cybersickness compared to continuous steering. However, most teleportation techniques are designed for horizontal ground-based movement and offer limited support for navigation to arbitrary 3D positions. In this paper, we comparatively evaluate three teleportation techniques for mid-air destinations: a previously proposed Two-Step technique in which horizontal position and elevation are specified sequentially, a ray-based approach that extends conventional pointing teleportation with controllable placement of safe mid-air destinations, and a novel technique in which users place teleport targets on a rotating plane dynamically coupled to head pitch. A within-subject study with 25 participants compared the three techniques in a target-based flying task and a large-scale wayfinding task. The results show that the ray-based technique achieved the best overall performance across most measured dimensions, leading to lower workload, higher usability, faster task completion, and fewer teleport actions than the other techniques. At the same time, the head-coupled plane technique was more frequently associated with precision and spatial orientation, revealing a trade-off between efficiency and spatial control. These findings demonstrate the potential of 3D teleportation for immersive VR and contribute to ongoing research on alternative approaches for arbitrary-height locomotion.

We present EMSWalker, a system that provides proprioceptive feedback during seated walking-in-place by delivering gait-synchronized electrical muscle stimulation (EMS) to the leg muscles involved in walking (quadriceps, tibialis anterior, and gastrocnemius). By mapping foot tapping to the human gait cycle, EMSWalker renders virtual walking sensations for level, uphill, and downhill locomotion while users remain seated. Through three user studies, we evaluate (1) cognitive remapping from seated walking-in-place to walking, (2) discrimination of terrain-specific EMS patterns, and (3) EMSWalker’s effect on virtual walking experience. Results show that EMSWalker significantly increased proprioceptive drift and embodiment relative to other conditions, enabled above-chance discrimination of slope-specific stimulation patterns, and improved presence in a VR walking experience with flat and sloped terrain. These findings demonstrate that gait-synchronized, terrain-adaptive EMS can enrich seated VR locomotion by providing more coherent proprioceptive feedback and a more realistic, spatially immersive walking experience.

To enrich interaction in constrained mobile VR systems, this paper introduces a low-cost hardware module that uses a simple prism to enable vision-based Walking-in-Place (WIP) on standard smartphones. We evaluated this controller-free system in a user study against Look-Down-To-Move (LDTM). Methodologically, we propose and apply Fréchet distance as a more robust metric for analyzing continuous and trajectory-based locomotion, moving beyond traditional accumulative metrics like total distance traveled. The empirical findings both align with and extend prior research. We replicate the key result that WIP is significantly more immersive than LDTM without a cost to objective performance. To further investigate subjective experience, we utilized a 7-point Likert scale questionnaire. The results were consistent with findings from categorical preference questionnaires but captured more nuance, revealing a critical trade-off: while confirming WIP's superior immersion, participants found both techniques reliable, although LDTM was rated as more reliable, highlighting a tension between embodiment and system efficiency.

Traditional VR viewport control relies on head and torso movement, which can be effortful and limiting in both constrained and extended-use settings. We introduce Looking Around by Looking Around (LALA), a gaze-based VR pitch-and-yaw viewport control technique designed for natural and effortless omnidirectional exploration via eye movements, without requiring or obstructing movement of the head, hand, or body, offering a low-effort and highly accessible interaction method. Because gaze is primarily used for perception and exhibits oculomotor and perceptual asymmetries, using it directly for control is difficult. To address this, we designed an asymmetric omnidirectional control profile for the eye, then built on it to exploit tendencies for eyes to stay within comfortable regions for viewport control. We evaluated LALA in a user study (N=18) featuring two contrasting tasks: alignment towards known directions and open-ended visual search towards unknown directions. LALA was strongly preferred over the traditional baseline, achieving competitive performance while enabling fully hands-free interaction with minimal physical movement

Maintaining balance is important for safe interaction in immersive virtual reality (VR), where visual–vestibular conflicts can affect postural stability. Most prior works on VR-based balance analysis rely primarily on motion signals from the head-mounted display (HMD), which capture head movement but provide only limited information about full-body biomechanics. As a result, directly predicting biomechanical balance indicators, such as the Center of Pressure (CoP), from VR sensing data remains challenging. In this paper, we present EgoBalanceFormer, a multimodal framework for predicting human balance during VR interaction by leveraging egocentric pose reconstruction as a body-aware sensing modality together with VR-native HMD motion signals. Unlike prior VR balance approaches that rely primarily on sparse device motion such as HMD trajectories, our approach uses egocentric vision to reconstruct full-body pose and derive biomechanical cues—including lower-body kinematics and center-of-mass (CoM) dynamics that are critical for modeling balance control. These pose-derived biomechanical cues are fused with VR-native HMD motion signals and processed through temporal encoders to capture motion dynamics over time. We further apply bidirectional cross-attention between lower-body joint kinematics, CoM dynamics, and rotational features to capture complementary biomechanical relationships. The refined multimodal representations are fused and decoded using a transformer with a learnable CoP query to regress CoPX and CoPY. We evaluate EgoBalanceFormer against several machine learning and deep learning baselines for CoP prediction. Experimental results show improved accuracy and stronger robustness across participants, demonstrating that egocentric pose-derived biomechanical cues significantly enhance balance prediction compared with using VR-device motion signals alone.

Boredom is a recognized but empirically underexplored factor affecting performance and safety in professional environments. In industrial workplaces, its consequences pose significant risks to health and well-being. Despite growing interest in using Virtual Reality (VR) for studying occupational phenomena, the field lacks standardized, ecologically valid procedures for reliably inducing and analyzing boredom in controlled experimental settings. This paper presents the exploratory evaluation of BoredomVRoom, a VR-based boredom-induction task inspired by the Test of Variables of Attention (TOVA), adapted as a repetitive quality control operation within a neutral virtual environment. In a user study (N=32), where the task was administered as a baseline condition within a broader, multi-condition VR protocol, a 15-minute task led to a significant increase in self-reported boredom, as measured by the Multidimensional State Boredom Scale (MSBS). This induction was accompanied by a significant decrease in affective valence and arousal, consistent with the theoretical affective signature of boredom. The raw NASA-TLX revealed low cognitive demand, high self-rated performance, and moderate frustration. This pattern aligns with the characterization of boredom as a state arising from cognitive underload. Furthermore, participants significantly overestimated task duration, a known temporal marker of the construct. These converging results suggest the proposed task's potential as a boredom-induction procedure for VR-based human-computer interaction research in industrial contexts, while acknowledging that its embedded administration necessitates future isolated normative validation. The application and Unity source code will be released as an open-source package to serve as a foundational tool for boredom induction in XR research.

Ensuring a safe virtual reality (VR) experience requires effective systems that can predict and respond when users lose their balance. While prior research has focused on predicting falls and motion sickness in VR, most approaches have been regression-based and have not fully explored postural state classification. This study presents an exploratory comparative analysis of machine learning (ML) and deep learning (DL) algorithms for classifying postural states in VR environments affected by visual perturbations. We used a publicly available multimodal dataset that contains kinematic, electromyographic (EMG), and electrodermal activity (EDA) signals. The data were prepared for a binary classification task that distinguishes balanced and imbalanced postural states. Participant-wise downsampling was applied to address class imbalance in the dataset. The ML and DL models that we compared, were then evaluated under a Leave-One-Participant-Out (LOPO) cross-validation protocol. Among the models, the Mamba-inspired CNN (MI-CNN) achieved the highest accuracy of 96.76%. The most important factors influencing the postural state classification were identified using SHapley Additive exPlanations (SHAP) analysis, which enhanced the interpretability of the model. The results from SHAP analysis indicate that kinematic data features are the most dominant in classifying postural states. We also evaluated the MI-CNN model using only the top two-third of features ranked by SHAP importance. Despite a 66% reduction in input dimensionality, the model retained strong classification performance (0.957 ± 0.022 accuracy and 0.957 ± 0.0218 F1-score), with only a marginal decrease of approximately 1% compared to the full-feature model. This study is an initial step toward using multimodal sensing and advanced learning models to better understand balance-related instability in immersive VR environments. The ability to accurately classify an imbalanced postural state may support early awareness of potential fall risks. Also, the findings contribute to the design of safer and more adaptive VR systems. Such systems may respond to early signs of instability and help improve safety and user experience in immersive applications.

Reduced walking speed in people with multiple sclerosis (MS) is closely associated with a higher risk of falls. But VR rehabilitation studies often overlook whether improvements in performance come with acceptable cognitive and physical effort. This study introduces a threshold-anchored efficiency framework that evaluates both performance and overall effort simultaneously. Here, a normative velocity target (T=1.29 m/s) was established from the mean walking speed of non-fallers in an independent MS gait dataset, representing the speed at which MS patients walk when falls are not occurring. 34 adults with MS were then tested across eight VR feedback conditions (unimodal, bimodal and multimodal). Each condition was evaluated on how much it closed the velocity gap to T and at what cognitive and physical burden cost. Pareto efficiency analysis identified five non-dominated conditions : Static Visual, Spatial Auditory, Auditory+Visual, Auditory+Vibrotactile, and Multimodal -- while Spatial Vibrotactile and Vibrotactile+Visual were dominated. Spatial Auditory offered the best efficiency, closing 72.7% of the gap at below-average burden. Multimodal closed the most gap (95.6%) but imposed the highest burden. These findings give clinicians a principled basis for selecting VR feedback strategies matched to each patient's capacity.

Small-scale objects that are difficult to see or distinguish from their surroundings are common in many assembly and maintenance procedures. Augmented reality (AR) instructions involving such small parts may be difficult to perceive, potentially causing misinterpretations. This paper explores the use of AR visualizations for small-scale instructions. In particular, we present the design and evaluation of AR magnification visualizations. Through iterative design, we explore how magnified AR views impact user behavior, visual attention, and subjective perception in the context of AR instructions. Our results demonstrate the effectiveness of live AR magnification in improving task guidance, providing a strong foundation for future work on AR-assisted instruction and precision interaction design.

The ability to see through the surface of the human body is central to medical augmented reality (AR) applications ranging from surgical planning to intraoperative guidance. However, accurately perceiving the depth of virtual structures relative to real anatomy remains a persistent challenge. This is particularly pronounced under the operating microscope, where high magnification, limited viewpoints, and tight error tolerances make depth and occlusion both essential and difficult to infer. Prior work has proposed methods for visualizing subsurface content using Video See-Through (VST) and Optical See-Through (OST) head-mounted displays (HMDs). However, the implementation of these approaches in surgical microscopes remains an open question due to the additive characteristics of microscope optics and display systems. To the best of our knowledge, this work is the first to investigate the perceptual accuracy of AR microscope visualization techniques for depicting virtual content inside the human body. We introduce a custom depth perception measurement apparatus based on a half-silvered mirror that enables alignment between a real measurement probe and subsurface virtual targets while viewing through a stereoscopic surgical microscope. We then propose a custom alpha-blending method and a virtual aperture that conforms to the three-dimensional surface geometry. Using our apparatus, we conducted a user study evaluating depth estimation accuracy and confidence across visualization techniques. Results showed that both the virtual aperture and alpha-blending significantly reduced absolute depth error compared to standard opaque rendering. Alpha-blending without an aperture achieved localization accuracy comparable to opaque rendering with a virtual hole while eliminating the systematic depth underestimation bias observed in traditional renderings. Furthermore, the presence of a virtual aperture increased users' confidence and was preferred over other methods. These findings provide insights for the design of mixed-reality visualization systems for microsurgical applications.

Inattentional blindness (IB) is a perceptual-attention phenomenon in which visual information enters the eye but fails to reach conscious awareness, thereby impairing situational awareness and leading to missed detection of low-probability yet high-risk events. In this paper, we present GAAC, a gaze-aligned augmented reality (AR) cueing framework that integrates scene perception and eye tracking to estimate IB occurrence and provide warning cues with AR display. GAAC combines an eyewear-based eye tracker with a forward-facing camera to localize anomalies, align gaze evidence with target ROIs, and enable attention-contingent AR cueing with latency suitable for interactive on-device use. To evaluate the framework, we design a controllable IB-induction paradigm in an assisted-driving task that enables repeatable measurement across groups. The user study shows that both non-contingent and attention-contingent cueing reduced IB relative to a no-cue baseline, while the attention-contingent condition achieved a lower IB rate. Subjective workload increased under non-contingent cueing but remained at baseline levels under attention-contingent cueing. These findings suggest that attention-contingent AR cueing is a promising approach for mitigating IB while limiting unnecessary user burden in attention-demanding settings.

Central vision loss (CVL) from macular diseases such as age-related macular degeneration and Stargardt disease is commonly simulated in extended reality (XR) using a single mechanism, typically a dark patch or blur overlay. Yet clinical reports suggest that people with CVL reject this depiction and instead report a broad range of perceptual effects, including spatial distortion, perceptual invisibility, dynamic noise, dimming, filling-in, and temporal disturbances. We present XR-VIEU, a modular, real-time, gaze-contingent framework for simulating CVL in XR through seven clinically grounded, independent, composable shader modules implemented in Unity HDRP, with formal mathematical specifications and over 120 configurable parameters per eye. To probe whether our simulation can capture lived CVL experiences, we conducted a co-participatory design study with two individuals with late-stage Stargardt disease. Despite sharing the same diagnosis, the two co-designers required significantly different module configurations: one profile was dominated by content displacement and dimming, while the other relied primarily on dense fractal noise. These cases provide initial evidence that modular, composable simulation is needed to represent the heterogeneity of CVL experience. The full seven-module pipeline adds only 0.20 ms of rendering overhead (2.9%) while maintaining approximately 90 Hz on the Varjo XR-4 at native resolution. XR-VIEU provides a foundation for more faithful and personalized CVL simulation in research, clinical education, and accessibility-oriented design.

Optical see-through (OST) augmented reality (AR) displays can render virtual content directly in the user's view of the physical world. In certain use cases, the physical world can include fixed emissive displays. For example, pilots, ship captains, and power plant operators equipped with AR displays might switch their visual attention between AR elements, physical interface components, and imagery on emissive flat panel displays fixed in the environment. Switching attention between these targets involves aspects of dynamic perception and cognition. We conducted a controlled study using XREAL One Pro AR glasses to examine the cognitive effects of switching attention between content displayed on a nearby flat panel display (FPD)/ screen and content displayed on a head-worn AR display (HWD), under two ambient brightness levels (10 lx and 150 lx). Participants completed an adapted Paced Visual Serial Addition Test (PVSAT), a continuous cognitive task that measures information processing speed and working memory under sustained demand, while switching between displays on every trial. Contrary to expectation, we found no general cost of switching over repeating a display. Instead, performance costs were specific to the direction of the transition: transitioning to the screen was significantly slower than transitioning to the HWD, no matter where the transition began. Higher ambient brightness improved response speed, reduced missed trials, and did not increase subjective visual fatigue. These findings suggest that in unpredictable multi-display environments, what display you transition to matters as much as the act of switching itself, with implications for the design of OST AR interfaces where transitions between head-worn and external displays are frequent.

Teleportation techniques for Virtual Reality (VR) locomotion that use a World-in-Miniature (WIM) for target selection typically rely on instant relocation. Although efficient, this abrupt transition can impair spatial orientation and reduce users’ sense of presence. To address this issue, this paper introduces transitional spaces as VR navigation concept. Users enter a transitional space containing a scaled-down walkable WIM, move (walking/steering) to the in- tended destination, and confirm their target position. They then exit the transitional space through a second portal, creating a percepti- ble and continuous transition rather than a direct relocation. In an exploratory user study, we compared classic WIM-based telepor- tation, portal-Augmented WIM, and the proposed walkable WIM. The results show a trade-off between efficiency and orientation: the walkable WIM achieved the best spatial orientation performance, but also has the longest completion times and a higher cognitive load due to additional navigation overhead. At the same time, both portal-based approaches provided a more positive user experience than the classic WIM, including a stronger sense of presence and higher hedonic quality. In addition, the Walkable WIM showed the significantly shortest reorientation time.

Conventional VR guardian systems communicate physical boundaries through grid-based overlays that effectively signal spatial limits, but disrupt narrative coherence and presence. This tension between safety and immersion reflects a trade-off that existing approaches have not fully resolved. To address this, we propose Contextual Blocking Entities (CBEs), a boundary design approach that transforms guardian boundaries into narrative-integrated virtual objects by embedding spatial constraints within the internal logic of the virtual world. We define CBE design through three dimensions: Function Match, which signals impassability through visual form, Context Consistency, which determines how naturally the object belongs to the world, and Constraint Type, which determines how the boundary geometry engages spatial processing. To operationalize these dimensions, we first conducted a formative study classifying 80 candidate virtual objects, then carried out a controlled within-subject user study using a 2x2x2 factorial design to compare CBE conditions against a grid baseline across presence, spatial awareness, and boundary compliance. All CBE conditions produced significantly higher presence than the grid baseline, while four conditions showed comparable safety to the baseline. The three dimensions operated asymmetrically, each governing a distinct stage of boundary interaction. Function Match governed initial approach deterrence, Context Consistency supported corrective behavior following a violation, and Constraint Type shaped movement range independently of conscious spatial judgment. These findings show that the safety-presence trade-off is not inherent to boundary systems, but depends on how spatial constraints are expressed within the virtual world, and provide guidelines for virtual object selection and placement in room-scale VR.

Extended reality (XR) is increasingly used to help users understand complex virtual environments through multiple viewpoints across different levels of immersion, positions, and scales. While numerous techniques address viewpoint transitions for self-guided exploration, many scenarios require externally-guided transitions where a system or presenter controls the user's viewpoint, leaving the user with limited spatial knowledge and control over the transition process which can increase susceptibility to disorientation and discomfort. We present three transition techniques for externally-guided multi-scale XR viewpoint changes and evaluate them against a fade-to-black baseline in a within-subjects study (N=20). Participants transitioned between world-in-miniature, street-level, and indoor destination views. We combined spatial recall measures, standardized questionnaires, and semi-structured interviews to assess orientation, workload, comfort, and continuity. Results showed that techniques externalizing reference frames and the user's pose improve multi-scale spatial recall relative to a fade, while same-scale recall was insensitive to transition type. We conclude with design implications for multi-scale XR viewpoint transitions.

Combat sports such as boxing require rapid tactical decision-making and precise spatio-temporal awareness, yet current analysis primarily relies on video replays and statistical charts. Fixed viewpoints and occlusion constrain video replays, while statistical data are displayed separately, requiring users to correlate information across sources mentally. This separation constrains effective reflection on athlete movements and tactical interactions. Although prior sports analytics research has explored event detection and action recognition, few systems support motion reconstruction and integrated spatial analysis using real match data in combat sports. To address this gap, we conducted a formative study with professional boxing coaches and developed In Sight of Insights, a 3D interactive tactical analysis system that reconstructs athlete motion and integrates match events within a unified virtual environment. By embedding contextual information directly within reconstructed scenes, the system enables spatially coherent observation and interaction. We evaluated our system through two user studies with professional athletes and experienced spectators. Results show that Free-viewpoint Virtual Representation improves observation accuracy and analytical efficiency compared with conventional viewing modes. Participants also reported that free-viewpoint perspectives and integrated contextual visualization enhanced tactical understanding. These findings demonstrate the potential of reconstructed 3D environments with embedded analytics to support more effective combat sports analysis.

Urban environmental sensing networks generate dense spatial data describing atmospheric conditions across city neighborhoods, yet this information remains largely inaccessible to residents within the environments it describes. We present a WebXR-based handheld augmented reality system that enables walk-up public access to city-scale environmental visualization through the mobile browser without application installation. The system integrates minute-level data from approximately 280 active weather stations across a 138-square-mile metropolitan area and visualizes atmospheric conditions as geospatially anchored particle fields within a browser-delivered urban model. Users access the experience by scanning QR codes, launching the AR interface in under ten seconds. A 48-hour deployment audit characterizes end-to-end pipeline latency, spatial binding accuracy, and rendering performance across heterogeneous mobile devices. A controlled in-situ within-subjects user study ($N = 16$) comparing the AR interface against a browser-based map presenting identical sensor data demonstrates that situated AR visualization significantly improves self-reported spatial comprehension of neighborhood-scale environmental conditions ($p = 0.005$, $d = 0.81$). The comprehension advantage is explained by reduced spatial translation effort: map users reported significantly greater cognitive work to relate the display to their physical surroundings ($p = 0.031$, $d = 0.89$). Preliminary behavioral comprehension probes with an independent sample provide initial convergent support for the self-report findings. These results position standards-based WebXR as a viable civic interface layer for distributed urban sensing infrastructures.

Collaborative Immersive Analytics (CIA) has emerged as a promising approach for supporting collective sense-making of data using immersive technologies. Over the past decade, a growing number of systems have been proposed, yet the design space and collaborative practices in CIA remain fragmented across the literature. This paper presents a systematic review of CIA research published between 2015 and 2025. Following PRISMA guidelines, we analysed 19 empirical studies across six dimensions: time--space configuration, analytical tasks, system-centred symmetry, user-centred symmetry, collaboration dynamics, and data characteristics and domain applications. Our results show that CIA systems predominantly supported synchronous collaboration and shared exploratory analysis coordinated through natural communication mechanisms such as verbal communication and deictic gestures. However, many systems were evaluated using datasets selected primarily to demonstrate system capabilities rather than real-world analytical applications. We conclude by outlining future directions for CIA research.

Immersive technologies such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) are being increasingly used in experimental psychology because they enable controlled environments while simultaneously producing high-frequency motion-tracking data from headsets, controllers, and body trackers. These motion traces provide continuous behavioral signals that may reflect affective, behavioral, and cognitive processes during human–human, human–computer, and human–virtual human interactions. Despite the growing use of motion measures in XR (Extended Reality) experiments, there is limited synthesis of how such data are collected, transformed, analyzed, and interpreted. This paper presents a systematic literature review of studies collected from the ACM Digital Library, IEEE Xplore, PsycINFO, PubMed, and Web of Science that use motion-tracking data from XR devices in experimental psychology, focusing on affect, behavior, cognition, or their intersections. The review examines the devices used, the body parts tracked, and the data formats employed in these studies. More importantly, it analyzes how raw motion-tracking data streams are transformed into kinematic or trajectory-level features and the analytical methods used to link them to psychological constructs. Our findings highlight substantial variability in device configurations and motion data extraction methods, but we observe a common pattern of using descriptive statistical measures to summarize motion patterns or applying machine learning models to learn from processed motion features. Based on these observations, we outline practical recommendations for selecting tracking setups, collecting and validating motion data, and designing analysis pipelines in experimental psychological studies. This review aims to support researchers seeking to incorporate motion-tracking data into XR-based psychological experiments and to promote more consistent and interpretable use of motion measures in immersive research.