LUIDA: Large-scale Unified Infrastructure for Digital Assessments based on commercial metaverse platform

1 Introduction

The advancement of experimental research in Human-Computer Interaction (HCI) and Virtual Reality (VR) has been accompanied by various tools and platforms, which remain highly fragmented: many of them target specific research domains (Gonzalez-Franco et al., 2020; Zenner et al., 2021) or isolated procedures such as participant recruitment (Ueshima, 2026; Sona SystemsLtd, 2026) or data collection (Bebko and Troje, 2020). Such fragmentation forces researchers to juggle multiple systems throughout the experimental workflow, compromising experimental design consistency and increasing workflow complexity.

Simultaneously, due to the growth of crowd-sourcing platforms such as Amazon Mechanical Turk ¹ , online experimental systems are gaining significant attention as alternatives or supplements to traditional in-person experiments, particularly since the outbreak of the COVID-19 pandemic in 2020. In the field of VR research, the possibility of online experimentation has also been demonstrated by the growing use of metaverse platforms such as VRChat ² . Studies have demonstrated that experimental configurations originally designed for physical settings can be effectively transferred to virtual environments (Radiah et al., 2021; Saffo et al., 2021), enabling access to a broader, more diverse participant pool. However, despite their promise, these online approaches come with significant limitations. Technical barriers persist, as crowd-sourcing platforms require expertise in networking or experiment automation; current metaverse attempts are restricted by limited customizability or data collection methods. Furthermore, the lack of a structured methodology burdens researchers with acquiring platform-specific expertise or forcing them to fall back on inefficient, manual methods. For instance, researchers implementing their experimental system with available toolkits (e.g., bmlTUX (Bebko and Troje, 2020)) are still required to find participant pools via recruitment platforms (e.g., Sona (Sona SystemsLtd, 2026)) or implement additional programs to retrieve data over the internet; experiments in social VR necessitate that researchers manually record data instead of using automatic logging (Saffo et al., 2021).

To address these challenges, we propose the experimental framework LUIDA (Large-scale Unified Infrastructure for Digital Assessments). Its novelty lies in (1) integrating typically fragmented experimental workflows, from recruitment to data collection, and (2) establishing a structured methodology for metaverse-based experimentation by leveraging inherent platform capabilities for automation and parallelization, while abstracting technical complexities via an adaptable low-code template. Specifically, LUIDA conducts participant recruitment directly within the metaverse via an in-world bulletin board; automates and parallelizes experiment execution by leveraging the metaverse’s capacity to host multiple virtual spaces for simultaneous sessions; provides automated, continuous motion tracking with customizable data collection uploaded to online storage; and offers a low-code Unity-based template for experiment implementation, with graphical interfaces for variables and state machine configurations.

LUIDA is designed with a platform-agnostic philosophy, allowing adaptation to different metaverse platforms. In this paper, we develop a prototype using the commercial metaverse platform Cluster to leverage its existing user base and content creation tools. To evaluate LUIDA’s effectiveness, we performed a two-step validation: a user study with VR researchers replicating representative experiments from perception, cognition, and user interface domains, followed by public deployment on Cluster with general participants.

Our main contributions include the following:

Unified experimental framework: This study proposed LUIDA, a framework that unifies fragmented VR experimental workflows by providing efficient solutions, including system implementation across multiple research domains, and automated, parallel participant recruitment and data collection.

Structured methodology for metaverse-based research: LUIDA provides a novel, structured methodology that requires minimal platform-specific expertise and removes reliance on inefficient, manual approaches.

Evaluation of experimental validity: We conducted user studies with VR researchers to use LUIDA, as well as online experiments with nearly 200 participants per experiment, a total of over 500 datasets in 1 week.

Pathway to an open platform: We identified limitations and future development pathways of LUIDA to release it as an open platform.

The remainder of this paper is structured as follows. Section 2 supports LUIDA’s novelty by reviewing the relevant literature, comparing LUIDA with current tools and prior metaverse-based attempts in addressing desirable functionalities for VR researchers. Section 3 details the system architecture, its components, and their implementation, designed to address the research needs identified in Section 2. Section 4 outlines the evaluation design using three representative experiments, while Sections 5, 6 report the two evaluation studies: a usability study with VR researchers and the subsequent large-scale online replications. Finally, Section 7 discusses the findings, Section 8 lists limitations and future work, and Section 9 concludes the research.

2 Related work

2.1 Desirable functionalities for comprehensive VR research

Based on a review of previous literature and existing tools, we identified desirable functionalities for improving the efficiency, quality, and validity of VR studies. We organize them by the four phases of the experimental workflow (participant recruitment, experiment execution, data collection, and experiment system implementation), and within each phase we discuss the requirements that most strongly determine whether a study can be run remotely and at scale.

2.1.1 Participant recruitment

Recruitment is shaped by two requirements that traditional methods struggle to satisfy together: reaching large and diverse participant pools, and reaching VR-experienced users. The first matters because VR studies, especially in psychology and cognition, increasingly require large samples of dozens or hundreds of participants (Ratan et al., 2020; Makransky and Petersen, 2021; Moreno et al., 2019). However, laboratory methods face time and resource constraints, and small, homogeneous samples yield underpowered studies with limited generalizability (Lanier et al., 2019; Delgado Rodriguez et al., 2023); larger and more diverse pools therefore improve statistical power and external validity.

The second requirement, access to VR-experienced users, is critical because traditional crowdsourcing tools struggle to recruit them (Ma et al., 2018), whereas metaverse platforms offer a global pool of users who already own the necessary hardware (Saffo et al., 2020). They also enable “in-the-wild” studies of naturalistic social behavior, serving as a “living laboratory” for interactions difficult to reproduce in controlled settings (Shriram and Schwartz, 2017; Wang and Bailenson, 2024).

Existing approaches could not resolve both requirements at once. For instance, external recruitment services such as Sona (Sona SystemsLtd, 2026) and Jikken-Baito (Ueshima, 2026) enable researchers to broaden their reach, but they are not tailored to users with VR experience.

2.1.2 Experiment execution

Execution involves the most requirements, because running a study remotely and at scale brings together multi-stage flow, avatars, asynchronous operation, infrastructure provisioning and deployment, and ongoing management. We review each of these requirements in the paragraphs below.

First, experiments often unfold over multiple stages, such as avatar familiarization (van Loon et al., 2018; Waltemate et al., 2018; Roth and Latoschik, 2020), in-VR questionnaires (Putze et al., 2020), tasks before or after the main trials (Cameirão et al., 2010), and consecutive scenarios like training (Gavish et al., 2015) or perspective-taking (van Loon et al., 2018), as well as having participants practice the device controls and the task procedure so that they can operate them before the main trials (Grübel et al., 2017). Some tools partially automate this (for example, AGENT generates experimental protocol runtimes (Le Moulec et al., 2017)), yet a unified protocol for managing such stages is still lacking.

Avatars are a second variable of interest: as users’ virtual representations, they foster a sense of embodiment that influences emotion, behavior, and even self-perception (Kilteni et al., 2012), which makes them powerful experimental factors for studying perception (Venkatakrishnan et al., 2023) and the Proteus effect, where avatar appearance shapes attitudes and behavior (Yee and Bailenson, 2007; Ratan et al., 2020). Shared avatar libraries such as Microsoft Rocketbox (Gonzalez-Franco et al., 2020) help researchers use them as controlled stimuli.

Scale is a third concern: one-on-one lab sessions limit efficiency, so some studies have built remote, networked infrastructures that support synchronous or asynchronous participation, either by distributing software and hardware (Radiah et al., 2021; Steed et al., 2020) or by hosting multi-user servers for large-scale studies (Aguilar et al., 2024; Zhao et al., 2018), an approach taken up in human-building-interaction research such as building-scale wayfinding studies Gath-Morad et al. (2024). Building and hosting such infrastructure, however, adds considerable burden in resources, coordination, and networking expertise, and participants who do not already own a headset may even need devices shipped to them. Metaverse platforms, by contrast, can host many virtual spaces asynchronously and in parallel for users who already own VR devices, which makes them a promising route to large-scale participation without placing this burden on researchers.

Finally, running studies must be managed and monitored: experimenters register and configure experiments, oversee their progress, and handle participant management and compensation (Aguilar et al., 2024). Without a single system to handle these tasks, researchers fall back on ad hoc scripts and spreadsheets, which are prone to error, especially in unsupervised, “in-the-wild” studies where no experimenter is present to catch mistakes (Steed et al., 2016; Delgado Rodriguez et al., 2023).

2.1.3 Data collection

A comprehensive framework must support the full path from capture to a usable, trustworthy dataset. We summarize the requirements that most affect remote VR studies regarding data: immersive questionnaire administration, real-time behavioral tracking, integration with external hardware, reliable assembly of distributed data, downstream analysis and replay, and data integrity and privacy.

Administering questionnaires within VRpreserves presence (Schwind et al., 2019) and avoids the immersion breaks caused by switching between virtual and real space (Putze et al., 2020), benefiting studies of presence (Schubert et al., 2001; Schwind et al., 2019), embodiment (Roth and Latoschik, 2020; Genay et al., 2022), and applications (Rose et al., 2018). Dedicated tools such as the VRQuestionnaireToolkit (Feick et al., 2020) and VRate (Regal et al., 2018) support it.

Real-time behavioral tracking, the continuous capture of positions and postures, enables quantifying subtle patterns such as proxemics (Yaremych and Persky, 2019) and avatar-induced movement changes (Kilteni et al., 2013). Software such as Virtual Observations (Howie and Gilardi, 2020) supports contextual capture and assessment of users’ actions.

Integration with external hardwareextends data collection beyond what headsets capture, such as biometrics (Lüddecke and Felnhofer, 2022), EMG (Al-Jumaily and Olivares, 2009), and eye gaze (Clay et al., 2019). It also enables outputs such as haptic feedback (Wang et al., 2019) or full-body tracking (Caserman et al., 2019).

Once data come from many distributed participants, reliable data assembly becomes essential: results must be gathered into a central store that can be queried (Aguilar et al., 2024), since relying on manual export or local files risks loss and inconsistency, as in social-VR experiments that recorded results by photographing an in-world wall (Saffo et al., 2021). A comprehensive framework ideally also supports downstream analysis and the replay of recorded sessions for inspection or re-coding (Aguilar et al., 2024; Grübel, 2023), which improve traceability and reuse. Recent systems like Ubiq-exp (Steed et al., 2022) and Virtual observations (Howie and Gilardi, 2020) support recording and replay, as well as exporting formatted data for later analysis.

Finally, data integrity and privacyare essential: unsupervised remote experiments risk non-compliance and low-quality data (Delgado Rodriguez et al., 2023; Steed et al., 2016), which can be reduced by device verification or careful filtering (Ma et al., 2018; Steed et al., 2016). Collecting and transmitting sensitive data online also requires robust anonymization and secure storage (Delgado Rodriguez et al., 2023).

2.1.4 Experiment system implementation

A final requirement is lowering the implementation barrier for researchers without strong technical backgrounds, since VR research is expanding into multidisciplinary fields such as psychology, social science, neuroscience, and healthcare (Parsons, 2015; Halbig et al., 2022; Rose et al., 2018). Implementing VR experiments typically relies on general-purpose game engines (e.g., Unity ³ , Unreal Engine ⁴ ) as the underlying development environment, which overwhelm domain experts who lack programming expertise (Parsons, 2015; Halbig et al., 2022); dedicated experiment frameworks built on these engines, or low/no-code templates, can lower this barrier. Such tools range from domain-specific toolkits (e.g., HaRT (Zenner et al., 2021) for hand redirection, Virtual SILCton (Weisberg et al., 2014) and OpenMaze (Alsbury-Nealy et al., 2022) for spatial navigation) to more general toolkits (e.g., VREX (Vasser et al., 2017), Toggle Toolkit (Ugwitz et al., 2021), non-Unity vexptoolbox (Schuetz et al., 2023)).

Many of these requirements, particularly for data collection and implementation, have been integrated in frameworks such as EVE (Grübel et al., 2016) and its physiological-data extensions (Weibel et al., 2018), and applied in studies of public health (Hackman et al., 2019), navigation (Schinazi et al., 2023), and environmental perception (Spielhofer et al., 2021). Our contribution is therefore not to discover these requirements but to integrate them into a single, structured workflow that runs on a commercial metaverse platform with a built-in user pool, which we argue is a promising direction for advancing VR research.

2.2 Situating LUIDA against existing tools and paradigms

2.2.1 Comparison with existing tools

Table 1 compares comprehensive, multi-component experimentation systems with LUIDA. To structure the comparison we adopt the six-pillar experimentation taxonomy of ExaC (Aguilar et al., 2024), whose full survey of systems appears in their paper’s Table 3.

TABLE 1. Comparison of comprehensive, multi-component VR/HCI experimentation systems, organized following the six-pillar experimentation taxonomy of ExaC (Aguilar et al., 2024) (pillars 2–6; pillar 1 is omitted as out of scope).

✓: supported; △: partially or limitedly supported; -: not supported. Components marked.

were added for metaverse-based experimentation (no counterpart in the original taxonomy). Single-purpose tools (e.g., recruitment sites, asset libraries, questionnaire toolkits) are discussed in Section 2.1 rather than graded here. Cell footnotes

platform-managed, not researcher-owned code.

server provided, app distribution manual.

platform-hosted, sessions created manually.

manual experimenter co-presence.

institutional pool via DeSciL.

on-screen desktop, not in-HMD.

our judgment.

MTurk crowd, not VR users.

fixed phases, no configurable state machine.

Across the workflow phases, existing comprehensive systems each leave gaps that LUIDA fills. For recruitment, platforms such as Jikken-Baito (Ueshima, 2026) and Sona (Sona SystemsLtd, 2026) handle recruitment alone and lack VR-experienced pools, and prior social-VR attempts (Saffo et al., 2020) reach this user base only through manual, in-person invitations, whereas LUIDA provides a dedicated virtual recruitment world for broad, ecologically valid access. For execution, frameworks such as QuickVR (Oliva et al., 2022), bmlTUX (Bebko and Troje, 2020), UXF (Brookes et al., 2020), USE (Watson et al., 2019), and Ubiq-exp (Steed et al., 2022) provide multi-stage management (event sequences, session–block–trial models, state machines (Wagner et al., 2006)) that LUIDA also adopts; Microsoft Rocketbox (Gonzalez-Franco et al., 2020) and QuickVR’s embodiment support motivate LUIDA’s avatar control; and where most tools lack asynchronous operation, LUIDA (like Ubiq-exp) runs automated, large-scale parallel sessions by using the platform’s capacity to host many worlds. For data collection, LUIDA automates behavioral tracking and customizable logging (matching HaRT (Zenner et al., 2021), bmlTUX, UXF, USE, and Ubiq-exp) while adding a web console for easy access, integrates in-VR questionnaires (otherwise found only in VRQuestionnaireToolkit (Feick et al., 2020) and Ubiq-exp), and supports external hardware through the Open Sound Control (OSC) protocol ⁵ , complementing QuickVR’s motion capture and USE’s hardware synchronization. For implementation, these frameworks are abstraction layers built on top of general-purpose game engines rather than authoring platforms in themselves; all of those compared here are Unity-based, and some are domain-specific (for example, QuickVR (Oliva et al., 2022) focuses on avatar embodiment experiments). UXF, USE, and Ubiq-exp require complex configuration files or expertise in the underlying engine and networking, while bmlTUX and HaRT offer friendlier interfaces but are limited to factorial designs or hand-redirection tasks; LUIDA therefore contributes an accessible low-code Unity template.

Most notably, provisioning and deployment, the two infrastructure components that Aguilar et al. ’s survey marks as never implemented (Aguilar et al., 2024), are covered by both LUIDA and Aguilar et al. ’s own proof-of-concept, SimpleExaCT, but through different routes: a managed platform versus researcher-owned infrastructure code. The two systems are largely complementary: SimpleExaCT automates researcher-owned infrastructure and analysis for desktop, browser-based VR, whereas LUIDA delivers immersive, multi-user VR with in-platform recruitment. This positions LUIDA as a metaverse-native counterpart to ExaC rather than a competitor.

Finally, by scope, LUIDA omits the documentation pillar and only partially supports analysis and monitoring components, with no session replay. We revisit these trade-offs and the resulting roadmap in Section 7.

2.2.2 Positioning within metaverse-based research and the experimentation lifecycle

Beyond individual tools, integrating metaverse platforms with research methodology has emerged as a key strategy for remote VR studies. Radiah et al. highlight its high accessibility to existing user bases and its reduced development overhead (Radiah et al., 2021), although challenges remain in data collection, experimental control, and device diversity, all of which can affect internal validity (Mohajan, 2017). Saffo et al. validated social VR for remote experiments, but they hosted sessions one by one with manual procedures, such as photographing results on a virtual wall (Saffo et al., 2021), which shows the need for automation and parallelization. Saffo et al. also identify two strategies: partnering with an existing platform, or building a dedicated one. The first offers established infrastructure and a user base but limited customizability, while the second offers full customizability at a high development cost. LUIDA adopts a hybrid strategy, partnering with an existing platform while adding research-tailored features, and is designed to be platform-agnostic so that its core functionalities remain portable across metaverse environments.

Finally, this positioning aligns with Grübel’s lifecycle view of VR experimentation (Grübel, 2023), which traces three generations of research frameworks: a first that handled the hardware setup, a second that raised the level of abstraction to experiment design (offering task templates and flow control), and an emerging third aimed at full-lifecycle automation and reproducibility, summarized as the DEAR principle (Design, Experiment, Analyse, Reproduce). Observing that existing research frameworks tend to “cannibalise” one another, Grübel argues that future frameworks of the third generation should build on existing work rather than re-implementing it. LUIDA follows this stance by adopting established second-generation patterns, such as state-machine flow control (Wagner et al., 2006) adapted from QuickVR’s embodiment workflow (Oliva et al., 2022), and instead contributing automation at the metaverse-platform layer, most directly advancing the DEAR “Experiment” phase (more discussion about LUIDA’s coverage of the full lifecycle in Section 7.6). In summary, LUIDA integrates the otherwise fragmented processes of VR experimentation: recruitment, execution, data collection, and implementation, into a structured, metaverse-native paradigm, whose detailed design follows in later sections.

3 The LUIDA framework: system architecture and design

This section presents LUIDA’s architecture and design. We first give an overview of the five components and how they interact, then describe each component in turn, pairing its design concept with how we implemented it on the metaverse platform Cluster. We finally discuss platform selection, constraints, and portability (Section 3.7) and the user workflow (Section 3.8).

3.1 Overview

Figure 1 shows the overall architecture of LUIDA, which consists of five components. The Web Console (A) is a web interface where researchers register experiments, configure questionnaires and avatars, and visualize and download the collected data. The Storage (B) is a central database/storage that holds experiment information and collected data. The Implementation Template (C) is a Unity-based editor that provides no-code interfaces for building an experiment, including variable management, state-machine control, and the definition of data-collection formats. The Experiment World (D) is the virtual environment, generated from the template, where the experiment runs; it displays questionnaires and records data to Storage. The Recruitment World (E) is the entry point where participants browse available experiments and move to a specific Experiment World.

These components interact through a small number of data flows. The Web Console reads and writes Storage to register experiments and retrieve results, and the Recruitment World queries Storage to keep its public bulletin board up to date. When a participant joins an experiment, the system creates a unique Experiment World from the world the researcher built with the Implementation Template; during the session, that Experiment World applies the parameters, state machine, and data formats defined in the template, loads questionnaire items from Storage by their identifier, and saves all collected data back to Storage, where researchers can access it through the Web Console. Because the Web Console and Storage interact with the worlds only through this data interface, they are largely independent of the underlying metaverse platform, a property we return to in Section 3.7.

3.2 Web console (A)

The Web Console is the researcher’s single entry point for managing a study. Through it, researchers register and edit experiment information (title, description, prerequisites, rewards, and consent text), configure questionnaires and their response options, upload and manage avatars, control whether an experiment is publicly visible, and visualize and download the collected data. Bringing these tasks into one interface addresses the management and monitoring needs discussed in Section 2.1.2.

In our prototype, the Web Console is a web application deployed on Vercel ⁶ (screenshots are provided in Supplementary Section 1).

3.3 Storage (B)

Storage is the central database/storage that holds both the experiment information registered through the Web Console and the data collected during sessions, which addresses the reliable data-assembly need in Section 2.1.3. To protect data integrity and participant privacy, every record is given a unique, non-traceable identifier that preserves anonymity while keeping each participant’s data consistent, together with a timestamp that establishes a reliable chronological order and helps researchers identify data affected by network delays.

In our prototype, experiment information, questionnaire items, and answers are stored on MongoDB Atlas, a cloud-hosted NoSQL database ⁷ . Data collected during the experiment with customized fields configured by the experimenter are stored on Amazon S3 ⁸ , a standard cloud storage service.

3.4 Implementation template (C)

The Implementation Template is a Unity project that lets researchers build an Experiment World without advanced manipulation or coding, which addresses the implementation-support need in Section 2.1.4. Basic operations such as placing virtual objects are handled by Unity’s native features, while the template focuses on automating the experimental flow through two no-code interfaces: a state machine that controls the sequence of events, and a variable system that generates trials automatically. The template is also where researchers link questionnaires and define data-collection formats, as described below.

3.4.1 Experiment flow management via state machine

LUIDA’s template uses a centralized finite state machine (Wagner et al., 2006; Unity Technologies, 2021) to control complex experimental flows, such as showing a virtual mirror for avatar acclimation, practicing tasks before trials, or triggering a questionnaire at a specific moment. Through a no-code interface, researchers define states and their transitions and control each state’s duration or repetition (Figure 2). Following QuickVR’s workflow system (Oliva et al., 2022), the template adopts the observer pattern (Gamma et al., 1994): researchers attach “state listeners” to in-scene objects through a graphical interface, and each listener runs actions at three points in a state’s lifecycle (On State Start, During State, and On State Exit). Available actions include changing the visibility or position of a player or object, setting text, sending haptic feedback, saving data, and sending an OSC message to external hardware; the complete list and the listener interface are provided in Supplementary Section 2, and a “Customized Action” code block is available for researchers with programming experience.

3.4.2 Variable configuration and trial generation

The template also configures experimental variables and generates the corresponding trials automatically, which reduces human error and preparation time and supports the factorial designs noted in Section 2.1.2. Through a no-code interface (Figure 3), researchers define within- and between-subject variables with their possible values, the number of repetitions, and randomization options. At run time, values for between-subject variables are assigned randomly, and dedicated trial states in the state machine iterate over all combinations of within-subject variables and repetitions. Variable values can also serve as conditions that decide whether an action runs.

3.4.3 Questionnaire and data-collection configuration

Questionnaires and data formats are also linked through the template. Researchers register questions and response formats (e.g., Likert or multiple-choice) in the Web Console, then set the matching questionnaire identifier in the template (Figure 4); during the session the Experiment World loads the items from Storage and generates the in-VR questionnaire automatically, keeping the questionnaire inside VR as discussed in Section 2.1.3. Researchers similarly define custom data formats in a dedicated file to record experiment-specific values, such as per-trial task scores; these are collected automatically and can later be downloaded from the Web Console.

3.4.4 Avatar configuration

The template also supports avatar-based studies: researchers upload and configure avatars through the Web Console, treat avatar characteristics such as appearance or movement as experimental variables, and use the state machine to help participants acclimate to a new body before trials. Switching a participant’s avatar mid-session, however, is constrained on Cluster, which assigns an avatar only on world entry or have the players switch by themselves. The present prototype works around this by registering several single-avatar worlds under one experiment and redirecting participants between them through portals, a workaround that adds world transitions. As future work, we plan to import humanoid models (e.g., VRM files) directly into the Experiment World and drive them with our inverse-kinematics scripts, so an avatar follows the participant’s motion and can be switched within a single world.

3.4.5 Implementation on cluster

In our prototype, the template is a Unity project built with the Cluster Creator Kit (CCK) ⁹ . The interfaces for questionnaire linking, state-machine management, variable configuration, and data-format specification were implemented in C# inside the Unity editor, and the configurations they produce are translated into CCK components and ClusterScript, so researchers do not write C# themselves. Once a world is uploaded, execution of these configurations is handled entirely by CCK and ClusterScript. On another platform, the same no-code interfaces and state-machine functionalities could be reimplemented with that platform’s world SDK and scripting layer (for example, the VRChat SDK with Udon), as we discuss in Section 3.7.

3.5 Experiment World (D)

The Experiment World is the virtual environment, generated from the Implementation Template, where a session actually runs. It executes the template’s state machine, variables, questionnaires, and data formats automatically, and it covers the execution and data-collection needs from Section 2.1.2 and Section 2.1.3.

To run experiments asynchronously and in parallel, LUIDA uses the platform’s ability to create separate world instances, generating a unique, isolated instance for each session from the Experiment World template. This enables many simultaneous sessions without researcher oversight and accelerates data collection. To protect data quality, the system prevents re-entry into completed experiments and can redirect non-VR users away from VR-only experiments.

During a session, the Experiment World records frame by frame position and rotation data from the HMD, controllers, and additional trackers (if available for participants), which supports the analysis of subtle behavioral patterns such as proxemics (Yaremych and Persky, 2019). It also connects to external software or hardware through the host platform’s data-streaming features, such as the OSC protocol supported by both VRChat and Cluster, extending LUIDA to experiments that need real-time signals such as EEG or eye tracking.

In our prototype on Cluster, the Experiment World leverages Cluster’s automatic instance generation: when the number of participants reaches a configured limit (one by default), the platform creates a new instance for the next participant, giving each session its own isolated world. This per-session instancing is central to LUIDA’s parallel execution; Section 3.7 discusses how far it carries to other platforms.

3.6 Recruitment world (E)

The Recruitment World is a shared space where recruitment happens in situ, which addresses the user-pool and VR-user-access needs in Section 2.1.1. Many metaverse platforms host individual virtual spaces (“Worlds” in Cluster ¹⁰ and VRChat, “Rooms” in Rec Room ¹¹ , or “Regions” in Second Life ¹² ) where users gather and socialize. LUIDA places a bulletin board in such a space that lists all available studies, so users can browse and join on their own and researchers no longer need to seek out participants actively.

In our prototype, the Recruitment World is a public Cluster world open to all users, with a tavern-like, role-playing-game appearance that lowers the psychological barrier to taking part. The bulletin board is populated from Storage, so the list of studies stays current as researchers publish or hide experiments through the Web Console.

3.7 Platform selection, constraints, and portability

We built the LUIDA prototype on Cluster for three reasons. First, Cluster has a diverse user base across age groups with a balanced gender ratio (Figure 5), which can improve the generalizability of findings. Second, its creation tools, the Unity-based CCK for no-code development and ClusterScript for optional, advanced scripting, suit researchers with different levels of expertise. Third, Cluster supports many SteamVR ¹³ -compatible headsets (Meta Quest, HTC Vive, Valve Index, Pico, and others), which broadens participant access; it also supports PC and mobile, extending LUIDA’s potential to wider HCI experiments.

Cluster also has constraints, such as lacking native finger tracking and pass-through; its avatar-assignment constraint and our current workaround are discussed in Section 3.4.4. Some constraints are not Cluster-specific but shared by all metaverse platforms. For instance, network latency can affect timing precision in experiments that need millisecond-level synchronization; LUIDA mitigates this by timestamping data so that researchers can identify and filter delayed records. Scalability also depends on the host platform’s capacity for many concurrent instances; given that platforms such as VRChat sustain on the order of 50,000 concurrent users in 2025 (SteamDB, 2026), we did not treat this as urgent, although formal stress tests remain future work.

Finally, LUIDA is designed to be loosely coupled and platform-agnostic, though the current prototype does not yet fully meet this goal. The Web Console and Storage are ordinary web and cloud services that reach the worlds only through a data interface, so they can already be reused on another platform without change. The in-world parts built with the Implementation Template, such as variable configuration and the state machine, are instead tied to Cluster’s SDK (CCK) and scripting (ClusterScript), so the present prototype cannot be ported directly to a platform such as VRChat; that would require rebuilding the Recruitment World, the Experiment World, and the Implementation Template with the target platform’s SDK (for example, the VRChat SDK with Udon or UdonSharp). A port would also need to reproduce platform capabilities LUIDA benefits from Cluster. Take VRChat as an example: first, LUIDA relies on automatic per-session instancing; while a VRChat world script cannot create instances or move players itself ¹⁴ , VRChat lets creators set a world’s capacity and sends extra users into a new instance of the same world once one is full, much like Cluster. A small capacity could thus give each participant an isolated instance without custom queuing. Second, LUIDA uploads data to external Storage during a session, while VRChat blocks arbitrary outbound networking, its recent SDK can call pre-registered (allowlisted) URLs ¹⁵ , though only GET requests are allowed. In future work, we plan to lower this porting cost with a thin abstraction layer that hides each platform’s SDK behind a common interface, so one template and state-machine definition could target several platforms, bringing LUIDA closer to its platform-agnostic goal. For now, we therefore present platform-agnostic portability as a design goal already met for the Web Console and Storage but only partially met for the in-world components.

3.8 User workflow

From the participant’s side, the workflow proceeds as follows (Figure 6). Participants enter the shared Recruitment World and browse the bulletin board, which lists each study with its requirements and rewards. After selecting an experiment, they step through a portal into the corresponding Experiment World, give consent, and begin the tasks; during the session they answer in-VR questionnaires and perform tasks while the system records positional data from the headset and applies it to the avatar’s full body through inverse kinematics. When the experiment ends, participants receive their reward and return to the Recruitment World, where they can join another study or leave.

From the researcher’s side, the workflow mirrors the components above. Researchers first use the Web Console to register the experiment information and questionnaire content, linking each questionnaire identifier to the items it should display. They then download the Implementation Template, build the Experiment World by linking questionnaires (Section 3.4.3), defining data formats (Section 3.4.3), and automating the flow with the state machine and variables (Section 3.4), and upload the finished world so that it becomes available from the Recruitment World. They can then toggle the experiment’s visibility through the Web Console, for example, hiding it once enough data are collected. As sessions complete, data are saved automatically to Storage, and researchers monitor progress and download results through the Web Console.

4 Evaluation design

4.1 Research questions

The following research questions (RQs) were formulated regarding LUIDA’s efficacy for experimentation:

• RQ1 How does LUIDA benefit researchers conducting VR experiments?
• RQ2 Can LUIDA accommodate a variety of VR experiments?
• RQ3 How does LUIDA promote the participation of participants in experiments?
• RQ4 Does LUIDA maintain experimental validity compared to traditional laboratory experiments?

We addressed these RQs through two studies. Before conducting them, we sought to select target research domains to validate whether LUIDA could accommodate a wide variety of VR experiments. Viewing the topics listed by major VR conferences such as IEEE VR (Society, I. C., the Visualization, and Committee, G. T., 2025), we observed that, excluding those focusing on infrastructure (e.g., “XR technology infrastructure”), the remaining user-study-oriented topics can be broadly classified into three domains: perception (e.g., “Cybersickness”, “Multimodal/cross-modal interaction and perception”), cognition (e.g., “Embodied agents, virtual humans and (self-)avatars”, “User experience and usability”), and user interfaces (e.g., “3D user interfaces”, “Locomotion and navigation”). For each domain, we chose a representative experiment for later validations: a hand redirection study by Zenner et al. (Section 4.2) for perception, a Proteus effect study by Kilteni et al. (Section 4.3) for cognition, and a 3D Fitts’ Law study by Clark et al. (Section 4.4) for user interfaces. Notably, such experiments are typically served by domain-specific tools, for example, HaRT (Zenner et al., 2021) for hand redirection and QuickVR (Oliva et al., 2022) for avatar embodiment. Replicating them on LUIDA without relying on these specialized tools shows its general-purpose reach across VR research domains.

Then, in the first study, we evaluated LUIDA’s benefits and versatility (RQ1, RQ2) by inviting VR researchers to replicate the one of the three chosen experiments on LUIDA and collecting feedback from them (Section 5). For the second study, we replicated the three chosen experiments on LUIDA and make them public to Cluster users. The online participation and collected data allowed us to examine participant engagement and experimental validity (RQ3, RQ4) (Section 6). All experiments were approved by the Cluster Metaverse Lab Ethics Committee (2024–003). Because this committee is internal to Cluster Metaverse Lab, its registration, composition, and independence from the research team are described in the Ethics Statement, and the internal-review limitation is noted in Section 8.2.

The three selected experiments are summarized in Table 2 and briefly introduced in the remainder of this section. Because they replicate established prior studies, we omit their full step-by-step procedures (and the corresponding figures) here and instead provide them in Supplementary Section 3.

TABLE 2. Overview of the three representative experiments replicated to evaluate LUIDA across VR research domains, together with the measurements taken and the hypotheses tested in each. Full step-by-step procedures are provided in Supplementary Section 3.

4.2 Measuring detection threshold of hand redirection

Hand redirection introduces discrepancies between real and virtual hand movements, often to enable interaction in a larger virtual space than the physical one affords. We replicated the “gain warp without distraction” condition from Zenner and Krüger (2019) in a within-subjects design: participants repeatedly performed a reaching task under 11 gain values (0.75–1.25, two trials each in randomized order) and reported whether their virtual hand felt faster or slower than their real hand, from which a detection threshold (DT) was computed (see Supplementary Section 3.1 for the full procedure).

4.3 Proteus Effect during drumming in VR

In study (Kilteni et al., 2013), participants played an African djembe drum while embodying either a Formal Light-skinned (FL) avatar in a suit or a Casual Dark-skinned (CD) avatar in casual clothing, and performance was quantified from body-part positions during drumming to assess how avatar appearance cognitively influences behavior. We replicated this experiment in a between-subjects design, in which participants first drummed in a baseline condition (transparent avatar) and then drummed again after being assigned and acclimating to their FL or CD avatar (see Supplementary Section 3.2 for the full procedure).

4.4 Extending fitts’ law to 3D environment

Fitts’ Law models movement time to a target based on its distance and size. Clark et al. (2020) extended this law to 3D virtual environments by having participants move a virtual object to targets of varying sizes and 3D positions, and compared their model with the original Fitts’ Law and other similar 3D extensions (Hoffmann, 1995; Murata and Iwase, 2001; Cha and Myung, 2013; Barrera Machuca and Stuerzlinger, 2019). We replicated this experiment in a within-subjects design in which participants completed 100 trials with varying target sizes and 3D coordinates (see Supplementary Section 3.3 for the full procedure).

5 Study 1: usability evaluation with VR researchers

We conducted a usability evaluation with VR researchers using LUIDA to investigate RQ1 and RQ2.

5.1 Experiment setup

5.1.1 Participants and procedure

We recruited 6 researchers with experience in VR experiments (age = 23.83 ± 1.83 years) using the kinship method. They were selected on two criteria: prior hands-on experience in running VR experiments, and a research focus that matched one of the three target domains (perception, cognition, or user interfaces). Each was assigned one of the three experimental designs from Section 4 that best matched their expertise and was tasked with implementing it using LUIDA, rewarded with a 15,000 JPY (≈ 100 USD) Amazon gift card.

The study was conducted asynchronously, with participants completing the following steps within a 1-week timeframe:

• Read an introduction and manual of LUIDA.
• Download the Implementation Template and implement the experiment according to the provided manual.
• Register and publish the implemented experimental system via the Web Console of LUIDA.

After implementation, each researcher tested their own experiment by accessing it from the Recruitment World, running the tasks, and reviewing the collected data on the Web Console. Finally, they provided feedback on LUIDA’s benefits and limitations through questionnaires and an interview.

5.1.2 Evaluation metrics

We used the System Usability Scale (SUS) (Brooke, 1996) for usability and NASA-TLX (Hart and Staveland, 1988) for workload. These quantitative metrics were collected twice: once for the Implementation Template specifically and again for the overall LUIDA workflow (from registration to data review). We also conducted interviews to gather qualitative insights into their ratings, perspectives, and suggestions for improvement.

5.2 Evaluation results for overall LUIDA usage

The following presents the results of our quantitative assessment and a summary of the researchers’ interview responses. A complete list of the actual responses can be found in Section 5 of the Supplementary Material, “Interview Responses from VR Researchers Participating in the Usability Evaluation.”

5.2.1 Quantitative assessment

Regarding the overall process of using LUIDA, the SUS score was 73.75 ± 13.39, interpreted as ‘Good’ among ‘Worst Imaginable’, ‘Awful’, ‘Poor’, ‘OK’, ‘Good’, ‘Excellent’ and ‘Best Imaginable’ (Bangor et al., 2008).

The NASA-TLX score was 24.11 ± 21.01, interpreted as ‘Medium’ among ‘Low’ (0–9), ‘Medium’ (10–29), ‘Somewhat high’ (30–49), ‘High’ (50–79) and ‘Very high’ (80–100) (Prabaswari et al., 2019).

5.2.2 Key benefits

The interviews revealed the following key benefits of LUIDA, with participants’ responses available in Section 5.1 of the Supplementary Material.

• Automation and Integration of Experimental Processes: The automation of participant recruitment and data collection streamlines cumbersome tasks typically associated with traditional laboratory experiments.
• Parallel Experiment Execution: The system manages the execution of the experiment autonomously, without direct involvement of the researcher.
• Enhanced Accessibility: Remote participation allows access to diverse groups of participants, including those from rural and international locations.
• Improved Information Management: The Web Console’s management of questionnaires and collected data improves efficiency and minimizes procedural errors.

5.2.3 Challenges and improvement suggestions

Participants identified challenges and potential improvements, with their actual responses available in Supplementary Section 5.2.

• Platform-Dependent User Base: Dependency on Cluster limits the participant pool and may introduce bias. To address this, participants suggested improving support for users unfamiliar with Cluster to broaden the participant base and reduce platform dependency.
• Challenges in Environmental Monitoring: Verifying participant engagement and proper adherence to instructions proved difficult. To resolve this, participants suggested real-time support features, including automated alerts, Q&A modules, and recording capabilities for post-experiment review.

5.3 Evaluation results for the implementation template

5.3.1 Quantitative assessment

The SUS score was 44.17 ± 11.80, which falls between ‘Poor’ and ‘OK’ (Bangor et al., 2008). The NASA-TLX score was 63.33 ± 15.45, interpreted as ‘High’ (Prabaswari et al., 2019).

5.3.2 Strengths and challenges

Participants suggest that the main strength of the Implementation Template is the integration of functions for rational experimentation, with their actual responses available in Section 6.1 of the Supplementary Material.

• Integrated Approach: The template eliminates the need to set up a dedicated server, allowing experiments to be run online directly on the platform.
• Built-in Functionalities: Automated questionnaire generation, state management, direct data recording, and CCK integration simplify experimental design and reduce manual overhead.

However, participants noted that there were challenges with complexity, debugging, and interface design:

• Complexity and Learning Curve: Although powerful, the state management feature was considered overly complex for simpler experimental designs and debugging, especially for users unfamiliar with CCK or scripting.
• User Interface (UI): Cluttered layouts and lack of intuitive visual cues affected usability.
• Partial No-Code Interface: Frequent switching between coding and no-code user experiences during implementation potentially introduced confusion.

5.3.3 Improvement suggestions

Participants suggested several solutions to address these challenges, with their actual responses available in Section 6.2 of the Supplementary Material.

• Modularization: Restructure the system to import only the necessary components (questionnaire, data upload, state management) rather than working with a monolithic package.
• Improving UI: Adopt a node-based or table-driven GUI with color-coded elements for clearer visualization of state transitions.
• Operation Streamlining: Add copy-and-paste functionality for state settings and streamline controls to reduce excessive clicking (e.g., [P4]’s suggestion for a Gantt chart style layout).
• Enhanced Tutorials and Documentation: Improve documentation with video tutorials, sample scenes, and detailed integration guides to reduce barriers to entry and facilitate user learning.

5.4 Discussion

Researchers highly valued LUIDA’s key features: automated recruitment, parallel execution, and streamlined data or questionnaire management, for their contributions to reducing workload, and increasing accessibility. The system was specifically praised for bridging the gap between urban and rural research institutions and removing barriers related to participants’ nationality or cultural background. While concerns regarding environmental control and platform-specific demographics were raised, the overall benefits outweighed these challenges, highlighting advantages over traditional laboratory experiments. This finding is supported by the high usability and moderate workload ratings for the overall LUIDA experience. To address these limitations, interview responses suggested expanding support to non-Cluster users and enhancing environmental control features.

In contrast, the Implementation Template scored poorly on usability and high on workload, even though the researchers valued its functions. The interviews point to four concrete causes (Section 5.3): (1) its monolithic structure, which forces users to face all components at once; (2) the conceptual load of the finite-state-machine model used for experiment flow; (3) repeated switching between no-code panels and hand-written code, which breaks the workflow; and (4) the assumed background in Unity and the Cluster Creator Kit. Together, these show that a rich set of automation features does not by itself lower the cost of learning. We expect part of this gap to close through better onboarding (video tutorials, sample scenes, and guided practice), and part through the redesign described in Section 7.4.

Besides, this user study asked the researchers to implement pre-existing experimental designs, which also serves as a limitation. We encourage future usability evaluation of LUIDA when being applied to entirely novel experiments.

In summary, our findings show that LUIDA offers significant benefits through automated recruitment, parallel experiment execution, and efficient questionnaire and data management. These features increase accessibility and reduce barriers in VR research, addressing RQ1: “What benefits does this framework offer researchers in conducting experiments?”. In addition, the successful implementation of all three experimental designs by researchers suggests that LUIDA can accommodate a wide variety of VR experiments (RQ2). Although challenges remain in environmental control, platform dependency, and the complexity of the Implementation Template, we address each through the planned improvements described above.

6 Study 2: large-scale replications

To evaluate how effectively LUIDA can promote experiment participation (RQ3) and the validity of LUIDA to host experiments (RQ4), we implemented and uploaded the three experiments described in Section 4 using LUIDA. These experiments were made publicly available for execution with participants outside the lab, and then we examined whether the data collected were consistent with the original experiments from which they were replicated.

6.1 Experiment procedure

To recruit participants, we made LUIDA publicly available on the Cluster for 1 week, inviting users to the Recruitment World through Cluster’s official events and social media. The recruitment information on the Recruitment World bulletin board stated that participation was open to adults aged 18 years or older, so minors were excluded by design and no parental-guardian consent procedure applied.

As part of the consent step of the participant workflow (Figure 6; Section 3.8), participants reviewed and agreed to an in-VR consent form upon arrival in the Experiment World, before beginning the experimental tasks. The form described the study’s purpose and procedure, the prerequisites to join the experiment (including the age requirement of 18 years or older), the data collected during the experiment, the compensation offered, the participant’s right to withdraw at any time without penalty, and the risk of VR sickness together with how to stop if it occurred. Its full text is provided in Supplementary Section 4 (Consent Form). In addition, before participating, each participant used a VR headset and completed a short questionnaire about their age, gender, and VR experience.

After the 1-week public period, we closed access to LUIDA and offered participants a 2,000 JPY (approximately 13 USD) Amazon gift card for each completed experiment (up to 6,000 JPY (approximately 40 USD) for participation in all three experiments). The results of each experiment were analyzed using the same analytical approaches used in the original studies to verify the degree of alignment.

6.2 Criteria for successful replication

We specified in advance what would count as a successful replication to assess LUIDA’s experimental validity (RQ4). Because the three original studies report different types of outcomes (perceptual thresholds, group-level behavioral differences, and regression model fits), we tailored the criterion to each experiment, as listed below and made concrete by the per-experiment hypotheses stated in the corresponding subsections:

• Hand redirection (perception): since the original study did not clarify their results’ confidence intervals (CIs), we treat the replication as successful if the detection thresholds reported in the original study fall within the 95% CIs obtained on LUIDA, formalized as HR-H1 (Faster threshold) and HR-H2 (Slower threshold).
• Proteus effect (cognition): the replication is successful if the group-level effects reproduce the direction and statistical significance reported in the original study, namely, no significant difference from baseline for the FL avatar (PE-H1), a significant increase from baseline for the CD avatar (PE-H2), and a significantly higher value for the CD than the FL avatar (PE-H3); each is evaluated for both d95 and total displacement (subscripts a/b).
• 3D Fitts’ Law (user interfaces): as the original study reports regression model fits rather than a single effect, the replication is successful if our data are explained by the same models at a level at least comparable to the original, namely, an equal or greater number of significant predictors (FL-H1), an equivalent or higher model fit (Adj. R 2; FL-H2), and an equivalent or lower standard error of the regression (S E; FL-H3).

Across the three experiments we adopted the same analysis method as each original study rather than a more elaborate alternative, for example, a direct Group × Condition interaction test for the Proteus effect or a participant-level mixed-effects or cluster-robust regression for 3D Fitts’ Law. Adopting the more elaborate alternatives may yield more refined estimates of each effect in the replicated experiments, but this is not the purpose of Study 2. Keeping the analysis identical to the original makes any divergence between our results and theirs attributable to the platform and participant pool rather than to a change in method, and preserving an original study’s procedure in this way is a recognized practice in replication research (Brandt et al., 2014). The results in Study 2 should therefore be interpreted as tests of platform equivalence rather than as refined estimates of each effect, and we give the analysis-specific cautions in the corresponding experiment sections.

6.3 Hand redirection experiment

6.3.1 Participants

A total of 185 participants completed the hand redirection experiment. The VR devices they used are listed in Table 3. After excluding 22 non-compliant participants (11.89% of the whole dataset) who consistently selected only ”faster” or ”slower” in all trials, valid data were obtained from 163 participants (106 males, 45 females, 12 others; mean age = 32.012 ± 9.187).

TABLE 3. VR devices used in replicated experiments. Oculus/Meta devices are grouped. Devices not explicitly noted as Standalone are PCVR devices used via SteamVR.

6.3.2 Results

Table 4 lists the detection thresholds from both the original study and our replicated experiment. Except for PSE, both the Faster and Slower Thresholds from the original study fell within the confidence intervals of our results. This finding supports both HR-H1 and HR-H2, suggesting that experiments conducted on LUIDA demonstrate sufficient validity in the field of perception research.

TABLE 4. Detection threshold results in gain-based hand redirection.

6.4 Proteus Effect experiment

Unlike the original study, which tracked multiple body parts, Cluster by default tracks only the HMD and controllers; although it supports additional trackers such as HTC Vive trackers ¹⁶ , we chose not to require them in order to lower the participation burden, which limited our analysis to the positions of the head and hands. Because of this constraint, in addition to d95, the performativity index used in the original study, we also analyze total displacement (the summed displacement of the head and hands over the task) as a supplementary metric. Both metrics, together with the corresponding hypotheses, are summarized in Table 2.

6.4.1 Participants

A total of 174 participants joined the Proteus Effect experiment, and the VR devices they used are listed in Table 3. Participants were randomly assigned to one of two experimental groups: the Formal Light-skinned (FL) group or the Casual Dark-skinned (CD) group. Each participant completed tasks first in a baseline condition and then in a condition where they embodied their assigned avatar, as described in Section 4.3. Among the participants, 16 dropped out (9.20% of the whole dataset) during the transition from the baseline world (without avatar) to the avatar displayed world. This resulted in valid data from 158 participants (102 males, 45 females, 11 others; mean age = 32.665 ± 9.331).

6.4.2 Results

A Shapiro-Wilk test indicated that the d95 data were not normally distributed ( p < 0.05 ). We conducted Wilcoxon signed-rank tests to compare the baseline and embodied conditions within the CD and FL groups. To compare the groups themselves, we applied Wilcoxon rank-sum tests to both their baseline and embodied conditions.

The results are presented in Table 5. In the FL group, there was no significant difference between the baseline and embodied conditions, supporting PE-H1a. The CD group showed a significantly higher d95 in the embodied condition than in the baseline condition, supporting PE-H2a. However, the absence of significant difference between the embodied conditions of the FL and CD groups fails to support PE-H3a.

TABLE 5. Analysis results for d95 in the baseline and avatar conditions of the CD and FL groups.

Similarly, the Shapiro-Wilk test showed that the total displacement metric was not normally distributed ( p < 0.05 ), so we applied the same set of non-parametric analyses. As shown in Table 6, the results mirrored the d95 findings. The FL group showed no significant difference between the baseline and embodied conditions (PE-H1b supported), while the CD group exhibited significantly higher total displacement in the embodied condition than in the baseline condition (PE-H2b supported). Again, the comparison between the embodied conditions of the FL and CD groups revealed no significant difference, failing to support PE-H3b.

TABLE 6. Analysis results for total displacement in the baseline and avatar conditions of the CD and FL groups (r: Cohen’s r).

In summary, the analyses of both d95 and total displacement yielded consistent results. These findings suggest that our experiment on the LUIDA platform partially replicated the Proteus effect observed by Kilteni et al. (Kilteni et al., 2013): the within-group patterns reproduced, but the between-group contrast (PE-H3) did not. We therefore interpret this as encouraging but incomplete evidence that LUIDA can maintain experimental validity in avatar-embodiment and related cognitive-science research as studied in the laboratory. We examine the divergence below (Section 6.4.3).

6.4.3 Discussion

As noted in Section 6.2, we analyzed this experiment with the original study’s separate within-group and between-group comparisons rather than a direct Group × Condition interaction, so the result is best interpreted as a test of whether moving the experiment to LUIDA leaves the outcome unchanged. Accordingly, the one criterion we did not reproduce was the between-group contrast (PE-H3). The CD avatar did not yield significantly greater displacement than the FL avatar, even though the within-group increase for the CD avatar did replicate.

One plausible explanation is cultural. The Proteus effect operates through stereotypes associated with an avatar’s appearance, and the strength and content of those stereotypes depend on participants’ cultural background. Our pool was recruited on a predominantly Japanese metaverse platform, whereas the original study drew on a Western sample, so the social meanings attached to the FL and CD avatars, and therefore the expected size of the between-group difference, may differ from the original context.

Importantly, because this manipulation uses skin tone and clothing as social-stereotype cues, it warrants interpretive and ethical caution. We do not treat the avatar categories as natural or essential attributes; they are stimulus conditions inherited from the replicated study, and our purpose was platform comparison rather than any claim about the groups the avatars evoke. Because participants briefly embodied these avatars, a structured debriefing would let us explain the purpose and confirm participants were comfortable, which we discuss as a limitation in Section 8.2.

6.5 3D fitts’ law experiment

6.5.1 Participants

A total of 197 participants (131 males, 54 females, 12 others; mean age = 32.352 ± 9.423) completed the experiment. None of the participants dropped out or were noncompliant with the task instructions. The VR devices they used are listed in Table 3.

6.5.2 Results

Following Clark et al. (2020), we excluded trials with movement times greater than 1.5 IQR above the upper quartile as outliers. Linear regression analyses were then performed, guided by Saffo et al. (2021), to model movement time using variables from the existing models listed in Clark et al. ’s study (see Section 4.4).

The results in Table 7 show that all spatial variables in each model, including Index of Difficulty, depth, inclination angle, azimuth angle, and target size × inclination angle, significantly predicted movement time, supporting FL-H1. As noted in Section 6.2, we fitted these as conventional pooled regressions, following the original study rather than participant-level mixed-effects models or cluster-robust standard errors. Because each participant contributed roughly 100 trials, treating trials as independent can understate the standard errors and overstate the significance of the coefficients, so the individual p-values in Table 7 should be read with that qualification. For FL-H1, however, because we compared our count of significant predictors against Clark et al. , who used the same pooled regression, this overstatement of significance is present in both analyses and largely cancels, leaving FL-H1 valid as a relative rather than an absolute claim.

TABLE 7. Models evaluated by Clark et al. (2020) and our replication (I D: Index of Difficulty, θ: inclination angle, φ: azimuth angle).

Meanwhile, all models explained a moderate portion of the variance, but Adj. R 2 ( 37.3 ∼ 38.8 % ) was lower than Clark et al. ’s work ( 50.1 ∼ 64.5 % ), failing to support FL-H2. In contrast, SE ( 285.779 ∼ 289.336 m s ) was lower than Clark et al. ’s work ( 364.162 ∼ 431.963 m s ), supporting FL-H3.

In addition, Saffo et al. also conducted a similar replication of Clark et al. on VRChat but under experimenter supervision and participants embodying an avatar during the experiment. A post hoc comparison with Saffo et al. ’s work shows that our work identified more significant variables but a worse model fit than theirs (lower Adj. R 2 and higher SE; see Table 7).

6.5.3 Discussion

The support for FL-H1 and FL-H3 suggests that our platform collects high-fidelity data: the lower S E implies less measurement noise, and the larger number of significant variables indicates higher sensitivity to subtle behavioral effects. The lower Adj. R 2, however, means that the established models explain less of the variance in our data. Three factors likely account for this difference in model fit.

First, hardware. Clark et al. used the Oculus Rift CV1 and Saffo et al. used headsets of a similar generation (for example, HTC Vive, Valve Index, and Oculus Rift S), whereas our participants mostly used newer headsets (Meta Quest 2 and 3) with higher resolution and different display technology. This newer and more varied hardware plausibly adds performance variance that the original models do not capture. Second, participants. Our large remote pool ( N = 197 ), who used their own equipment, is inherently more varied than a controlled lab sample (Clark et al., 2020) or a supervised one (Saffo et al., 2020). Third, avatars. Saffo et al. noted that their embodied avatars may have added depth cues, which could lower their S E relative to ours.

We therefore read the lower fit not as a failure of the platform but as a finding for modern VR research: models built on older, uniform hardware do not fully capture the performance variance of today’s diverse devices and users. The post hoc comparison with Saffo et al. also points to practical improvements for LUIDA, such as an optional supervision feature, and either restricting device types before participation or grouping data by device during analysis to manage hardware heterogeneity.

7 Discussion

Table 8 summarizes how each research question was tested, with its main metric, the section that reports it, and the outcome.

TABLE 8. How each research question is tested, with the main metric and result.

7.1 Overall practical benefits of LUIDA

Our practical findings position LUIDA as a unified and structured methodology for metaverse-based research and an efficient alternative to traditional lab work. By more effectively leveraging the inherent functionalities of the metaverse than prior attempts, LUIDA automates and parallelizes both participant recruitment and experiment execution. These core benefits address critical challenges in VR studies, such as scalability, participant diversity, and the need for technical expertise or efforts, while largely preserving experimental validity across the three replications (full in perception, partial in cognition and interfaces).

While its ability to remotely recruit large numbers of participants is comparable to traditional crowdsourcing, LUIDA shows lower data exclusion rates and better access to the target population of actual metaverse users, yielding findings that more closely reflect the true ecosystem. Crowdsourcing studies often have to discard a large share of their data; for example, one study found careless behavior in 45.9% of participants (Brühlmann et al., 2020), and a review reported that some studies discarded 16%–49% of responses as invalid or incomplete (Pickering and Blaszczynski, 2021). By contrast, our three replications excluded only 11.89%, 9.20%, and 0% of the data (Section 6). We attribute this in part to the requirement to wear a VR device, which raises the barrier to entry and filters out inattentive users, so the data quality on LUIDA is likely to hold even as the metaverse user base grows.

7.2 Beyond infrastructure: methodological implications

The contributions above are primarily practical, but our results also carry methodological implications for how VR behavioral science is conducted. First, large-scale metaverse replication is not merely confirmatory. Reproducing established 3D Fitts’ Law models on a far more heterogeneous device-and-participant pool than the original laboratory studies surfaced a finding the original work could not: models fitted on older, uniform hardware leave more variance unexplained on today’s diverse headsets, even as measurement error (SE) decreases (Section 6.5.3). “In-the-wild” replication can thus act as a stress test of a model’s external validity, turning the DEAR “Reproduce” phase into a source of new knowledge rather than a confirmatory checkbox (Section 7.6). Second, the consistently low data-exclusion rates (11.89%, 9.20%, and 0%; Section 7.1) suggest that requiring a VR headset functions as an implicit quality filter, with direct consequences for the statistical power and sampling assumptions of online VR studies. Third, and more speculatively, lowering the cost of recruiting hundreds of embodied participants may shift which questions are routinely asked: as crowdsourcing platforms originally built only to recruit workers came to support a broad experimental literature, a tool like LUIDA may make large-scale, naturalistic metaverse studies a standard method and reshape norms around effect sizes, power, and ecological validity (Section 8.4). We therefore locate LUIDA’s scientific value not in any single replicated effect, but in enabling a class of studies whose scale and ecological setting were previously impractical.

7.3 Compensation for data loss and experimental control

Our evaluation also identified areas for improvement in usability and flexibility to address more experimental needs. One key challenge lies in data collection, particularly data loss caused by the lack of participant monitoring or support, which risks compromising experimental validity. To address this, we plan to introduce system-level restrictions to mitigate common misunderstandings of experimental procedures (e.g., direct redirection instead of using world portals during world transitions). It would also be beneficial to leverage methods from traditional crowdsourcing studies, such as developing a VR-adapted Instructional Manipulation Check (Oppenheimer et al., 2009) to detect noncompliance. Additionally, based on user feedback, we aim to implement features that help researchers support participants, such as in-VR Q&A forms, recording functionality, or allowing researchers to join ongoing sessions for monitoring.

Another risk is that if participants log off the platform, either deliberately or accidentally due to network problems, LUIDA currently does not natively support either automatic detection of lost data or session recovery upon their return, resulting in the loss of experimental state. As a temporary alternative, researchers can manually verify whether a participant completed the session by checking the data log, as all collected data is timestamped. To fundamentally address this issue, while metaverse platforms like Cluster provide APIs for saving and loading world states, which could be used by researchers within LUIDA’s Implementation Template’s customizable code blocks, a native, user-friendly data recovery and session resume feature should be included in future development. This would allow researchers to more easily manage incomplete sessions without requiring custom scripting.

There are other aspects to consider in order to improve experimental control. First, to prevent overly recruiting participants, experiments should automatically become invisible in the Recruitment World once participant quotas are reached. Second, to ensure balanced sampling, the Experiment World could selectively reject participation based on self-reported demographics or detected hardware specifications. Finally, an automated testing feature would allow researchers to verify system functionality and compatibility within LUIDA.

7.4 Refactoring for a more lucid implementation template

The Implementation Template’s low usability and high workload scores, together with the interview feedback, point to several concrete problems and matching fixes, which we summarize in Table 9. Responding to this feedback, we have begun implementing these improvements, which together aim to raise usability and lower workload.

TABLE 9. Usability problems of the Implementation Template found in Study 1 and their corresponding fixes, all currently under way in response to user feedback.

7.5 Leveraging metaverse user community

The metaverse user community also played an unexpected role in promoting our experiments: participants shared selfies from the Recruitment World on social media and introduced LUIDA to friends within Cluster. Many noted how easy it was to take part, which further supports RQ3, and this word-of-mouth promotion from people they know may have encouraged more committed participation and thus helped lower the share of invalid data. Such casual sharing can also reveal experimental conditions and threaten validity, although we found no such disclosure during the week our experiments were public. To keep this benefit while limiting the risk, the Recruitment World could offer a designated photo spot that encourages sharing, together with clear signs stating which information may or may not be disclosed.

7.6 LUIDA’s coverage of the experimentation lifecycle

We finally interpret LUIDA through the lifecycle view introduced in Section 2.2.2, in which Grübel characterizes a third generation of experimentation frameworks to cover the lifecycle of the DEAR principle: Design, Experiment, Analyse, Reproduce) (Grübel, 2023). Although LUIDA does not cover the whole lifecycle evenly, here we read our results through this lens to clarify both where its value lies and where it should grow.

LUIDA’s clearest contribution is to DEAR’s “Experiment” phase, which is the phase least served by second-generation, Unity-project-level toolkits. This is also where our results are strongest, where the large-scale replications collected more than 500 datasets within a week, with exclusion rates well below those typical of crowdsourcing (Section 6). In addition, our 3D Fitts’ Law replication, where reproducing an established model on modern, varied hardware lowered the model fit yet reduced measurement error (Section 6.5.3), shows that DEAR’s “Reproduce” phase can produce new findings rather than simple confirmation. In contrast, DEAR’s “Design” phase, supported by the Implementation Template, received the weakest evaluation, namely, a low usability score for the template (Section 5). This suggests that LUIDA’s present value comes mainly from automating execution rather than from authoring, which is consistent with what the DEAR view predicts for the third generation.

This lens also explains the gaps left open in Table 1. LUIDA only partially covers the analysis and monitoring components, and it does not yet support session replay, leaving a gap in the DEAR “Analysis” phase that falls outside LUIDA’s current scope. That said, this scope can be extended. For instance, although replay is not currently supported, LUIDA already records frame-level tracking data, meaning sessions could in principle be reconstructed from the logs. This points toward a potential replay feature in future work, bringing LUIDA closer to fully covering the “DEAR Analysis” phase.

Finally, we map our planned work onto the DEAR phases. The Implementation Template redesign (Section 7.4) targets “Design”; recording/replay, supervision, and session resume (Section 7.3) target “Analyse”; and repository-backed data pipelines with automatic preregistration would target “Reproduce”, closing a reproducibility loop that LUIDA does not yet close. As we carry out the planned work above, we argue that LUIDA has the potential to be a pioneering metaverse-based system aligned with the DEAR lifecycle, advancing in particular its least-served Experiment phase. Similarly, ExaC and SimpleExaCT also argued the same potential through researcher-owned, self-hosted code (Aguilar et al., 2024), which represents a different design choice from LUIDA, as LUIDA provides managed lifecycle automation on a commercial metaverse platform. Both rely on external services, but LUIDA shifts more of the work onto the platform, trading some control for accessibility and a built-in VR user pool.

7.7 Toward a consolidating framework, not another tool

Proposing a new framework like LUIDA risks becoming yet another entry in a fast-growing list of VR experimentation frameworks, such as Landmarks (Starrett et al., 2021), OpenMaze (Alsbury-Nealy et al., 2022), CReST (Cools et al., 2023), StudyFramework (Ehret et al., 2024), and the authoring tool of Helgert et al. (2025), as warned by the DEAR principle. Each framework or tool adds value, but together they make tools harder to choose, learn, and maintain, and may cause researchers to build their own ones. We take this concern seriously and aim not to add one more isolated tool, but to work toward a metaverse-native platform that others can build on. Because LUIDA’s Web Console, Storage, and template model are loosely coupled and largely platform-agnostic (Section 3.7), they could, like Experiments as Code (Aguilar et al., 2024), host or work with authoring tools from other frameworks instead of replacing them.

LUIDA is not yet such a platform, and getting there would require several efforts, including the platform-abstraction layer of Section 3.7 so one experiment definition can run on several metaverse platforms; adapters that import the experiment specifications of existing toolkits; and open, shared formats for experiments and data so that tools combine rather than compete. We see this consolidation, rather than another standalone tool, as the most useful long-term contribution LUIDA can make.

8 Limitations and future work

8.1 Sampling and validity

Our findings carry three limits of sampling and validity. First, Study 1 evaluated only six researchers (Section 5), a small and homogeneous group (graduate researchers, mean age 23.83, recruited from the authors’ network for their VR-experiment experience and a matching research domain). The ratings therefore might not automatically apply to a broader population of researchers. Beyond this study, however, LUIDA has begun to be utilized by other researchers to implement and execute entirely novel experiments Tokida et al. (2026). This provides initial evidence that LUIDA supports original experimental designs in practice. We suggest that future work continue to invite a larger, more diverse group of researchers, including senior researchers and those with less technical expertise, to build their original experiments on LUIDA, alongside formal usability evaluation.

Second, our validity claims rest on the pre-specified, per-domain replication criteria defined in Section 6.2 rather than on formal equivalence testing. In each case we adopted the strongest criterion that the available reference data allowed: for hand redirection, we tested whether the original thresholds fell within the 95% confidence intervals obtained on LUIDA; for the Proteus effect, we required the direction and statistical significance of the group-level effects to match the original study; and for 3D Fitts’ Law, we compared the number of significant predictors, model fit, and regression error against the originally reported values. Formal procedures such as two one-sided tests (TOST) would offer a stronger guarantee of equivalence, but they require an a priori equivalence margin derived from the original studies’ variability (SD or CI), which several of the replicated studies do not report. Where such variability is available, or can be obtained from the original authors or from pilot data, applying TOST or effect-size-based equivalence is a natural next step for confirming, rather than merely corroborating, large-scale metaverse replications.

Third, the prototype runs on a single metaverse platform, Cluster. Despite the broader geographic reach this affords, it may introduce other biases: participants are limited to those with the socioeconomic status to afford VR devices, those from the Japanese population (as Cluster’s main market is currently Japan), and those familiar with Cluster’s culture, norms, and features. Our findings of usability, replication, recruitment rate, and low-data-loss results may therefore depend in part on factors specific to this population. For example, promotion via communities may depend on the culture and norms of Cluster’s user community; although community-building features such as adding friends and holding gatherings in shared worlds are common across metaverse platforms, the same effects cannot be assumed within other platforms’ communities, and applying these findings there warrants caution and further validation. Furthermore, future controlled comparisons with laboratory or crowdsourcing samples might help clarify which findings are specific to the metaverse user pool and which generalize to broader, more diverse populations.

8.2 Ethical oversight

This study has several limitations in ethical oversight. First, Study 2 restricted participation to adults aged 18 years or older, a criterion stated both on the Recruitment World bulletin board and in the consent form, and every self-reported age collected in the pre-experiment questionnaire fell within the adult range. Because minors were excluded by design, no parental-guardian consent procedure applied. However, we had no means, in either LUIDA or the platform, to verify age beyond this self-report, so eligibility ultimately rests on it. Cluster’s terms of service require blanket parental or guardian consent for a minor to use the platform itself, but we do not treat that platform-level consent as a substitute for study eligibility or for research consent. Reliance on self-reported age is a limitation inherent to unsupervised recruitment on public metaverse platforms, and future deployments should pair clearer eligibility screening with platform-level age-verification safeguards.

A further limitation concerns debriefing. Study 2 included no formal post-session debriefing. Participation was based on informed consent, the task and procedure were disclosed in advance, and no deceptive cover story was used, so participants were not misled about what they did. We nonetheless recognize that a structured debriefing is good practice, especially for the Proteus effect replication, whose manipulation uses skin tone and clothing as social-stereotype cues, and we will incorporate one in future deployments.

Moreover, our consent process addressed data handling at only one of two layers. LUIDA collects data for the study, such as headset and controller motion, questionnaire responses, and experiment-specific measures. These research-layer data are governed by the in-VR research consent form and are stored and anonymized as described in Section 3.3. The Cluster platform also captures data as part of its normal operation, and these platform-layer data are disclosed in, and governed by, Cluster’s privacy policy, which all users accept independently of this research. The research consent form did not separately restate the platform’s own data handling, so participants relied on Cluster’s privacy policy for that layer. We make this two-layer distinction explicit here, and in future deployments we will surface it to participants at the point of consent.

Finally, as detailed in the Ethics Statement, the studies were reviewed by an ethics committee internal to Cluster Metaverse Lab, the organization that develops the platform under study. The committee is nationally registered, includes members external to the organization, and excludes a study’s own researchers from its review, and no author is a committee member. Nonetheless, we cannot confirm whether external members reviewed this specific study, and an internal review remains a limitation relative to a fully independent, third-party review.

8.3 System and operation

Several further limitations concern how the prototype runs and how thoroughly we could evaluate it; for each, we note its impact, where it is discussed in this paper, feasible mitigation or improvement plans. First, because sessions are unsupervised, a few participants dropped out or misunderstood the procedure; continuous tracking let us detect and exclude these cases to protect validity (Section 6), and we discuss further support features in Section 7.3. Next, as a networked platform, LUIDA can introduce timing variability from network latency. This variability did not affect our replicated experiments, but it matters for studies that need millisecond precision; there, timestamped logs partially compensate for it, letting researchers identify and filter delayed data.

Avatar switching is a further prototype limitation. As mentioned in Section 3.4.4, switching a participant’s avatar during an experiment session currently requires registering multiple worlds and redirecting participants between them, which adds transitions that can interrupt the session. As future work, we plan to import humanoid models directly into the Experiment World and drive them with built-in inverse-kinematics scripts, so that avatars can be assigned or switched within a single world without redirection.

LUIDA inherits a limitation common to remote experimentation frameworks: running studies remotely and at scale rules out most external hardware. Although many metaverse platforms support communication with external devices through designated protocols (e.g., OSC for Cluster and VRChat), remote participants use their own consumer headsets and have no access to the additional devices a study might require, such as haptic feedback devices or physiological sensors (e.g., EEG or heart rate monitors); supporting such hardware in fully remote studies is outside our current scope. We emphasize, however, that LUIDA is not confined to the large-scale, public-metaverse setting. Its Web Console and Storage are ordinary web and cloud services that couple to the worlds only through a data interface (Section 3.7), so the same experiment definition can also be deployed on-site in a controlled setting, where an experimenter can attach the designated hardware and supervise participants directly. Such use forgoes the large-scale, asynchronous recruitment that motivates LUIDA but retains its other capabilities, including no-code experiment authoring, automated experiment flow, in-VR questionnaires, avatar switching, and continuous data logging. In this sense LUIDA spans the same DEAR lifecycle (Section 7.6) as code-based frameworks such as ExaC, complementing rather than replacing controlled laboratory workflows.

Finally, two limitations remain regarding the evaluation. The Implementation Template scored low on usability (SUS = 44.17, between ‘Poor’ and ‘OK’), and we describe ongoing improvement tasks to improve its usability in Section 7.4. Our evaluation also lacks a direct comparison of the effort of building the same study with and without LUIDA, which we leave to a future controlled comparison.

8.4 Future directions

Now that LUIDA is public, collecting feedback from field users at a larger scale is the immediate next step. On the system side, our roadmap follows the lifecycle view in Section 7.6: redesigning the Implementation Template (Section 7.4) and adding recording, supervision, and session resume (Section 7.3). Beyond these, a direction worth pursuing is to quantify LUIDA’s efficiency benefits directly rather than infer them, through a controlled comparison that measures the development effort and setup time needed to build the same experiment with and without LUIDA, against conventional VR experiment workflows.

In light of the community effects observed in Section 7.5, another direction is to leverage the metaverse user community more actively. For example, the Recruitment World could be strengthened as a venue in its own right by incorporating “Third Place” design elements (Ducheneaut et al., 2004) that foster an active, research-interested participant community and help bridge research with society.

On the scientific side, this paper is primarily a practical validation, showing that such a framework can reliably prepare and host experiments and efficiently recruit participants. Looking ahead, a broader question for future work could be how a tool like LUIDA changes research practice itself. A useful parallel is crowdsourcing platforms, which were built simply to recruit workers for small tasks yet came to support experiments across fields such as psychology and sociology. LUIDA may follow a similar path, making large-scale, “in the wild” metaverse studies a common method, and raising methodological questions like how such studies affect effect sizes, statistical power, and ecological validity.

9 Conclusion

This paper introduced LUIDA, a novel framework as a structured methodology for VR/HCI experiments by leveraging advantages of commercial metaverse platforms. Its core design unifies participant recruitment, execution, and data collection into a single, streamlined workflow.

A prototype developed on the metaverse platform Cluster was validated through two studies. A usability evaluation with VR researchers confirmed LUIDA’s ability to streamline experiment setup across multiple domains, while three large-scale replicated experiments demonstrated its capacity to rapidly recruit hundreds of participants and reproduce findings from original lab studies. These results indicate that LUIDA largely maintains experimental validity (full in perception, partial in cognition and interfaces) while offering substantial advantages in scale and automation for VR research. Eventually, we outline future work of LUIDA: refining the system (Implementation Template usability, data recovery, and supervision), strengthening the Recruitment World to foster participant communities, and studying how large-scale metaverse experimentation reshapes research practice.

Generative AI statement

The author(s) declared that generative AI was used in the creation of this manuscript. AI tools were employed solely for English editing purposes, including phrasing refinement and grammar correction. No AI tools were involved in data analysis, interpretation, or scientific content generation.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2026.1847904/full#supplementary-material

Supplementary material Supplementary material at the source

Footnotes

Edited by: Giancarlo Fortino, University of Calabria, Italy

Reviewed by: Jascha Grübel, Central European University, Austria

Ro’Ifah Ro’Ifah, Universitas Muhammadiyah Surabaya, Indonesia

Footnotes

https://www.mturk.com/

https://hello.vrchat.com/

https://unity.com/. Unity is a trademark or registered trademark of Unity Technologies or its affiliates in the U.S. and elsewhere. This work is not sponsored by or affiliated with Unity Technologies.

https://www.unrealengine.com/

https://ccrma.stanford.edu/groups/osc/index.html

https://vercel.com/

https://www.mongodb.com/products/platform/atlas-database

https://aws.amazon.com/s3/

https://docs.cluster.mu/creatorkit/en/

https://cluster.mu/

https://recroom.com/

https://secondlife.com/

https://store.steampowered.com/steamvr

https://creators.vrchat.com/worlds/udon/

https://creators.vrchat.com/worlds/udon/external-urls/

https://www.vive.com/accessory/tracker3/

https://cluster.mu/w/29ac4e39-e7af-4467-b3ba-dea011473eb0

https://luida.cluster.mu

https://github.com/cluster-lab/project-luida-bar