Daedalus-XR: Valutazione di un Workflow Agentico Scene-Aware per l'Authoring Immersivo tramite Large Language Model Locali

Authoring 3D virtual environments in contemporary game engines requires nav- igating complex desktop interfaces, causing workflow fragmentation between the in-editor hierarchy and web browsers for asset retrieval. In Virtual Reality (VR) development, this friction is further aggravated by continuous context switching between desktop monitors and the Head-Mounted Display (HMD). While emerging Multimodal Large Language Models (MLLMs) open new possibilities for natural language scene authoring, several recent systems rely on runtime dynamic code gen- eration (such as Roslyn-compiled C# scripts), introducing unpredictable compilation crashes, or cloud APIs with high latency, costs, and data privacy risks. This work presents the design, software engineering, and evaluation of Daedalus-XR, a local, scene-aware agentic framework for immersive 3D authoring in Virtual Reality. The system executes its complete AI pipeline on a consumer-grade personal computer with a local AI core, querying external network APIs exclusively for optional online multimedia asset retrieval. The architecture integrates streaming speech recognition (Whisper), neural text-to-speech synthesis (Piper running via Unity Inference Engine), and multimodal agentic orchestration (Qwen 3.5 4B via Ollama). Instead of arbitrary runtime code, the agent outputs schema-constrained JSON tool calls mapped to a pre-compiled, deterministic execution space across four unified families: Object- Level Editing, Multimedia Retrieval (Sketchfab 3D models and Freesound spatial audio), Shader-Level Editing, and Visual Assistance. Continuous awareness of the virtual scene graph enables a stateless multi-turn interaction cycle with predictable low latencies (median voice-to-action latency of 3.87 seconds) and full bidirectional persistence with the professional desktop Unity Editor. The system was validated through a within-subject Latin-square user study (N = 30 Unity users across Novice, Advanced, and Expert tiers) against the standard desktop Unity Editor across three matched authoring tasks (T1–T3) and a multimodal assistance task (T4). Results show an overall 40.4% reduction in total authoring time (T1–T3), a 48.4% reduction in task-completion time on fragmented tasks (reaching 67.4% on atmosphere and spatial audio), high usability (SUS score of 77.42), and lower frustration and temporal demand (not significant after correction), preserving creative agency for 89.7% of creators. Exploratory analyses show pronounced descriptive workflow acceleration for novice and advanced users in asset retrieval, without reliable moderation surviving multiple-comparison corrections. Combining scene- aware agentic orchestration with deterministic in-engine execution thus provides a viable, responsive foundation for immersive spatial authoring.

È presente un sunto esteso in italiano nel documento.