Towards a Better Evaluation of 3D CVML Algorithms: Immersive Debugging of a Localization Model

Simple item page
dc.contributor.authorLin, Ticaen_US
dc.contributor.authorYuan, Junen_US
dc.contributor.authorMiao, Kevinen_US
dc.contributor.authorKatolikyan, Tigranen_US
dc.contributor.authorWalker, Isaacen_US
dc.contributor.authorCavallo, Marcoen_US
dc.contributor.editorAigner, Wolfgangen_US
dc.contributor.editorAndrienko, Nataliaen_US
dc.contributor.editorWang, Beien_US
dc.date.accessioned2025-05-26T06:37:00Z
dc.date.available2025-05-26T06:37:00Z
dc.date.issued2025
dc.description.abstractAs advancements in robotics, autonomous driving, and spatial computing continue to unfold, a growing number of Computer Vision and Machine Learning (CVML) algorithms are incorporating three-dimensional data into their frameworks. Debugging these 3D CVML models often requires going beyond traditional performance evaluation methods, necessitating a deeper understanding of an algorithm's behavior within its spatio-temporal context. However, the lack of appropriate visualization tools presents a significant obstacle to effectively exploring 3D data and spatial features in relation to key performance indicators (KPIs). To address this challenge, we explore the application of Immersive Analytics (IA) methodologies to enhance the debugging process of 3D CVML models. Through in-depth interviews with eight CVML engineers, we identify common tasks and challenges faced during the development of spatial algorithms, and establish a set of design principles for creating tools tailored to spatial model evaluation. Building on these insights, we propose a novel immersive analytics system for debugging an indoor localization algorithm. The system is built using web technologies and integrates WebXR to enable fluid transitions across the reality-virtuality continuum. We conduct a qualitative study with six CVML engineers using our system on Apple Vision Pro, observing their analytical workflow as they debug an indoor localization sequence. We discuss the advantages of employing immersive analytics in the model evaluation workflow, emphasizing the role of seamlessly integrating 2D and 3D visualizations across varying levels of immersion to facilitate more effective model assessment. Finally, we reflect on the implementation trade-offs and discuss the generalizability of our findings for future efforts in immersive 3D CVML model debugging.en_US
dc.description.sectionheadersEvaluation and Guidance
dc.description.seriesinformationComputer Graphics Forum
dc.identifier.doi10.1111/cgf.70111
dc.identifier.issn1467-8659
dc.identifier.pages12 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.70111
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf70111
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Human-centered computing → Mixed / augmented reality; Virtual reality; Visual analytics; Computing methodologies → Computer vision; Machine learning
dc.subjectHuman centered computing → Mixed / augmented reality
dc.subjectVirtual reality
dc.subjectVisual analytics
dc.subjectComputing methodologies → Computer vision
dc.subjectMachine learning
dc.titleTowards a Better Evaluation of 3D CVML Algorithms: Immersive Debugging of a Localization Modelen_US
Files
Original bundle
Now showing 1 - 2 of 2
Thumbnail Image
Name:
cgf70111.pdf
Size:
11.5 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
1061-file-i8.mp4
Size:
265.74 MB
Format:
Video MP4
Download
Collections
44-Issue 3