Necessary but not Sufficient: Limitations of Projection Quality Metrics
Loading...
Date
2025
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association and John Wiley & Sons Ltd.
Abstract
High-dimensional data analysis often uses dimensionality reduction (DR, also called projection) to map data patterns to human-digestible visual patterns in a 2D scatterplot. Yet, DR methods may fail to show true data patterns and/or create visual patterns that do not represent any data patterns. Projection Quality Metrics (PQMs) are used as objective measures to gauge the above process: the higher a projection's scores in PQMs, the more it is deemed faithful to the data it represents. We show that, while PQMs can be used as exclusion criteria - low values usually mean poor projections - the converse does not always hold. For this, we develop a technique to automatically generate projections that score similar or even higher PQM values than projections created by well-known techniques, but show different, often confusing, visual patterns. Our results show that accepted PQMs cannot be used as an exclusive way to tell whether a projection yields accurate and interpretable visual patterns - in this sense, PQMs play a role akin to that of summary statistics in exploratory data analysis. We also show that not all studied metrics can be fooled equally well, suggesting a ranking of metrics in their ability to reliably capture quality.
Description
CCS Concepts: Mathematics of computing → Dimensionality reduction; Computing methodologies → Machine learning; Humancentered computing → Information visualization
@article{10.1111:cgf.70101,
journal = {Computer Graphics Forum},
title = {{Necessary but not Sufficient: Limitations of Projection Quality Metrics}},
author = {Machado, Alister and Behrisch, Michael and Telea, Alexandru},
year = {2025},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.70101}
}