mediumConceptMidSeniorStaff
When does dimensionality reduction help, and when does it hide the problem?
Dimensionality reduction helps when features are redundant, noisy or too expensive to serve, but it can hide the signal. PCA keeps variance, not relevance, and t-SNE or UMAP plots should guide exploration rather than prove model quality.
4 min readUpdated 2026-08-15
Answer
Short answer
Use dimensionality reduction when you need compression, denoising or a representation a downstream method can handle. Avoid treating PCA, t-SNE or UMAP as proof of signal because low-dimensional projections can discard label information or create misleading visual clusters.
Was this answer helpful?
Likely follow-ups
- Why must PCA be fitted inside the cross-validation loop rather than on the full dataset?
- Your t-SNE plot shows four beautiful clusters. What would you do before believing them?
- When would you prefer feature selection to feature extraction?
- How does PCA behave if one feature is in pounds and another in thousands of seconds?
Related questions
- You need to flag anomalies and nobody has labelled a single one. How do you build it?mediumAlso on unsupervised-learning4 min
- You've clustered the customer base and there are no labels. How do you know the clustering is any good?mediumAlso on unsupervised-learning4 min
- Explainability is a hard requirement for this model. How does that change what you build?hardAlso on interpretability6 min
- When is a neural network the wrong choice?mediumAlso on interpretability5 min