Unsupervised Learning: K-Means, Hierarchical Clustering, and t-SNE

No labels, no “right answer” to check against. Unsupervised learning is about finding structure in data when nobody’s told you what the structure is supposed to look like.

Three techniques cover most of what you’ll actually reach for: K-Means (groups data into clusters), hierarchical clustering (builds a hierarchy of clusters you can cut at any level), and t-SNE (squashes high-dimensional data down to something you can actually plot and look at).

A Quick Word on Scaling First

Distance-based methods like K-Means compare data points by measuring distance between them. If your features are on wildly different scales (age from 0 to 100, salary from 0 to 1,000,000), the distance calculation ends up dominated by whichever feature has the biggest numbers, even if that feature isn’t actually more important.

Standardization (subtract the mean, divide by the standard deviation) fixes that. StandardScaler in scikit-learn does the job. I cover scaling and general model evaluation in more depth in [my ML fundamentals post] (link it here), so I won’t repeat all of it. The short version for this post: scale your features before running K-Means, or your clusters will reflect feature magnitude more than actual structure.

K-Means Clustering

K-Means groups data into k clusters. You choose k up front.

The algorithm finds a centroid (the mean point) for each cluster and assigns every data point to the nearest one. Once trained, new points get assigned the same way, by proximity to the nearest centroid.

The goal: tight clusters, where points in the same cluster sit close together. K-Means measures this with inertia, basically how spread out the clusters are. Lower inertia means tighter clusters.

Picking k: The Elbow Method

Here’s the catch with inertia: it keeps dropping as you add more clusters. More centroids means every point has an easier time finding something close to it. Taken to the extreme, k equal to your number of data points gives you zero inertia and zero usefulness.

So you’re not trying to minimize inertia outright. You’re looking for the point of diminishing returns.

The Elbow Method: plot k on the x-axis, inertia on the y-axis. Look for the “elbow,” the point where adding more clusters stops meaningfully reducing inertia. That’s usually your sweet spot between cluster quality and not over-segmenting your data into clusters nobody can interpret.

Do the Clusters Actually Mean Anything?

K-Means will always produce clusters. Whether they’re meaningful clusters is a separate question.

If you have known categories to check against, cross-tabulation is the move: build a table showing how many points from each real category landed in each cluster. A cluster that’s mostly one category is a good sign. A cluster that’s an even mix of everything suggests the clustering isn’t capturing anything useful.

Hierarchical Clustering

Instead of picking a number of clusters up front, hierarchical clustering builds the whole hierarchy and lets you decide later.

The result is a dendrogram, a tree showing how clusters merge at different levels.

Agglomerative clustering (the common approach) works bottom-up:

1. Every data point starts as its own cluster.
2. Merge the two closest clusters.
3. Repeat until everything’s in one cluster.

The height at which two clusters merge in the dendrogram represents how far apart they were. Want more clusters? Cut the dendrogram lower. Want fewer? Cut higher.

There’s also divisive clustering, the reverse: start with everything in one cluster and split top-down. Agglomerative is the one you’ll run into more often in practice.

t-SNE

t-SNE isn’t for clustering. It’s for looking at your data.

It takes high-dimensional data and compresses it down to two or three dimensions, low enough to actually plot. The core idea: points that were close together in the original high-dimensional space stay close together in the compressed version. Points that were far apart stay far apart.

That makes it genuinely useful for spotting clusters and structure visually, especially in data with way too many dimensions to reason about directly.

The catch: t-SNE is built to preserve local structure, meaning relationships between nearby points. It doesn’t guarantee the same for distant clusters. If two clusters look far apart on a t-SNE plot, don’t read too much into exactly how far apart. That distance isn’t necessarily meaningful the way it would be in the original space.

K-Means vs. Hierarchical Clustering vs. t-SNE

Technique	What it’s for
K-Means	Grouping data into a fixed number of clusters
Hierarchical clustering	Building a full hierarchy of clusters, decide cluster count after the fact
t-SNE	Compressing high-dimensional data down for visualization, not for clustering itself

Worth remembering: t-SNE and clustering solve different problems. People sometimes reach for t-SNE expecting it to cluster their data. It doesn’t. It just makes existing structure easier to see.

FAQ

What is K-Means clustering?

An unsupervised learning algorithm that groups data into k clusters by assigning each point to its nearest cluster centroid.

How do you choose the number of clusters in K-Means?

The Elbow Method: plot the number of clusters against inertia, and look for the point where increasing k stops meaningfully reducing inertia.

Why does scaling matter for K-Means?

K-Means relies on distance calculations between points and centroids. Features with larger numeric ranges can dominate that calculation even when they’re not actually more important, so scaling puts every feature on equal footing.

What is hierarchical clustering?

A clustering method that builds a full hierarchy of clusters, represented as a dendrogram, so you can choose how many clusters you want after the fact rather than specifying it up front.

What is t-SNE used for?

Reducing high-dimensional data to two or three dimensions so it can be visualized and its structure explored.

Does t-SNE preserve distances between all points?

No. It preserves local structure well (relationships between nearby points), but distances between far-apart clusters on a t-SNE plot aren’t necessarily meaningful.