12.4: K-Means Clustering

K-means clustering is an unsupervised machine learning technique used to group similar data points into distinct clusters. It is widely applied in marketing, customer segmentation, pattern recognition, and data compression. Unlike supervised learning, k-means does not require labeled output variables; instead, it identifies natural groupings in the data based on similarity.

The main goal of k-means clustering is to partition a dataset into k distinct, non-overlapping subsets or clusters, where each data point belongs to the cluster with the nearest mean. The mean of a cluster is referred to as its centroid.

The algorithm works through the following steps:

1. Select the number of clusters, k, which must be specified in advance.
2. Randomly initialize k points as the initial centroids.
3. Assign each data point to the nearest centroid based on a chosen distance metric, commonly Euclidean distance.
4. Recalculate the centroids by computing the average of all points assigned to each cluster.
5. Repeat steps 3 and 4 until the centroids no longer change significantly or a maximum number of iterations is reached.

K-means attempts to minimize the within-cluster sum of squares, also called inertia. This means it seeks compact clusters with points closely packed around their centroids.

One of the challenges in using k-means is choosing the optimal number of clusters. Techniques like the elbow method can help determine this number by plotting the total within-cluster variation against different values of k and identifying the point where adding more clusters yields diminishing returns.

K-means clustering assumes that clusters are spherical and roughly equal in size, and it can be sensitive to outliers and the initial placement of centroids. Despite these limitations, it remains a powerful and efficient tool for clustering tasks in a wide variety of business and academic applications.

Strengths of K-Means

• Computational Efficiency: K-Means is highly efficient, especially with large datasets.
• Simplicity: The algorithm is easy to understand and implement.
• Interpretability: Results are intuitive—each cluster is defined by a centroid.

Limitations of K-Means

• Pre-specification of k: The number of clusters must be defined in advance.
• Sensitivity to Outliers: Outliers can skew centroids.
• Assumption of Spherical Clusters: Performs best on convex, similarly sized clusters.

Outputs and Interpretation

The outputs of a K-Means model include cluster assignments, centroids, and Within-Cluster Sum of Squares (WCSS). These are typically visualized with scatter plots. Dimensionality reduction techniques like principal component analysis (PCA) may be used for higher-dimensional data.

Visualizations

Figure 1. Cluster assignments displayed with centroids shown as black 'X' markers. This helps visualize groupings.

Figure 2. Elbow plot showing WCSS across different values of k. The 'elbow' is a visual cue for the ideal number of clusters.

Evaluation Metrics

• Silhouette Score: 0.599 (values closer to 1 indicate better cohesion and separation).
• Davies-Bouldin Index: 0.590 (lower scores indicate better-defined clusters).

Click on the following video to see a visual representation of K-means cluster: What is K-Means Clustering?

Summary

K-Means is a go-to clustering algorithm known for its speed and simplicity. By minimizing WCSS, it creates compact, well-separated groups that are easy to interpret. However, its effectiveness depends on choosing the right number of clusters and preprocessing the data. With proper implementation and validation using visual and numeric tools, K-Means can deliver powerful insights in various business contexts.