Practice: Clustering, and What the Objective Assumes

(a) Assignment.

The two clusters of three sit around $x=0$ and $x=3$, and the gap between them ($3.0$) is far larger than the spread within either (under $0.5$), so each point is nearest the centroid in its own group.

Checking one explicitly, for $(0.4,0.3)$: squared distance to $(0.0,0.0)$ is ${0.4}^2+{0.3}^2=0.16+0.09=0.25$; to $(3.0,0.0)$ it is $(-2.6{)}^2+{0.3}^2=6.76+0.09=6.85$. Nearest is the first.

Labels: $[0,0,0,1,1,1]$.

(b) Update.

Cluster 0 mean: $x=(0.0+0.4+0.2)/3=0.6/3=0.2$, $y=(0.0+0.3-0.3)/3=0$. So $(0.2,0.0)$.

Cluster 1 mean: $x=(3.0+3.4+3.2)/3=9.6/3=3.2$, $y=(0.0+0.3-0.3)/3=0$. So $(3.2,0.0)$.

(c) Second assignment, and the stopping condition.

Each centroid moved only within its own group, and the groups remain separated by roughly $3$ units, so every point is still nearest the same centroid. Labels are unchanged at $[0,0,0,1,1,1]$.

The algorithm has converged, and the condition checked is that the labels did not change between successive assignment steps, not that a fixed number of passes was completed. Stopping at a count would leave it unsettled whether this is a stable partition or a snapshot of one still moving.

(d) Final WCSS.

Cluster 0, distances to $(0.2,0.0)$:

• $(0.0,0.0)$: $(-0.2{)}^2+0^2=0.04$
• $(0.4,0.3)$: ${0.2}^2+{0.3}^2=0.04+0.09=0.13$
• $(0.2,-0.3)$: $0^2+(-0.3{)}^2=0.09$

Subtotal $0.26$.

Cluster 1, distances to $(3.2,0.0)$, by the same arithmetic with $x$ shifted by $3$: $0.04$, $0.13$, $0.09$. Subtotal $0.26$.

(a) Both satisfy the stopping condition. Convergence in k-means means two things, and both hold in each run: no single point lowers the objective by switching groups, and every centroid sits at the mean of its assigned points. These are local conditions, statements about what one point or one centroid can do alone. The worse run reaches a partition where two centroids share the distant group at $x=20$, splitting three points between them, while the third centroid covers all six left-hand points and settles at their mean, near $(1.7,0.0)$. A location with no data close to it. Check whether any single point can improve: each left point is already nearest that centroid, since the alternatives are about $18$ units away; each right point is already with its nearer right-hand centroid. Nothing moves. The run is converged by the definition, and it scores $14.0850$ against the optimum's $0.7800$. (b) Why an arithmetic error is the wrong diagnosis. An arithmetic slip would break one of the two conditions. A centroid off its members' mean, or a point assigned to a centroid that is not its nearest. Both runs can be checked against those conditions directly, and both pass. There is no error to find. What produced the difference is where the centroids started. The escape route from the worse partition requires one of the two right-hand centroids to relocate to the left region, and a centroid only ever moves to the mean of its currently assigned points, all of which are on the right. Meanwhile no left point can improve by switching to a centroid $18$ units away. Escaping needs several points to move together, and the algorithm has no mechanism for coordinated moves. The partition is locally unimprovable and globally poor, which is precisely what a local optimum is. (c) What should have been done. Restart. Run the algorithm several times from different initialisations and keep the solution with the lowest objective value. Nothing inside a single run reveals the problem: $14.0850$ is just a number, and it does not announce that $0.7800$ was available. On this dataset the risk is not marginal, over $2000$ random initialisations, $508$ of them, or $25.4\mathrm{\%}$, converge to a solution worse than the optimum. A quarter of single runs would have reported something worse than the best answer. What makes the claim checkable is reporting the restart count alongside the result: "lowest WCSS over 50 restarts, $0.7800$" lets a reader judge how hard the search tried. "k-means gave this partition" does not, because it conceals whether the search was tried once or fifty times.

(a) The drops. Dataset P: | $k$ | WCSS | drop | as % of previous |
|---|---|---|---|
| 1 | $386.9600$ | — | — |
| 2 | $194.9600$ | $192.0000$ | $49.6\mathrm{\%}$ |
| 3 | $98.9600$ | $96.0000$ | $49.2\mathrm{\%}$ |
| 4 | $2.9600$ | $96.0000$ | $97.0\mathrm{\%}$ |
| 5 | $2.5250$ | $0.4350$ | $14.7\mathrm{\%}$ |
| 6 | $2.0900$ | $0.4350$ | $17.2\mathrm{\%}$ | Dataset Q: | $k$ | WCSS | drop | as % of previous |
|---|---|---|---|
| 1 | $447.5038$ | — | — |
| 2 | $186.5206$ | $260.9831$ | $58.3\mathrm{\%}$ |
| 3 | $90.1482$ | $96.3724$ | $51.7\mathrm{\%}$ |
| 4 | $54.0073$ | $36.1409$ | $40.1\mathrm{\%}$ |
| 5 | $37.9526$ | $16.0547$ | $29.7\mathrm{\%}$ |
| 6 | $31.1203$ | $6.8323$ | $18.0\mathrm{\%}$ |
| 7 | $24.6415$ | $6.4789$ | $20.8\mathrm{\%}$ | (b) What each supports. Dataset P supports $k=4$. The drop into $k=4$ removes $97.0\mathrm{\%}$ of the remaining objective; the next removes $14.7\mathrm{\%}$. That is a discontinuity, not a gradual flattening. The fourth centroid is the last one doing substantial work, and the fifth is dividing a group that is already tight. The reading is defensible because the sequence changes character abruptly at one place. Dataset Q supports no $k$ at all. Its reductions decay smoothly: $58.3$, $51.7$, $40.1$, $29.7$, $18.0$, $20.8$ percent. There is no step where a large drop is followed by a much smaller one. Note also that the final drop ($20.8\mathrm{\%}$) is larger than the one before it ($18.0\mathrm{\%}$), so the sequence is not even monotone. There is no consistent flattening to point at. The defensible answer for Q is to name no number of clusters. That is a finding, not a failure to answer: the curve does not identify a cluster count because the data has no separated groups for it to identify. Note what would happen to an analyst determined to find an elbow in Q. They might point at $k=2$, since $58.3\mathrm{\%}$ is the largest single drop, but P's largest drops are $49.6\mathrm{\%}$ and $49.2\mathrm{\%}$, both smaller than Q's first, and neither marks P's elbow. So a large drop is not the signal. (c) Why a downward slope is uninformative. WCSS falls as $k$ rises by construction. Any partition into $k$ groups can be refined into $k+1$ by splitting one group, and splitting cannot raise the within-cluster sum of squares. The two new centroids are each at least as close to their members as the single old one was. At $k=n$, every point is its own centroid and the objective is exactly zero. So a monotonically decreasing curve is guaranteed regardless of whether the data contains any group structure. Dataset Q is a single elongated cloud with no groups, and it still produces a respectable-looking descending plot. The information, when there is any, lies in where the sequence of drops breaks, not in the fact that it descends.

(a) The diagnosis is wrong, and the two figures settle it. The returned partition scores $263.9069$. The by-ring partition scores $396.0000$. Since k-means minimises this quantity, the half-moon answer is the better one by the criterion applied, and the by-ring answer is worse by $132.0931$. So the algorithm did not fail to find the rings. It evaluated them and rejected them, correctly. More restarts cannot help: they search for partitions with lower objective values, and the rings have a higher one. Every additional restart would either return the half-moons again or something worse. This is the general diagnostic. When a clustering disagrees with what you expected, compute the objective value of the partition you wanted. If it beats what was returned, the search failed and restarts are the fix. If it loses, the search succeeded and the mismatch is elsewhere. (b) What the objective scores. Within-cluster sum of squares totals each point's squared distance from its cluster's mean. A partition scores well when every point sits close to its group's centre, which builds in an assumption that a cluster is a roughly round region around a point. A ring has no such centre. The mean of the twelve inner points is the origin, and the mean of the twenty-four outer points is also the origin: for both rings, the centre of mass lies in the hole where no data is. Every point of a ring is far from its own group's mean, radius $1$ or radius $4$ away, so the by-ring partition is penalised heavily on exactly the quantity being measured. The half-moon partition, by contrast, places each centroid inside a dense arc of points, so distances to the mean are small. It is a bad answer about the data and a good answer to the question asked. The failure is therefore in the criterion, not the algorithm. Nothing in the search procedure can override which partitions the objective scores well, because that criterion is the method's entire definition of a group. (c) Single linkage, and its cost. Single linkage merges the two groups whose nearest members are closest, so its criterion is connectivity rather than compactness. It never computes a group mean, and never asks whether a cluster has a centre. Adjacent points around a ring are close to each other, neighbouring points on the inner ring are about $0.5176$ apart, and the gap between the rings is $3$, so each ring chains together through short links long before the two rings connect. That recovers the $12$ and $24$ split exactly. What it gives up is robustness. A criterion based on the single nearest pair is decided by one link, so a handful of points lying between the rings would chain them into one cluster and destroy the partition entirely. The chaining failure mode. k-means does not have this weakness: a few stray points shift a mean slightly and no more. Neither method dominates. Single linkage gains the ability to follow elongated and nested structure, and pays for it with sensitivity to individual bridging points; k-means buys stability against stray points, and pays for it by assuming clusters are blobs. Which trade is right depends on what the grouping is for, not on which method is better.

(a) What stability establishes. Stability means the algorithm reliably finds the same partition from different starting points. That is a fact about the objective's landscape, that one solution has a wide basin of attraction, and it is genuinely useful: it rules out the local-optimum problem, where different runs return solutions of different quality. It establishes nothing about whether the segments correspond to real distinctions among customers. A method given $k=3$ returns three groups whatever the data, and if those groups are the best three-way split of a single continuous cloud, it will find that same split every time. The concentric-rings case makes the point sharply: k-means returns the same wrong half-moon partition repeatedly, with complete stability. Stability is consistency, not correctness. The report has shown the search worked, and presented it as evidence that the finding is real. (b) What is missing. The scaling decision. Clustering was done on raw age, spend and purchase count. Squared Euclidean distance adds these directly, so a variable measured in euros with a range in the thousands dominates one measured in years with a range in the tens. The partition is therefore mostly a partition on spend, and the age and frequency descriptions may be incidental. Either standardise, or state why the raw units reflect how much each variable should count, but the choice must appear, since the partition depends on it. The objective value and the restart count. Without WCSS the segmentation cannot be compared with any alternative, including a two- or four-group one. The evidence for $k=3$. The report does not say how three was chosen. If it came from an elbow plot, the drop sequence should be shown; if from a business constraint, that should be said, which is a legitimate reason. Anything distinguishing these groups from an arbitrary partition of one cloud. Three groups of $412$, $388$ and $377$ from a total of $1177$ are nearly equal in size, which is what splitting a single continuous population tends to produce. It is not proof of anything, and it is the pattern that should prompt the question. The descriptions are not evidence. Any partition of a multivariate dataset yields groups that can be characterised by their centroids, and fluency of description tracks the analyst's writing rather than the data's structure. (c) What could settle the number. Could not: the stability already reported; the interpretability of the descriptions; the fact that a WCSS curve slopes downwards, which it does by construction whatever the data. Could: a drop sequence with a genuine discontinuity, a large reduction at one $k$ followed by a much smaller one, which is evidence when present and absent when the drops decay smoothly. An external criterion is equally legitimate: if the business can run exactly three campaigns, then $k=3$ is a constraint rather than a discovery, and saying so is more honest than deriving three from a curve that does not contain it. The strongest check is external validation. If the segments predict something not used in the clustering, response to a campaign, retention, subsequent spend, that is evidence they track a real distinction. Without it, the segmentation is a proposal about how to treat customers, which may still be operationally useful, and is not a finding about who they are.

(a) Why the raw partition groups by income. Squared Euclidean distance adds the squared age difference and the squared income difference directly, as though a year and a euro were the same unit. The age range here is $42$ years; the income range is $450$ euros. Squared, those are $1764$ against $202500$, income differences are roughly two orders of magnitude larger, so the objective is dominated by income and age is very nearly invisible. $\{A,C\}$ pairs the two low incomes ($40000$, $40150$) and $\{B,D\}$ the two high ones ($40300$, $40450$). Each pair spans just $150$ euros, at the cost of spanning $40$ years of age, which the objective barely registers. The alternative $\{A,B\}$, $\{C,D\}$ pairs by age, and each of those pairs spans $300$ euros of income. Since income dominates, the wider income spread is what the objective sees:

The by-age partition loses by a factor of about $3.73$, and it loses on income spread alone. (b) Standardising. Column means: age $41.0$, income $40225.0$. Population standard deviations: age $20.025$, income $167.7051$. | | $z_{\text{age}}$ | $z_{\text{income}}$ |
|---|---|---|
| A | $-1.0487$ | $-1.3416$ |
| B | $-0.9488$ | $+0.4472$ |
| C | $+0.9488$ | $-0.4472$ |
| D | $+1.0487$ | $+1.3416$ | Now the optimal partition with $k=2$ is $\{A,B\}$, $\{C,D\}$, grouped by age, with

The by-income partition $\{A,C\}$, $\{B,D\}$ now scores $4.790025$, so it loses. In z-score units the age gap between the young and old pairs is about $2.0$ standard deviations, while the income gap within each age pair is smaller, so once both variables are expressed on a common scale, age is the more pronounced separation. (c) What decided each, and what is being chosen. | | raw | standardised |
|---|---|---|
| by-income partition | $24100.0000$ (wins) | $4.790025$ |
| by-age partition | $90004.0000$ | $3.209975$ (wins) | In the raw case the outcome was decided by the units the data happened to arrive in. Income is recorded in euros, so its numbers are large, so its differences dominate a sum of squares. Had income been recorded in thousands of euros, the same customers would have grouped by age. Nothing about them would have changed. In the standardised case the outcome was decided by relative spread: each variable is expressed in units of its own standard deviation, so what counts is how far apart the points are compared with the overall variation in that variable. What the analyst chooses is how much each variable should count. Standardising is not a neutral cleanup step that removes an arbitrary artefact; it asserts that a one-standard-deviation difference in age matters as much as a one-standard-deviation difference in income. That is a substantive claim, and it may be wrong, if the question is about spending power, raw euros might genuinely be the right metric. What is not defensible is making the choice without noticing. Both partitions above are optimal, both are correct answers to a well-posed question, and which question was asked is settled entirely by a decision taken before any clustering ran. That decision belongs in the report, because a reader cannot otherwise tell whether they are looking at a fact about customers or a fact about euros.

(a) It cannot, and the geometry is the reason. Healthy readings trace a closed band at a roughly constant distance from the fleet's average operating point. A ring. The mean of a ring lies in the hole at its centre, where no healthy data sits. WCSS scores best on partitions whose points lie close to their cluster's mean, so with $k=4$ the method will do what it does on any ring: carve the band into arcs, placing a centroid inside each dense arc. Four tidy, well-scoring clusters. Now consider a faulty pump, drifting toward the middle of the loop. It sits near the centre of the band, which is near where a centroid would be if the ring were treated as one cluster, and in any case closer to the fleet's average operating point than the healthy readings are. Under a criterion that rewards proximity to a centre, such a point is unremarkable or better than average. The faults are not merely missed; they score well on the very quantity being optimised. This is the concentric-rings failure with consequences. The method is not converging badly. It is answering correctly a question whose criterion treats "close to the middle" as normal, when for this process the middle is exactly where failure lives. (b) What one run leaves open. A single converged run establishes only that no point improves the objective by moving alone and every centroid sits at its members' mean. It carries no information about whether a better partition exists: the reported objective value is a number, and it does not announce that a lower one was available. What establishes it: restarts. Run from many different initialisations and keep the lowest objective value, reporting the restart count so a reader can judge how hard the search tried. The risk is not marginal. On a small nine-point dataset with three well-separated groups, far easier than this one, $25.4\mathrm{\%}$ of $2000$ random initialisations converged to a solution worse than the optimum, one of them by a factor of $18.0577$. A single run on real data with many points and four centroids is not safer than that. Restarts address the search, and the problem in (a) is not the search. Even the globally optimal four-cluster partition would miss the faults, because the criterion scores the wrong structure well. Restarts are necessary and would not rescue this analysis. (c) "No anomalous regime detected" is not a finding. $k=4$ was supplied by the team. The method partitions into exactly the number of groups it is given, whatever the data contains, and it has no mechanism for reporting that a group is unnecessary or that a fifth would have been better. Asking for four regimes and receiving four is not evidence that there are four, and it is certainly not evidence that there is no fifth, anomalous one. For the absence to mean anything, the analysis would need to show that a partition allowing an additional cluster does not isolate a distinct low-vibration, low-temperature group, which requires fitting at several values of $k$ and examining the result, not asserting an absence from a single fit at one $k$. Even then the conclusion would be weak, because of (a): a fault group near the band's centre is not one this objective separates at any $k$. (d) Stability. Stability across the day means the algorithm reliably finds the same partition. A fact about the objective's landscape, namely that one solution has a wide basin of attraction. It genuinely rules out the local-optimum problem, and that much is worth reporting. It says nothing about correctness. The same wrong answer returned consistently is still the wrong answer: on the concentric-rings data, k-means returns the identical incorrect half-moon partition on essentially every restart. Perfect stability, perfectly wrong. The team has evidence that their search converged, and has presented it as evidence that their conclusion holds. (e) What to run instead. A density-based method such as DBSCAN, or a connectivity-based one such as single linkage. Their criteria reward groups of points that are mutually close or reachable through chains of near neighbours, never asking whether a group has a compact centre. A ring is dense and connected along its length, so both can represent it as one cluster, and a sparse point drifting into the middle is then genuinely isolated, which is exactly the signal the team wants. DBSCAN additionally labels such points as noise rather than forcing them into a cluster, which is the right output shape for anomaly detection. What it costs them: - Fragility to bridging points. Connectivity criteria are decided by the nearest link, so a thin trail of readings from the band toward the centre, a pump degrading gradually, could chain the anomalous region to the healthy one and hide the very thing being looked for. k-means does not have this failure mode.
- New parameters that are equally consequential. DBSCAN's neighbourhood radius and minimum-points threshold replace $k$, and the result depends on them as strongly as k-means depends on $k$. The choice has moved, not disappeared.
- No centroids to describe. Density-based clusters have no representative mean, so the tidy per-regime summaries in the current report would not survive the change.

(a) From centroids $\{0,1\}$.

Round 1, assign. Point $0$ is nearer $0$; points $1,2,10,11,12$ are nearer $1$. Groups $\{0\}$, $\{1,2,10,11,12\}$.

Round 1, update. Centroids become $0$ and $(1+2+10+11+12)/5=7.2$.

Round 2, assign. Midpoint $3.6$: points $0,1,2$ join the first, $10,11,12$ the second.

Round 2, update. Centroids become $1$ and $11$.

Round 3, assign. Midpoint $6$; the same points fall the same side. Labels unchanged, so the algorithm stops.

Final partition $\{0,1,2\},\{10,11,12\}$ with

(b) From centroids $\{1,11\}$. The first assignment already gives $\{0,1,2\}$ and $\{10,11,12\}$; the update returns $1$ and $11$ unchanged; the next assignment repeats. Converged immediately, same partition, $\mathrm{WCSS}=4$.

Both runs agree here, which is what well-separated data does. The instructive case is the one that does not: on data whose groups sit closer together, two centroids can settle inside one natural group while the other is absorbed whole, and no single point improves the objective by moving alone.

(c) What convergence establishes. Exactly two things: no single point lowers the objective by changing clusters, and every centroid is at the mean of its members. That is a local optimum.

What it does not establish is that the partition is the best available. The algorithm halts because the objective stopped falling, not because it reached the lowest value; the assignments are finite and the search follows a path determined entirely by where it started. A converged run reports a stopping condition, not a proof.

What would detect a worse solution. Multiple restarts from different initialisations, compared by objective value: the lower WCSS is the better solution found. That reduces sensitivity to initialisation and can find a better partition, but establishes no guarantee of global optimality, which would require exhaustive or global optimisation. A defensible report therefore states the objective value and the number of restarts behind it.