The vertical line is a guess c for the data's center. Drag it.
The dot rides the error curve for the guess and shows its value.
Distance to Center
The animation on the Graphics tab illustrates two methods,
least squares and least absolute distances,
to find the best center of a set of data, x1, x2, …, xn.
We begin with least squares.
Least Squares
Imagine trying all possible choices of center, c ∈ (0, ∞) for data
x1, x2, …, xn. For each c, compute the
sum of squared distances of data from c:
. (1)
The value of c minimizing the sum of squared distances is called the
least squares solution. Perform the following steps to understand least squares.
Open the Graphics tab to see a data set of points on the x-axis. Click anywhere to choose
a center, c.
The vertical line is your choice of c.
The blue rectangle reports the value of c and the sum of squared
distances of points from that center, Σ(xi − c)2.
The blue parabola plots Σ(xi − c)2
as a function of c, and the blue dot corresponds to
Σ(xi − c)2 for your choice of c.
Slide c left or right to choose a new c. The vertical line and blue dot and rectangle are updated
based on your new choice of c.
Continue sliding your center until the blue dot is at the bottom of the parabola. This occurs at
c = x̄ = 4.36, showing that c = x̄ is the least squares
solution. This holds for every dataset.
Least Absolute Ditances
We can replace the sum of squared distances (1) with the sum of absolute distances:
. (2)
The c minimizing (2) is called the least absolute distance solution.
Click on “Least Absolute Ditance” at the top left of the graph.
Again click on a proposed center and slide it left or right to see the same information as before,
but for the sum of absolute distances of data points from c in orange.
The orange curve is very different than the previous blue curve. It looks like several line segments
joined together rather than a smooth curve.
The lowest point on the orange curve occurs at c = 5, which is the median of the data.
The median is the least absolute distance solution for any dataset.
One advantage of absolute, over squared, distances is that outliers have less effect. The minimizing
value of c is the median, and the median is less affected by outliers than is the mean.
Summary
There are two common ways of finding the center of a set of data: least squares, which minimizes the
sum of squared distances (Equation (1)), and least absolute distances, which minimizes the sum of
absolute values of distances (Equation (2)). Least squares and least absolute distances produce the
sample mean and median, respectively, as best center. Least squares is mathematically more convenient,
but least absolute distances is preferred when there are outliers.