What is the Sum of Squares?

Sum of squares is like an appetizer…

When you sit down for a meal, you might not want to wait for the main course. So, perhaps you order a salad or appetizer to tide you over while you wait for the entre. That appetizer (sum of squares) is just the first course (step) in completing the dining experience. Like an appetizer, the sum of squares might be all you want. But, more often, you’ll use it to get to a different, more meaningful statistic.

The sum of squares is a statistical measure of dispersion. It quantifies the distance numbers are from the average of a dataset, or a regression model (estimate of one variable using another). Those differences are squared then summed to determine the sum of squares value. The greater a sum of squares becomes, the more spread out a dataset is, and the worse a regression is at predicting an outcome. Because deviations are squared, more considerable differences cause the sum of squares to increase by much more than smaller variances do.

As a measure of spread, the total sum of squares is an indicator of volatility, and therefore, signals that there’s more risk. However, analysts usually use the sum of squares to calculate other measures of volatility rather than using it directly. In linear regression models, the total sum of squares is divided into the explained sum of squares (the variation explained by the regression model) and the residual (unexplained) sum of squares. A statistical model with a high residual sum of squares has less explanatory power than one with a lower value.

The sum of squares formula might look a little intimidating at first, but it’s actually quite simple. It just says to take the difference for each data point, square it, then add all the values together. Here is what it looks like:

Total Sum of Squares = ∑ (Xi – Xavg) 2

Xi = data point i

Xavg = average of all data points in the set

∑ = instruction to sum the values together

In words, the formula says to do this:

Step 1: Calculate the average value of the dataset. Do this by adding all of the numbers together and dividing that sum by the number of data points.

Step 2: Determine the difference between each data point and the average value.

Step 3: Square (multiply by itself) each of the differences you developed in Step 2. This action turns all of the numbers into positive values.

Step 4: Add up all of the squared deviations from Step 3.

Say you wanted to understand the variability of the an Airline common stock price in April 2022 at closing. You would put those data into an Excel spreadsheet. Here is the data, along with the other three columns calculated for you:

The third column is the average value for the entire dataset. In the fourth column, you’ll find the difference between the data point and the average. The last column takes the square of the deviations from the average. At the bottom, the squared differences are added together to determine the sum of squares.

A word of caution: The function in Excel =SUMSQ() squares then sums the values you give it. So, you can’t apply it to the data in the first column and be done. You’ll need to determine the differences first, then apply the function to the third column.

There is a mathematically equivalent way to get the same answer, which might be a little easier in some cases. It involves doing the steps in a different order. The alternative formula looks like this:

Total Sum of Squares = ∑ (Xi2) – 1 / n ∑ (Xi) 2 Xi = data point i n = number of data points ∑ = instruction to sum the values together

The formula doesn’t look like much of a shortcut, but the steps might illustrate the advantage.

Step 1: Multiply each of the data points by itself and add up all of the resulting values.

Step 2: Add up all of the numbers in the dataset, and square the result.

Step 3: Divide the result from Step 2 by the number of data points.

Step 4: Subtract the result in Step 3 from the answer in Step 1.

The total sum of squares (TSS or SST) tells you how far the data points in a dataset are from the center. It’s a descriptive statistic called a measure of spread or dispersion. Dividing the TSS by the number of observations in the dataset gives you the average variability within the data, which is called the variance. Taking the square root of the variance generates the standard deviation (another measure of dispersion).

In a linear regression analysis, the total sum of squares is partitioned into two pieces — The explained sum of squares (ESS or SSE) and the residual sum of squares (RSS or SSR). These values are related, in that:

TSS = ESS + RSS

That formula just says that the total variability equals what is explained by a regression model plus what is left unexplained (the residual). The best fit for a regression line is the one that minimizes the RSS, which is called the ordinary least square (OLS).

Analysts look at the percentage of the variability that is explained by the best fit line to determine how valuable a regression model is at predicting one variable based on observations of others. That value is called the R-squared. It’s calculated using the sum of squares as follows:

R2 = ESS / TSS

R-squared values closer to one explain almost all of the variability in the data. A value closer to zero has little explanatory power.

In practice, an analyst could use the total sum of squares (TSS) as a direct measure of variability. However, the TSS is unscaled. In other words, comparing the TSS of one data set to another wouldn’t tell you anything. The variability of the size of pumpkin seeds will almost certainly be a smaller number than the variability of pumpkins. But that doesn’t mean the seeds are more like each other than pumpkins are. Maybe it’s more likely for a pumpkin seed to be twice the size of the average seed than it is likely for a full-grown pumpkin to be twice the size of the average full-grown pumpkin. Comparing TSS won’t tell you that.

That is why it’s important to scale the measure of variability to the data in question. For that reason, analysts don’t use TSS very often. Instead, they use TSS to determine variance, standard deviation, and R-square values. Each of those statistics is scaled, which allows analysts to compare them to other data. A stock with a larger standard deviation than another stock is more volatile. A regression equation with a bigger R-squared has more explanatory power than another model with a smaller R-squared.

The practical application of the sum of squares is to develop more meaningful measures of volatility. Those measures help analysts determine the associated level of risk in a security, and the required potential reward necessary to compensate for that risk.