Total sum of squares formula

[vadim0x60]

In _r2_score_compute r2.py:80 Total Sum of Squares is defined as

mean_obs = sum_obs / num_obs
tss = sum_squared_obs - sum_obs * mean_obs

i.e.

$$\text{TSS}= \sum_{i=1}^{n}y_{i}^2 - (\sum_{i=1}^{n}y_{i})\bar{y}$$

which is different from the usual formula

$$\mathrm{TSS}=\sum_{i=1}^{n}\left(y_{i}-\bar{y}\right)^2$$

Am I missing someting? Could this be a mistake

[github-actions]

Hi! thanks for your contribution!, great first issue!

[SkafteNicki]

Hi @vadim0x60, thanks for opening this issue.
The two formulations are equal:

$$TSS = \sum_{i=1}^n (y_i - \bar{y})^2$$

substitute the definition of

$$\bar{y} = \frac{\sum_{i=1}^{n} y_i}{n}$$

into the expression we get

$$TSS = \sum_{i=1}^n (y_i - \frac{1}{n}\sum_{i=1}^{n} y_i)^2$$

then expand the square term

$$TSS = \sum_{i=1}^{n} \left( y_i^2 - \frac{2y_i}{n} \sum_{i=1}^{n} y_i + \frac{1}{n^2} \sum_{i=1}^{n} y_i^2 \right)$$

Next, distributing the summation across the terms:

$$TSS = \sum_{i=1}^{n} y_i^2 - \frac{2}{n}\sum_{i=1}^{n} y_i^2 + \frac{1}{n}\sum_{i=1}^{n} y_i^2$$

And finally, simplifying the expression:

$$TSS = \sum_{i=1}^{n} y_i^2 - \frac{1}{n}(\sum_{i=1}^{n} y_i)^2$$

the last term we can rewrite again by using the definition of the average (in reverse order)

$$TSS = \sum_{i=1}^{n} y_i^2 - \frac{n^2 \times \bar{y}^2}{n}$$

expand again

$$TSS = \sum_{i=1}^{n} y_i^2 - \frac{n^2 \times \bar{y} \times \bar{y}}{n}$$

split

$$TSS = \sum_{i=1}^{n} y_i^2 - \frac{n^2 \times \bar{y}}{n} \times \bar{y}$$

simplify

$$TSS = \sum_{i=1}^{n} y_i^2 - n \times \bar{y} \times \bar{y}$$

again use the definition of the average to convert into the expression we want

$$TSS = \sum_{i=1}^{n} y_i^2 - \left( \sum_{i=1}^{n} y_i \right) \bar{y}$$

Hopefully this convince you that the two expressions are equal. The reason for why we use one over the other is kind of lost on me right now, but if I remember correctly one expression is a bit more numerical stable than the other, which is why we have chosen it.

[vadim0x60]

Thanks for taking the time to lay this out!