Overview: Linear Regression

A statistical method used to predict the value of one variable based on another

Dependent Variable (Y)

Variable being predicted.

Independent Variable (X)

Variable used for prediction.

Regression Equations

A mathematical equation used for prediction.

Simple Linear Regression:

One independent variable.Multiple Linear Regression

Multiple Linear Regression:

Two or more independent variables

Fitting Regression:

Finding the straight line that best represents the relationship between X and Y using the given sample data.

Scatter Diagram:

A graphical representation of paired data (X, Y).

Each pair is plotted as a point.

Best-fit line is the one that minimises the sum of squares of residuals:

\[S^2=\sum(y_i-\hat{y}_i)^2\]

Residual: \[e_i=y_i-\hat{y}_i\]

Y = a + bX

where b = b_YX = regression coefficient of  Y on X

\[b_{_{YX}}=\frac{\operatorname{cov}(X,Y)}{\operatorname{var}(X)}\]

\[=\frac{\frac{\sum\left(x_i-\overline{x}\right)\left(y_i-\overline{y}\right)}{n}}{\frac{\sum\left(x_i-\overline{x}\right)^2}{n}}\]

\[=\frac{\sum x_iy_i-n\bar{x}\bar{y}}{\sum x_i^2-n\bar{x}^2}\]

\[a=\overset{-}{\operatorname*{y}}-b\overset{-}{\operatorname*{x}}\]

\[X=a^{\prime}+b^{\prime}y\]

where b' = b_XY = regression coefficient of X on Y

\[b_{_{XY}}\quad=\quad\frac{\operatorname{cov}(X,Y)}{\operatorname{var}(Y)}\]

\[\begin{array}
{cc} & \frac{\sum\left(x_i-\overline{x}\right)\left(y_i-\overline{y}\right)}{n} \\
= & \frac{\sum\left(y_i-\overline{y}\right)^2}{n}
\end{array}\]

\[b_{XY}=\frac{\sum x_iy_i-n\bar{x}\bar{y}}{\sum y_i^2-n\bar{y}^2}\]

\[\begin{array}
{rcl}a^{\prime} =\overline{x}-b^{\prime}\overline{y}
\end{array}\]

1.\[b_{_{XY}}.b_{_{YX}}=r^{2}\]

2. If  b_YX > 1, then b_XY < 1.

3. \[\left|\frac{b_{yx}+b_{xy}}{2}\right|\geq|r|\]

4. Regression coefficients are independent of a change of origin but are affected by a change of scale.