Linear regression
Basic Introduction of linear model Youtube:StatQuest
1. least square of residuals
Youtube: Deriving the least square of the slope and intercept
\(\hat{Y}=\hat{\beta}_0+\hat{\beta}_1{X}+\varepsilon\)
\(\varepsilon ~ N(0,\sigma^2)\)
estimation of \(\beta_0\) and \(\beta__1\)
\(minQ=\sum_{i=1}^{n}(y_i-(\beta_0+\beta_1x_i))^2\)
\(\frac{\partial{Q}}{\partial{\beta_0}}=-2\sum_{i=1}^n(y_i-(\beta_0+\beta_1x_i))=0 \)\(\bar{y}=\beta_0+\beta_1\bar{x}\)
\(\frac{\partial{Q}}{\partial{\beta_1}}=-2\sum_{i=1}^{n}(y_i-(\beta_0+\beta_1x_i))x_i=0\)
Remember our next aim is to get \(\beta_1\)
\(\because \beta_0=\bar{y}-\beta_1\bar{x}\)
\(\therefore \sum_{i=1}^{n}(y_i-(\beta_0+\beta_ix_i))x_i=\sum_{i=1}^{n}(y_i-(\bar{y}-\beta_1\bar{x}+\beta_1x_i))x_i \)
\(=\sum_{i=1}^{n}x_i(y_i-\bar{y})-\beta_1\sum_{i=1}^{n}x_i(x_i-\bar{x})=0 \)
\(\therefore\beta_1=\frac{\sum_{i=1}^{n}x_i(y_i-\bar{y})}{\sum_{i=1}^{n}x_i(x_i-\bar{x})}=\frac{Cov(X,Y)}{Var(X)} \)
###estimation of \(\sigma^2\) Youtube:
2. \(R^2\)
The left means sum of squared mean SS(mean); the right means sum of sqaured residuals, called fit SS(fit), which is variation not be explained by the regression line.
\(R^2\) is the percentage of variation explained by the relationship between two variables
The proof of why Pearson correlation coefficient = R-square in simple linear model
