Testing the Individual Model Parameters

Preliminary

Variable definitions

Some variables that will be used throughout this page:

• $n:$ Number of observations
• $p+1:$ Number of coefficients ($\beta$) in the model
• $t_{q,v}:$ Point percent function of a Student’s-t distribution at the $q-$th quantile
• $v=n-(k+1):$ The degrees of freedom in $t_{q,v}$

Warning

Failing to reject $H_0$ does not mean that the independent variable does not explain the dependent variable.

Instead, several conclusions are possible:

• There is no relationship
• A relationship exists, but a Type II error occurred
• A relationship exists, but is different than the hypothesized model

The most you can say after testing is:

• If $H_0$ is rejected: There is a sufficient evidence for the hypothesized relationship
• Else: There is insufficient evidence for the hypothesized relationship

Recommendations

1. First, test the overall model adequacy.

   If $H_0$ is rejected, continue to step 2

   Else, consider hypothesizing a different model

2. Conduct t-tests on the most “important” $\beta$ coefficients. Usually only involves $\beta$s involved with higher-order terms

   Conducting a series of t-tests leads to an overall high Type I error rate

Assumptions

General form of the model

$$Y={\beta }_0+{\beta }_1{X}_1+\cdots +{\beta }_k{X}_k+ϵ;ϵ\sim NIID(0,{\sigma }^2)$$

Where:

• $Y:$ Response variable
• $X_j:$ $j$-th predictor variable
• ${\beta }_0:$ Intercept
• ${\beta }_j:$ $j$-th regression coefficient
• $ϵ:$ Error component
• $k$ : Amount of predictor variables

Model assumptions

• $y_i$ mutually indepentent between each other
• $\mathrm{Var}(y_i)={\sigma }^2:$ The model has constant variance
• $Y,ϵ:$ Probabilistic part of the model
• $E(Y|X)={\beta }_0+\cdots +{\beta }_k{X}_k:$ Deterministic part of the model

The assumptions for the error component ${ϵ}_i$ is the same as the one on simple linear regression model: Assumptions for the Error Component.

Interpretation of model components

• Slope: ${\beta }_0$ is the mean value of $Y$ at $X_1=X_2=\cdots =X_p=0$
• Regression coefficients: For an increase in $X_i$ by 1 unit, then mean of $y$ increases by ${\beta }_i$ (assuming other predictors are constant)

Elasticity

Elasticity measures the relative change in the dependent variable $Y$ due to a relative change in $X_k$.

Semi-elasticity measures the relative change in the dependent variable $Y$ due to an (absolute) one-unit-change in $X_k$.

For a linear regression, the elasticity of $Y$ with respect to $X_k$ is

$$\begin{array}{rl}\frac{\partial E[y_i|x_i]/E[y_i|x_i]}{\partial x_{ik}/x_{ik}}=& \frac{\partial E[y_i|x_i]}{\partial x_{ik}}\cdot \frac{x_{ik}}{E[y_i|x_i]}\\ =& \frac{x_{ik}}{x_i^{\prime }\beta }{\beta }_k\end{array}$$

Linear regression for $\log (y_i)=x_i^{\prime }\beta +{\epsilon }_i$, the elasticity of $Y$ with respect to $X_k$ is

$$\frac{\partial E[y_i|x_i]/E[y_i|x_i]}{\partial x_{ik}/x_{ik}}={\beta }_k{x}_{ik}$$

${\beta }_k$ measures the relative change in $Y$ due to a change in $X_k$ by one unit. Here, $b_k$ is called the semi-elasticity of $Y$ with respect to $X_k$.

Test statistic

$$t=\frac{{\hat{\beta }}_i}{s_{{\hat{\beta }}_t}}\sim t_v$$

Hypotheses

	Two-tailed	Lower-tailed	Upper-tailed
Null hypothesis	$H_0:{\beta }_i=0$	$H_0:{\beta }_i=0$	$H_0:{\beta }_i=0$
Alternative hypothesis	$H_a:{\beta }_i\ne 0$	$H_a:{\beta }_i<0$	$H_a:{\beta }_i>0$
Rejection region	$	t	> t_{\alpha/2,v}$

P-value

$$\begin{array}{rl}{H}_a& :{\beta }_i>0,p\text{-value}=\{\begin{array}{ll}P/2& t>0\\ 1-P/2& t<0\end{array}\\ & \\ H_a& :{\beta }_i<0,p\text{-value}=\{\begin{array}{ll}1-P/2& t>0\\ P/2& t<0\end{array}\end{array}$$

Confidence interval

A $100(1-\alpha )\%$ confidence interval for a $\beta$ parameter is found by:

$${\hat{\beta }}_i\pm t_v^{-1}(\alpha /2)\times s_{{\hat{\beta }}_t}$$