Global F tests and Family-Wise Error Rates
========================================================
### Code for Biostatistics Methods 2, UMass-Amherst, Spring 2014
### by [Nicholas Reich](http://nickreich.github.io)

If you include a lot of predictor variables in a regression and are looking to evaluate the significance of many of them, you should consider using global F tests. Here is a simple example that illustrates why this test is important. 

We start by picking a number of observations ($nObs$) and the number of parameters in our model, $p$. We then generate $p-1$ independent covariates, plus a column of 1s for the design matrix.
```{r}
nObs <- 1000
p <- 100
x <- matrix(rnorm(nObs*p), nrow=nObs)
x <- data.frame(x)
colNames <- paste0("x", 1:p)
colnames(x) <- colNames
```

Now we will generate our $y$s completely independently of all of our covariates. None of our $x$ variables are associated with our outcome!
```{r}
y <- rnorm(nObs)
```

But if we fit a linear model that assumes that there ARE relationships bewteen our outcome and all of our $x$ variables, do we see any individually significant $\beta$s? If so, how many are significant and are these indiciative of real associations? Let's start by constructing a linear model formula that includes each of our $x$ variables. We have suppressed the printing out of this formula in this write-up but if you run these two command below, you will see the formula that is created.
```{r, results='hide'}
fmla <- formula(paste0("y ~ ", paste(colNames, collapse="+")))
fmla
```

Now we fit the model and evaluate how many of the individual $\beta$ coefficients are significant at the $\alpha=0.05$ level. 
```{r}
mlr1 <- lm(fmla, data=x)
coefs <- summary(mlr1)$coef
sum(coefs[,"Pr(>|t|)"]<.05)
```

Alternatively, we could use a Global $F$-test to test whether any of the $x$ variables add significant explanatory power to our model. We do this by fitting a "null" model that just includes an intercept. What conclusion do we draw from this test?
```{r}
mlr0 <- lm(y ~ 1, data=x)
anova(mlr0, mlr1)
```

Do the results about the significance of the model coefficients from the Global $F$-test and the individual $\beta$ $t$-tests agree? Why or why not?

