Hypothesis Testing Revisited: Normal, t, and Chi-Squared Distribution Tests

welcome back so we've talked a lot about hypothesis testing and we started off with pretty simple examples but very quickly uh we built in a lot of complexity with the T distribution and the Ki Square distribution and so here I just want to do a really quick recap summary overview of hypothesis testing with the three main types uh of testing that we have discussed so far because this can be a little hard to kind of straighten out all of the details when to use what how but I want to kind of point out that hypothesis testing has a really simple kind of procedural uh formula and knowing when to use what test statistic for what hypothesis is actually not that complicated okay so we started with the most simple kind of hypothesis test which is uh it's literally called a simple hypothesis as opposed to a composite hypothesis where the hypothesis we're testing is we have some data X um some samples from a system and we're trying to test the hypothesis does it belong to a normal distribution with uh a mean mu and some variance Sigma squar so the reason we use this was for example um to test the hypothesis you know did some drug or medical intervention change the mean of a population or did some you know marketing campaign change the average number of web traffic or you know average clicks per day something like that but generally speaking that is a type of hypothesis um does my data adhere to a normal distribution with a mean mu this mean mu would be um you know in the case of um the medical example we're testing did this intervention change the mean and so the hypothesis the null hypothesis h0 is that the mean was unchanged that X actually comes from a normal distribution with my previous mean mu this is my kind of control group mean or my before treatment mean and if we have data X that's different enough we will be able to reject this null hypothesis and make some statistical assertion that new data actually has a different mean that's that was the first hypothesis testing we looked at and the test statistics so all of these T's are what we call our test statistic we build a test statistic from our actual data we collect data an ensemble of n samples of data I equals 1 to n and the test statistic for this hypothesis for this null hypothesis is the sample mean xar just the average of all my data minus the nominal or putative mean mu divided by Sigma over root n where Sigma is the uh standard deviation of the distribution I'm comparing against and root n is square root of the size of my sample of data we've shown from the central limit theorem that this test statistic should follow a unit standard normal gausian distribution and so what essentially that means is that I can draw this standard unit normal I can I compute this test statistic from actual data this is a number little Z and what I can do is for this hypothesis I can define a P value a significance value let's say I want a p of. 05 meaning I want to be 95% sure before I reject the null hypothesis I want strong you know I want statistical significant data to support you know rejecting that null hypothesis and I can define a rejection region if my data if my test statistics Z is in this rejection region then I can reject my null hypothesis with that P value that's that level of statistical significance so if my P value is 0. 05 then I kind of have a 95% confidence in rejecting you know that if I reject my null hypothesis I'm actually right okay and so you literally calculate this zv value based on your actual data um and you know you see where that zv value lives with respect to this standard unit normal and if it's in the rejection region we say we reject the null hypothesis I'm just saying how we say this we reject the null hypothesis and that means that the alternative hypothesis that the treatment worked at change of the mean is true with some statistical significance given by the P value associated with this rejection region that was the most simple kind of hypothesis testing we've done so far now we have shown that this kind of assumes

that we have access to Sigma the standard deviation of the actual distribution we're trying to compare it against and if you have access to Sigma nothing changes you do this you know you build this test statistic it follows a normal distribution you build a rejection region and you test the null hypothesis okay nothing strange but if I don't know the variance or the standard deviation of my distribution if I only know mu then I have to do what's called bootstrapping this standard deviation this Sigma so instead of this Sigma here I have to replace it with SN which is called the sample standard deviation this is an approximation of Sigma that I compute from my data I literally compute the sample variance and I take its square root and that's SN this test statistic if I don't know Sigma if I have to bootstrap it from the data this follows the students T distribution with n minus1 degrees of freedom now this T distribution looks a lot like a gaussian and for large n it actually converges to the normal distribution but for small n it has fatter tails and so its rejection region might actually be different I might actually get a different uh result if I use this rejection region versus this rejection region so for example if I don't know Sigma and I have to bootstrap it and I use a I don't have a large number of samples if n is small then I have to use the student T distribution and similarly I can build a rejection region based on this probability density function you know same P value let's say p equals 05 but now my T value might be just to the left of that rejection region and in this case we say that we fail to reject H knot which means that we don't have enough statistical evidence to reject hot so H knot for all our intents and purposes we have to consider it a a possibility that H knot is actually true that the mean did not change that the treatment did not work okay um and so this little difference in the fatness of this tail for small n can make the difference between accepting or rejecting the null hypothesis so it's actually important okay and then the third type of hypothesis um that we've been testing is a really cool and I really like this one it's whether or not our data does or does not belong to a certain distribution so here we're just trying to test did the mean change but down here we might want to test does it even belong to that distribution in the first place is my data normally distributed at all is my data Plus on distributed at all and so what we do in this case is we bin up our data and distribution we build histograms essentially of the data and the distribution and we compare bin by bin the values The observed value and the expected value bin by bin and we compute this test statistic we call it X2 and this follows a Ki squar distribution with n minus one degrees of freedom so same idea I compute this test statistic and I can literally this is a number and I plot it in my Ki Square distribution I have a rejection region same P value same significance and if I compute this um test statistic called the Pearson Kai squ test statistic maybe my value lands here and it's not in the rejection region so I fail to reject the null hypothesis which means in fact that the null hypothesis is likely to be true and my data did come from that distribution so this means that my data did come from the distribution which is really cool so that's another hypothesis we can test okay um that is kind of the big picture overview of how hypothesis testing Works in these different cases I just wanted to put them all next to each other so you could see you know it's not like we're doing totally different things it's the same procedure you have a hypothesis a null hypothesis the opposite of this is our alternative hypothesis you build a test statistic from data and different test statistics you know depending on what we know and what we don't know what we're testing but you build a test statistic that test statistic has some distribution that we know and then what you do is in that distribution normal student t or ki^ squar you design a rejection region based on some significance value some P value and you calculate did my test statistic lie inside or outside of this rejection region if it's inside the rejection region you reject the null hypothesis and your alternative hypothesis is likely to be true if you're not in the rejection region then you fail to reject the null hypothesis and you don't have enough evidence to say that that's this null hypothesis is

not true cool okay really nice um just a couple of facts I probably want to tell you some things about the T distribution and the Ki Square distribution I'll probably do that in the next video because I'm guessing this is already getting long um but in the next video I'll just write down a couple of properties of this distribution and this distribution I'm not going to prove anything I'm just going to give you some brief hints some trail of breadcrumbs as to where these come from what they're related to some things you should know about these distributions but this is why we're going to look at these kind of exotic distributions okay thank you