Friday, October 25, 2013

Quick Note about P-value Confusion

Lately I've been working on learning about p-values and hypothesis testing. I'm still learning introductory statistics and am having trouble keeping the idea so I am going to note it here for future reference. A comment with a mnemonic at cross validated is "p is low, H0 must go". At this point, I believe that means the probability of seeing the data assuming that the null hypothesis is true is too low to believe. It is highly unlikely we would see this data if the original assumption was correct, and so we should reject the null hypothesis. I hope I have this in the right order and that it is not always the probability of the null hypothesis being true given the data collected. The wikipedia page has a list of misunderstandings and criticisms of the p-value that I hope to check out later. Based on a forum post, to interpret p-value again, if the null hypothesis is true then p percent of experiments should show a test statistic that is the same or more extreme than what was collected with the current experiment being analyzed. Hence by repeating the experiment one can become more confident that the data from a previous experiment was not an extreme event. Once more, based on the wikipedia page, the p-value is the probability of observing the test statistic or something more extreme.