p-value again

A NY Times statistics article describes p-value in a more approachable manner than what I noted earlier. 

Today, this kind of number is called a p-value, the probability that an observed phenomenon or one more extreme could have occurred by chance. Results are usually considered “statistically significant” if the p-value is less than 5 percent.

So the null hypothesis would lead us to use a probability distribution of chance. Statistical hypothesis testing can then disprove this hypothesis by showing that the observation or something more extreme is unlikely to have occurred if the cause were random because calculating the probability based on this picked distribution shows its chances at less than 5 percent.

I believe doing it this way helps make testing the null hypothesis repeatable, but I haven't found an approachable explanation yet.