The difference in the variable must first be of a certain size before we
have evidence of causality.
Second, the size of that difference depends on the size of our
Finally, we are still only dealing with probabilities. Even with
a large sample, there is some chance, however tiny, that in our
study relating a cram course with SAT scores, random process
assigned all the geniuses to the experimental group.
So if the difference between the two groups is x, then to be
precise we should state our result as follows: there is only a y
percent probability that an x point difference could occur by
chance, given the size of the groups. Most researchers consider
a result statistically significant if y is less than 5 percent.