Improving statistical inferences 1_2

What is a p-value?

1. Why is the p-value so successful in science? In some sense it offers a first line of defense against being fooled by randomness, separating signal from noise. 

2. P-value tell you how surprising the data is, assuming there is no effect. 

3. e.g. The difference is never exactly zero. A difference of e.g., 0.11 means:

A) Probably just random noise.

B) Probably a real difference.

From the M, SD, and N, we calculate a test-statistic, and compare it against a distribution. (examine precognition use a paranormal distribution).

(A p-value is the probability of getting the observed or more extreme data, assuming the null hypothesis is true).

A p-value is the probability of data not the probability of a theory.

4. P(D * | H) != P(H | D)  (if you want to know the probability that the theory is true, you need to use Bayesian statistics.

5. When there is a true effect, the p-value distribution depends on the statistical power.

6. When there is no effect, p-value are uniformly distributed