The Basic Diagram

The Diagram

This diagram is central to the whole story. Any point on the floor corresponds to a combination of population effect size (front to back) and sample effect size (left to right). The vertical axis shows the probability density or likelihood (depending on context) for that combination of population and sample effect size.

This figure shows the set of sample effect sizes, $r_s$ that are expected from a population with effect size $r_p=0$ for a random sample with $n=42$ .

This figure shows the set of sample effect sizes, $r_s$ that are expected from a population with effect size $r_p=0$ for a random sample with $n=42$ .

Sampling Distribution

The distribution shown in the first figure is a fact called the sampling distribution. It is exactly the distribution of $r_s$ that would be obtained with an arbitrarily large set of samples, given $r_p=0$ and $n=42$ . The assumption that it requires is that the samples are properly randomly drawn from the population.

The probability of getting a sample effect size that is exactly some specified value is zero – because there is an infinite number of possible values you could get. But the probability of a sample effect size lying between two specified values is not 0. In fact it is given by how much of the sampling distribution lies between the two specified values. To get that probability we calculate the area under the probability density curve between the two specified values. To mark this, the vertical axis is “Probability Density”.

Probability is a term that we will reserve for unknown, future events. We can talk about the probability that a particular sample will come by sampling a specific population.

Likelihood Function

The next figure shows another distribution, but drawn at right angles to the sampling distribution. This is called a likelihood function. The vertical axis is now likelihood. The distribution shows the likelihood that our sample $r_s=0.25$ came from different population effect sizes.

This version of the figure shows the likelihood of different population effect sizes for a given sample effect size.

Likelihood is a term that we will use to refer to inferences from known events that have happened. We can talk about the likelihood that a particular sample came from a specific population.

Probability and Likelihood

Likelihood and probability are terms that are used to talk about outcomes. We will use probability to refer to unknown future outcomes with known causes; likelihood refers to known past outcomes with unknown sources. In each case, the term relates to what we expect of the unknown quantity – outcome or source.

There is one very important technical difference. Probability has an absolute meaning: the sum of the probabilities of all the possible (mutually exclusive) outcomes must equal 1; or equivalently, the sum of a probability density function has to equal 1. Likelihood doesn’t have to do that. Because of that, likelihood has no absolute meaning, unlike probability. It does have a relative meaning: we can say that one population is more likely than another.

An illustration:

If I ask you to randomly choose any integer number between 1 and 100, then I can say that the probability that you will choose 42 is 1/100.
If I ask you to randomly choose any number (without limit), then we have a problem. There is an infinite range of numbers, so the probability that you will choose 42 is 1/infinity=0.
- Whatever number you end up choosing, the probability that you will do so is zero.
In both cases, the relative likelihood of you choosing 42, compared with say, 66, is the same. Those two outcomes are equally likely and we would say that their likelihood ratio is 1.