Conditional Probability

Sometimes, we may have some additional information about the outcomes of an experiment, which could allow us to restrict the set of possible outcomes. For a simple example, if you know the outcome of a roll of the dice is an even number, you can remove the odd numbers from the list of possibilities. This additional information changes what the sample space under consideration is and reduces the number of possibilityes. The new probabilities that take into account the additional information are called conditional probabilities.

Definition: Suppose $A$ and $B$ are events with $P(B) \ne 0$. The probability of $A$ conditioned on $B$ is $$P(A|B) = \frac{P(A\cap B)}{P(B)}.$$ We can also read this as 'The probability of $A$ given $B$.'

Once we know that the event $B$ has occured, the sample space (which is the space of all possible outcomes) collapses from $\mathcal{S}$ to $B$. So the probability of $A$ in the new sample space can be thought of as the 'size' of $A$ relative to the size of $B$.

Example: You roll two six sided dice. If the sum of the two numbers is 6, what is the probability that one of the dice rolled was a 2?

Definition: A partition of a set $\mathcal{S}$ is a collection of subsets of $\mathcal{S}$, $E_1,...,E_n$ that are mutually disjoint ($E_i \cap E_j = \varnothing$ if $i\ne j$) and satisfy $$E_1 \cup E_2 \cup ... \cup E_n = \mathcal{S}.$$

For example, if $\mathcal{S} = \{1,2,3,4,5,6\}$ then the sets $E_{odd} = \{1,3,5\}$ and $E_{even} = \{2,4,6\}$ are a partition of $\mathcal{S}$.

The Law of Total Probability: If $\mathcal{S}$ is a sample space and $E_1,...,E_n$ is a partition of $S$, then for any event $A$ we have $$ P(A) = \sum_{j=1}^n P(A|E_j)P(E_j)$$

Example: Billy is a building inspector. During August he inspected 30 residential buildings, 20 commercial buildings and 10 industrial buildings. 5 residential buildings had code violations, 2 commercial buildings had code violations, and 3 industrial buildings had code violations. If Billy follows up with one of the damaged buildings at random, what is the probability that building is a commercial property?

Our sample space in this problem is to 60 buildings that Billy visited over the month of August. Let $E_V$ be the event that a building has a code violation, $E_C$ be the event that a building is a commercial property, $E_R$ the event it is a residential property, and $E_I$ be the even it is an industrial property. The events $E_C,E_R,$ and $E_I$ are a partition of the sample space. \begin{align} P(E_V) &= P(E_V|E_C)P(E_C)+P(E_V|E_R)P(E_R)+P(E_V|E_I)P(E_I)\\ &= \frac{2}{20}\frac{20}{60} + \frac{5}{30}\frac{30}{60}+\frac{3}{10}\frac{10}{60}\\ & = \frac{1}{6} \end{align} Therefore the probability that it was a commercial building, given there was a code violation is \begin{align} P(E_C|E_V) &= \frac{P(E_V\cap E_C)}{E_V}\\ & = \frac{\frac{2}{60}}{\frac{1}{6}}\\ & = \frac{1}{5} \end{align}

It's often not easy to calculate $P(A\cap B)$. In fact, in some cases it may be easy to calculate the conditional probability $P(A|B)$ by examining the sets. In the previous example, it was easy to see $P(E_V|E_C)=2/20$, however $P(E_V\cap E_C)$ and $P(E_C|E_V)$ were not as obvious. Fortunately, there is a way to manipulate conditional probabilities so we only have to calculate the easiest versions of them.

Bayes' Theorem: Suppose $A$ and $B$ are events with $P(B) \ne 0$. Then $$P(A|B) = \frac{P(B|A)P(A)}{P(B)}.$$

Example: Suppose I have three unmarked boxes. Each box has two coins in it. In one box, there are two loonies, in the next box there is one loonie and one toonie, and in the final box there are two toonies. Suppose you choose a coin at random from one of the boxes and it is a toonie. What is the probability the other coin is also a toonie?

Exercises

  • Suppose you roll two dice.
    1. You can see that one of the dice is 2. What is the probability that the two dice sum up to 7?
    2. Both dice sum to 4. What is the probability that both dice are even?
    3. Both dice are odd. What is the probability that both dice are the same number?
  • AA batteries for sale at the dollar store have quality control problems. In an arbitrarily chosen pack of 4 batteries, there is a 10% chance that exactly one battery is dead, a 5% chance that exactly two are dead, and a 1% chance that three are dead, in the remainder of packs, all of the batteries work. Suppose you bought a pack and the first battery you checked worked. What is the probability that all of the remaining batteries work?
  • A family has two children. One of these children is a boy who was born on a Tuesday. What is the probability that the other child is also a boy? The answer is not 1/2. (Assume that "boy" and "not boy" are equally likely, and that there is an equal probability to be born on each day of the week.)