We are now in a position to define what we mean by "the probability of an event". In essence, a probability is a way for us to compare the different sizes of events relative to the sample space $\mathcal{S}$.

Definition: A probability measure is a function $P:\mathcal{A} \to \mathbb{R}$ from an even space $\mathcal{A}$ to the real numbers that satisfies

1. The probability of the full set is 1: $P(\mathcal{S}) = 1$
2. If $E\in \mathcal{A}$ then $P(E)\ge 0$
3. For any $E_1,E_2\in \mathcal{A}$ with $E_1\cap E_2 = \varnothing$, we have $P(E_1 \cup E_2) = P(E_1)+P(E_2)$

Probability measures give a numerical method for quantifying how likely an event is, or the chance that some event occurs.

It's often helpful to think of the probability of an event as the area of an event relative to the area of the sample space.

If we draw Venn diagrams we can visualize each of the properties fairly easily.

More Properties:

1. If $E$ is an event, then $P(E^c) = 1-P(E)$. This is true since $E^c \cap E =\varnothing$ and $E^c \cup E = \mathcal{S}$. So by property III of the definition of probability, we have $1 = P(\mathcal{S}) = P(E^c \cup E) = P(E_c)+P(E)$. Rearranging this gives $P(E^c) = 1-P(E)$. We can also see this pretty readily by inspecting a Venn diagram.
2. $P(\varnothing) = 0$. This is true since $\mathcal{S}\cup \varnothing = \mathcal{S}$ and $\mathcal{S}\cap \varnothing = \varnothing$. So by property III of the definition we have $P(\mathcal{S}) = P(\mathcal{S}\cup \varnothing) = P(\mathcal{S})+P(\varnothing)$. Rearranging this gives $P(\varnothing) = 0$. This also intuitively makes sense, since the empty set should have no area.
3. If $E_1$ and $E_2$ are events then $P(E_1 \cup E_2) = P(E_1)+P(E_2) - P(E_1\cap E_2).$
4. If $E_2\subseteq E_1$ then $P(E_1 \setminus E_2) = P(E_1) - P(E_2)$.

Events with Equally Likely Outcomes

For some experiments, every outcome in the sample space has the same probability. This is not true for every experiment, or even most experiments. A meme misunderstanding of probability is that there is always a 50% chance of rain every day. Clearly this is not correct, since the two different outcomes "it will rain" and "it will not rain" have different probabilities. However for some simple experiments, such as rolling dice, picking playing cards, or flipping coins, every outcome has an equal probability. In these experiments, if there are $n$ outcomes, the probability of each outcome is $1/n$.

Example: Consider a simple experiment where we flip a coin. There are two outcomes to this experiment (discounting the highly improbable chance that the coin lands on its edge) heads ($H$) and tails ($T$). If the coin is fair, then there is an equal probability of either outcome occuring. Since there are 2 outcomes we have $$ P(H) = \frac{1}{2},\qquad P(T) = \frac{1}{2}.$$

Example: Consider a simple experiment where a fair 6-side die is rolled. What is the probability that:

1. A 6 is rolled?
2. An even number is rolled?

Solutions: (click to show)

Problem: Consider a simple game where you roll two dice. You win a dollar if the sum of both dice is 7, you win two dollars if both dice are sixes, and you lose a dollar if both dice are ones.

1. What is the event space for this game?
2. What is the probability of winning something?
3. What is the probability of losing a dollar?
4. what is the probability of winning exactly one dollar?

Exercises:

• Use Demorgan's laws to show that $P(A^c\cup B^c) = 1 - P(A\cap B)$. Confirm this using a venn diagram.
• The function $P(\{n\}) = \frac{1}{2^{n+1}}$ defines a probability measure on $\mathcal{S} = \mathbb{N} = \{0,1,2,3,...\}$. Use the properties of probabilites to find the value of $$ \sum_{j=2}^{\infty} \frac{1}{2^{n+1}} = \frac{1}{8} + \frac{1}{16} +\frac{1}{32} + ...$$
• Hex nuts come in packs of 14. Quality control made a mistake and mixed three metric 6mm nuts into a pack of imperial 1/4" nuts. What is the probability that a randomly selected nut from this pack is 6mm?