Big Idea #1
Two events are independent if knowing that one event occurred doesn't change the probability that the other occurred.
Example: Toss a coin. Knowing that heads came up on the first toss change the probability of heads coming up on the second toss. These two events are independent.
Example: Draw two cards from a standard, well-shuffled deck without replacement. Knowing that an ace was drawn first changes the probability drawing a second ace. These two events are NOT independent.
Example: On the other hand two cards from a standard, well-shuffled deck with replacement. Knowing that an ace was drawn first does not change the probability drawing another ace. These two events are independent.
Sampling With and Without Replacement
When we repeatedly sample from a population with replacement, each individual outcome is independent of the other. For example if we draw a card from a deck, put the card back, shuffle again, draw another card, each draw is independent.
On the other hand, if we sample repeatedly without replacement, the individual outcomes are NOT independent. For example, if we draw a card and do not put it back in the deck, the next outcome is influenced by the previous.
Definition: Two events $A$ and $B$ are independent if $$P(A|B)=P(A).$$ This definition has three equivalent variations...
Independent Events
The events $A$ and $B$ are independent if and only if:
1) $P(A|B)=P(A)$
2) $P(B|A)=P(B)$
3) $P(A \cap B)=P(A) \cdot P(B)$
That is, if you can show that either 1), 2), or 3) hold, then $A$ and $B$ are independent.
Suppose $U$ and $V$ are independent events where $P(U)=0.7,$ and $P(V)=0.2.$
Find $P(U \cap V).$
$$
\begin{array}{lll}
\displaystyle P(U \cap V)&\displaystyle=P(U)\cdot P(V) &\mbox{since $U$ and $V$ independent}\\
\displaystyle &\displaystyle=0.7\cdot 0.2 &\mbox{}\\
\displaystyle &\displaystyle=0.14 &\mbox{}\\
\end{array}
$$
Example: A student goes to the library.
Let events $B$ = the student checks out a book
Let $D$ = the student checks out a DVD.
Then, $B \cap D$ = the student checks out a book and checks out a DVD.
Suppose that $P(B) = 0.40,$ $P(D) = 0.30,$ and $P(B \cap D) = 0.20.$
Are $B$ and $D$ independent?
If $B$ and $D$ were independent, then
$$
\begin{array}{lll}
P(B \cap D) &= P(B)\cdot P(D) &\mbox{assuming independence}\\
&=0.4\cdot 0.3&\\
&=0.12&\\
\end{array}
$$
But $P(B \cap D) = 0.20$ as given above.
We conclude that $B$ and $D$ are NOT independent.
We conclude that $B$ and $D$ are NOT independent.
Other Examples
Flip three fair coins. What is the probability of getting at least one tail?
Method 1: List all the outcomes in the sample space.
Method 2: Think complementary events.
Method 1
Let $A$ be the event of "getting at least one tail."
The possibilities for three coins can be represented as the sample space $$ S=\left\{\mbox{HHH},\mbox{THH},\mbox{HTH},\mbox{HHT},\mbox{TTH},\mbox{THT},\mbox{HTT},\mbox{TTT}\right\} $$ We see that the event, "getting at least one tail" is $7$ possibilities out of $8.$ That is, $$ P(A)=\frac{7}{8} $$
Method 2
Let $T_1,$ $T_2,$ and $T_3$ be events of the first, second, and third coins coming up tails.
The event of "at least one coin comes up tails" can be represented as $A=T_1 \cup T_2\cup T_3.$
This is not an easy probability. On the other hand, let's consider its compliment: "none of the coins come up tails."
That is, "all three coins come up heads." We may represent this complementary event as $$ A'=H_1 \cap H_2\cap H_3 $$ where $H_1,$ $H_2,$ and $H_3$ are the events of the first, second, and third coins come up heads.
We may now use the rule for complimentary events to compute the probability. $$ \begin{array}{lll} \displaystyle P(T_1 \cup T_2\cup T_3)&\displaystyle=P(A) &\mbox{}\\ \displaystyle &\displaystyle=1-P(A') &\mbox{}\\ \displaystyle &\displaystyle=1-P(H_1 \cap H_2\cap H_3) &\mbox{}\\ \displaystyle &\displaystyle=1-P(H_1)\cdot P(H_2)\cdot P(H_3) &\mbox{using independence of each coin from the other}\\ \displaystyle &\displaystyle=1-\frac{1}{2}\cdot\frac{1}{2}\cdot\frac{1}{2} &\mbox{}\\ \displaystyle &\displaystyle=1-\frac{1}{8} &\mbox{}\\ \displaystyle &\displaystyle=\frac{7}{8} &\mbox{}\\ \end{array} $$
Let $A$ be the event of "getting at least one tail."
The possibilities for three coins can be represented as the sample space $$ S=\left\{\mbox{HHH},\mbox{THH},\mbox{HTH},\mbox{HHT},\mbox{TTH},\mbox{THT},\mbox{HTT},\mbox{TTT}\right\} $$ We see that the event, "getting at least one tail" is $7$ possibilities out of $8.$ That is, $$ P(A)=\frac{7}{8} $$
Method 2
Let $T_1,$ $T_2,$ and $T_3$ be events of the first, second, and third coins coming up tails.
The event of "at least one coin comes up tails" can be represented as $A=T_1 \cup T_2\cup T_3.$
This is not an easy probability. On the other hand, let's consider its compliment: "none of the coins come up tails."
That is, "all three coins come up heads." We may represent this complementary event as $$ A'=H_1 \cap H_2\cap H_3 $$ where $H_1,$ $H_2,$ and $H_3$ are the events of the first, second, and third coins come up heads.
We may now use the rule for complimentary events to compute the probability. $$ \begin{array}{lll} \displaystyle P(T_1 \cup T_2\cup T_3)&\displaystyle=P(A) &\mbox{}\\ \displaystyle &\displaystyle=1-P(A') &\mbox{}\\ \displaystyle &\displaystyle=1-P(H_1 \cap H_2\cap H_3) &\mbox{}\\ \displaystyle &\displaystyle=1-P(H_1)\cdot P(H_2)\cdot P(H_3) &\mbox{using independence of each coin from the other}\\ \displaystyle &\displaystyle=1-\frac{1}{2}\cdot\frac{1}{2}\cdot\frac{1}{2} &\mbox{}\\ \displaystyle &\displaystyle=1-\frac{1}{8} &\mbox{}\\ \displaystyle &\displaystyle=\frac{7}{8} &\mbox{}\\ \end{array} $$
Other Examples
Flip $10$ fair coins. What is the probability of getting at least one tail?
Method 1: List all the outcomes in the sample space. (Not recommended in this case.)
Method 2: Think complementary events. (Recommended!)
Method 1
No one in their right mind would list out all the $2^{10}=1024$ possibilities of the sample space.
Method 2
Let $A$ be the event of "getting at least one tail."
Altos, let $T_1,$ $T_2,\ldots, T_{10}$ be events of the first through tenth coins coming up tails.
The event of "getting at least one tail" can be represented as $A=T_1 \cup T_2\cup \cdots \cup T_{10}.$
This is not an easy probability. On the other hand, let's consider its compliment: "no coin comes up tails."
That is, "all ten coins come up heads." We may represent this complementary event as $$ A'=H_1 \cap H_2\cap \cdots \cap H_{10} $$ where $H_1,$ $H_2,\ldots, H_{10}$ are the events of the first through tenth coins coming up heads.
We may now use the rule for complimentary events to compute the probability. $$ \begin{array}{lll} \displaystyle P(T_1 \cup T_2\cup \cdots \cup T_{10})&\displaystyle=P(A) &\mbox{}\\ \displaystyle &\displaystyle=1-P(A') &\mbox{}\\ \displaystyle &\displaystyle=1-P(H_1 \cap H_2\cap \cdots \cap H_{10}) &\mbox{}\\ \displaystyle &\displaystyle=1-P(H_1)\cdot P(H_2)\cdot \cdots \cdot P(H_{10}) &\mbox{using independence of flips}\\ \displaystyle &\displaystyle=1-\underbrace{\frac{1}{2}\cdot\frac{1}{2}\cdot \cdots \cdot\frac{1}{2}}_{10 factors} &\mbox{}\\ \displaystyle &\displaystyle=1-\left(\frac{1}{2}\right)^{10} &\mbox{}\\ \displaystyle &\displaystyle\approx 0.9990234375 &\mbox{}\\ \end{array} $$
No one in their right mind would list out all the $2^{10}=1024$ possibilities of the sample space.
Method 2
Let $A$ be the event of "getting at least one tail."
Altos, let $T_1,$ $T_2,\ldots, T_{10}$ be events of the first through tenth coins coming up tails.
The event of "getting at least one tail" can be represented as $A=T_1 \cup T_2\cup \cdots \cup T_{10}.$
This is not an easy probability. On the other hand, let's consider its compliment: "no coin comes up tails."
That is, "all ten coins come up heads." We may represent this complementary event as $$ A'=H_1 \cap H_2\cap \cdots \cap H_{10} $$ where $H_1,$ $H_2,\ldots, H_{10}$ are the events of the first through tenth coins coming up heads.
We may now use the rule for complimentary events to compute the probability. $$ \begin{array}{lll} \displaystyle P(T_1 \cup T_2\cup \cdots \cup T_{10})&\displaystyle=P(A) &\mbox{}\\ \displaystyle &\displaystyle=1-P(A') &\mbox{}\\ \displaystyle &\displaystyle=1-P(H_1 \cap H_2\cap \cdots \cap H_{10}) &\mbox{}\\ \displaystyle &\displaystyle=1-P(H_1)\cdot P(H_2)\cdot \cdots \cdot P(H_{10}) &\mbox{using independence of flips}\\ \displaystyle &\displaystyle=1-\underbrace{\frac{1}{2}\cdot\frac{1}{2}\cdot \cdots \cdot\frac{1}{2}}_{10 factors} &\mbox{}\\ \displaystyle &\displaystyle=1-\left(\frac{1}{2}\right)^{10} &\mbox{}\\ \displaystyle &\displaystyle\approx 0.9990234375 &\mbox{}\\ \end{array} $$
Big Idea #2
Two events are mutually exclusive if they have no outcomes in common.
Example: Suppose you roll a die. The events "roll an even number" and "roll an odd number" are mutually exclusive since these events have no outcomes in common.
Definition: Two events $A$ and $B$ are mutually exclusive if $$P(A \cap B)=0.$$ Example: Suppose you roll a die. The events $A$="roll an even number" and $B$="roll an odd number" cannot occur simultaneously. That is $P(A \cap B)=0.$
Fact: If $A$ and $B$ are mutually exclusive events, then $$P(A \cup B)=P(A)+P(B).$$
Example: Suppose $U$ and $V$ are mutually exclusive events with $P(U)=0.4,$ and $P(V)=0.5.$ Find the following probabilities:
(a) $P(U \cap V)$
(b) $P(U|V)$
(c) $P(U \cup V)$
(a) Since $U$ and $V$ are are mutually exclusive events, it follows by definition that
$$
P(U \cap V)=0.
$$
(b) $P(U|V)$ is the probability of event $U$ occurring given that $V$ has occurred.
Since $U$ and $V$ cannot occur simultaneously (they are are mutually exclusive events), we have $$ P(U|V)=0. $$ Scenic Alternative $$ \begin{array}{lll} \displaystyle P(U|V)&\displaystyle=\frac{P(U \cap V)}{P(V)} &\mbox{definition of conditional probability}\\ \displaystyle &\displaystyle= \frac{0}{P(V)}&\mbox{since $U$ and $V$ are are mutually exclusive}\\ \displaystyle &\displaystyle=0&\mbox{}\\ \end{array} $$ (c) Yet again, since $U$ and $V$ are are mutually exclusive events, it follows that $$ \begin{array}{ll} P(U \cup V)&=P(U)+P(V)\\ &=0.4+0.5\\ &=0.9\\ \end{array} $$
Since $U$ and $V$ cannot occur simultaneously (they are are mutually exclusive events), we have $$ P(U|V)=0. $$ Scenic Alternative $$ \begin{array}{lll} \displaystyle P(U|V)&\displaystyle=\frac{P(U \cap V)}{P(V)} &\mbox{definition of conditional probability}\\ \displaystyle &\displaystyle= \frac{0}{P(V)}&\mbox{since $U$ and $V$ are are mutually exclusive}\\ \displaystyle &\displaystyle=0&\mbox{}\\ \end{array} $$ (c) Yet again, since $U$ and $V$ are are mutually exclusive events, it follows that $$ \begin{array}{ll} P(U \cup V)&=P(U)+P(V)\\ &=0.4+0.5\\ &=0.9\\ \end{array} $$