Probability

This page provides an introduction to Probability.

Overview

Probability is a measure of the likelihood or chance that a specific event will occur. It ranges from 0 to 1. Let's understand some of the terminlogies of probabilities as below:

Random Experiment

Random experiment is the experiment whose outcome is known.

Sample Space

Sample space is set of all possible outcomes. For e.g. throwing a dice has sample space as below, and it is denoted by $\text{S}$:

$$\text{S} = \{1, 2, 3, 4, 5, 6\}$$

Event

Event is Subset of sample space, and it is denoted by $\text{E}$.

Equally Likely Events

Equally likely events are those who has equal chances of occurence or have equal probability.

$$\text{P(E}_1) = \text{P(E}_2)$$

Mutually Exclusive Events

Mutually exclusive are those whose simultaneous occurance is impossible. It is also known as Disjoint or Incompatible events.

$$\text{E}_1 \cap \text{E}_2 = \phi$$

Mutually Exhaustive Events

Mutually exhaustive events are those whose union is sample space.

$$\text{E}_1 \cup \text{E}_2 = \text{S}$$

Classic Probability

If a random experiment has total $\text{(m + n)}$ mutually exclusive and equally likely outcomes, out of which $\text{m}$ favours event $\text{E}$, then probability of occurrence of Event $\text{E}$ is denoted by $\text{P(E)}$.

$$\text{P(E)} = \frac{\text{favourable outcomes}}{\text{total outcomes}} = \frac{\text{m}}{\text{m+n}}$$

info

When outcomes are not equally likely $\text{P(E)} = \text{Sum of probability of favourable outcomes}$.

Below are some of the important characterstics of probability:

• Sum of probabilities of each sample point or probability of sample space $= 1$.

• If $\text{P(E)} = 0,$ then event $\text{E}$ is impossible event.

• If $\text{P(E)} = 1,$ then event $\text{E}$ is sure event.

• Probability of non-occurrence of an event is:

$$\text{E} = \text{P}(\overline{\text{E}}) = 1 - \text{P(E)}$$

• Formula for odds in favour of any event is:

$$\text{E} = \frac{\text{Number of favourable outcome}}{\text{Number of unfavourable outcome}}$$

• Formula for odds against event is:

$$\text{E} = \frac{\text{Number of unfavourable outcome}}{\text{Number of favourable outcome}}$$

Example 1

A fair dice is rolled, calculate the probability of getting a prime number.

Solution

There are total three prime numbers on the fair dice ${2, 3, 5}$ in a sample set $\{1, 2, 3, 4, 5, 6\}$ for throwing a dice. So probability of getting a prime number will be:

$$\text{P(E)} = \frac{3}{6} = \frac{1}{2}$$

Example 2

Two coins are tossed simultaneously, what is the probability of getting at least one head.

Solution

Cases where we will get at least one head when two coins are tossed will be:

• Case 1: H H
  
• Case 2: H T
  
• Case 3: T H
  

So total probability of getting at least one head when two coin are tossed will be:

$$\text{P(E)} = \frac{3}{4}$$

Example 3

A dice is rolled, if it shows prime number then a coin is tossed, what is the probability of getting head.

Solution

Let's write the sample set first:

$$\text{S} = \{ (1), (2, \text{H}), (2, \text{T}), (3, \text{H}), (3, \text{T}), (4), (5, \text{H}), (5, \text{T}), (6) \}$$

Now, let's write probability of each of the event:

$\text{P(1)} = \large\frac{1}{6}$

$\text{P(2, H)} = \large\frac{1}{6} * \frac{1}{2}$

$\text{P(2, T)} = \large\frac{1}{6} * \frac{1}{2}$

$\text{P(3, H)} = \large\frac{1}{6} * \frac{1}{2}$

$\text{P(3, T)} = \large \frac{1}{6} * \frac{1}{2}$

$\text{P(4)} = \large\frac{1}{6}$

$\text{P(5, H)} = \large \frac{1}{6} * \frac{1}{2}$

$\text{P(5, T)} = \large \frac{1}{6} * \frac{1}{2}$

$\text{P(6)} = \large\frac{1}{6}$

As in this case all the events in the sample space doesn't have equally likely probability, we apply formula below:

$$\text{P(E)} = \text{Sum of probability of favourable outcomes}$$

So probability of getting a head on coin toss after rolling a prime number on a die will be:

$$\text{P(E)} = \frac{1}{4}$$

Example 4

An urn contains nine balls of which three are red, four are blue and two are green. Three balls are drawn at random without replacement from the urn. calculate the probability that three balls have different color.

Solution

Number of ways we can draw $3$ balls from $9$ balls without replacement will be:

$$^9\text{C}_3$$

Number of ways we can draw $3$ balls from $9$ balls and all of them are of different color will be:

$$^3\text{C}_1 \ \ * \ \ ^4\text{C}_1 \ \ * \ \ ^2\text{C}_1$$

Let's apply classic probability formula as drawing a ball from urn has equal probability. So probability that three balls have different colors will be:

$$\frac{3 \ \ * \ \ 4 \ \ * \ \ 2}{^9\text{C}_3} = \frac{2}{7}$$

Example 5

Two numbers are selected randomly from the set $\text{S} = \{1, 2, 3, 4, 5, 6\}$ without replacement one by one. Calculate the probability that minimum of the two numbers is less than $4$.

Solution

Let's generate the cases for favourable outcomes.

Case 1: Both of them are less than $4$.

$$^3\text{C}_2$$

Case 2: One of them is less than $4$.

$$^3\text{C}_1 \ \ * \ \ ^3\text{C}_1$$

So probability that minimum of the two numbers is less than $4$ will be:

$$\frac{(^3\text{C}_2 \ \ + \ ^3\text{C}_1 \ \ * \ \ ^3\text{C}_1)}{^6\text{C}_2}$$

Example 6

Three boys and two girls stand in a queue. Calculate the probability such that the number of boys ahead of every girl is at least one more than the number of girls ahead of her.

Solution

This problem can't be solved with permutation and combinations as this is not asking you to just arrange $2$ boys across $3$ girls, but there is a condition attached to it which say number of boys ahead of every girl is at least one.

Let's make cases for favourable outcomes.

• Case 1: $\text{B}_1\text{B}_2\text{B}_3\text{G}_1\text{G}_2$
• Case 2: $\text{B}_1\text{B}_2\text{G}_1\text{G}_2\text{B}_3$
• Case 3: $\text{B}_1\text{B}_2\text{G}_1\text{B}_3\text{G}_2$
• Case 4: $\text{B}_1\text{G}_1\text{B}_2\text{G}_3\text{B}_3$
• Case 5: $\text{B}_1\text{G}_1\text{B}_2\text{B}_3\text{G}_2$

Now, $3$ boys can arrange among themself in $3!$ ways and $2$ girls can arrange among themself in $2!$ ways, it gives us total number of favourable outcomes as $5 * 2! * 3!$.

So probability such that the number of boys ahead of every girl is at least one more than the number of girls ahead of her will be:

$$\frac{5 \ \ * \ \ 2! \ \ * \ \ 3!}{5!}$$

Infinite Trials

In probability theory, infinite trials refer to a situation where an experiment or process is repeated an infinite number of times.

Below are some of the examples of infinite trials:

Example 1

$\text{A}$ and $\text{B}$ inorder throws a dice, whoever gets $6$ first wins the game, find the probabiliy that $\text{A}$ wins.

Solution

$$\text{P}(\text{AWins}) = \text{P(A)} \ \ + \ \ [\text{P}(\overline{\text{A}}) * \text{P}(\overline{\text{B}}) * \text{P(A)}] \ \ + \ \ [\text{P}(\overline{\text{A}}) * \text{P}(\overline{\text{B}}) * \text{P}(\overline{\text{A}}) * \text{P}(\overline{\text{B}}) * \text{P(A)}] \ \ + \ \ ... \infty$$

Where,

$\text{P}(\text{AWins})$ is Probability that $\text{A}$ wins the game.

$\text{P(A)}$ is Probability that $\text{A}$ gets $6$.

$\text{P}(\overline{\text{A}})$ is Probability that $\text{A}$ does not gets $6$.

$\text{P(B)}$ is Probability that $\text{B}$ gets $6$.

$\text{P}(\overline{\text{B}})$ is Probability that $\text{B}$ does not gets $6$.

Substituting the values in the formula above we get:

$$\text{P}(\text{AWins}) = \frac{1}{6} \ \ + \ \ [\frac{5}{6} * \frac{5}{6} * \frac{1}{6}] \ \ + \ \ [\frac{5}{6} * \frac{5}{6} * \frac{5}{6} * \frac{5}{6} * \frac{1}{6}] \ \ + \ \ ... \ \ \infty$$

Simplifying equation above we get:

$$\text{P}(\text{AWins}) = \frac{1}{6} \ \ * \ \ [1 + (\frac{5}{6})^{2} + (\frac{5}{6})^{4} + \ \ ... \ \ \infty]$$

Applying formula for sum of infinte geometric progression we get:

$$\text{P}(\text{AWins}) = \frac{1}{6} * \frac{1}{1 \ \ - \ \ \large(\frac{5}{6})^{2}}$$

Example 2

The probability of a man hitting a target is $\large\frac{1}{10}$. Calculate the least number of shots required such that probability of man hitting the target at least once is greater than $\large\frac{1}{4}$.

Solution

Since we want to calculate probability of hitting the target atleast one, we apply below formula:

$$\text{P(Hitting atleast once) = Total probability - P(Hitting never)}$$ $$\text{P(Hitting at least once)} = 1 - (\frac{9}{10})^\text{n}$$

Now, we want $\text{P(Hitting target at least once in n attempts)}$ to be $> \frac{1}{4}$, so we get:

$$1 - (\frac{9}{10})^\text{n} > \frac{1}{4}$$

Simplifying the equation further we get:

$$10^\text{n} * 3 > 9^\text{n} * 4$$

Now, we can use hit and trial method for value of $\text{n}$ which satisfy the equation and get $\text{n} = 3$.

So least number of short required such that probability of man hitting the target at least once is greater than $\large\frac{1}{4}$ will be $3$.

Example 3

Calculte the minimum number of times a fair coin needs to be tossed, such that probability of getting atleast two heads is atleast $0.96$?

Solution

Let's say we have to toss the coin $\text{n}$ times so that probability of getting atleast two heads is atleast $0.96$.

Since we want to calculate probability of getting at least two heads, we apply below formula:

$$\text{P(Atleast 2 heads) = Total probability - P(Exactly 0 head) - P(Exactly 1 head)}$$

Now,

$$\text{P(Exactly 0 head)} = \ \ \frac{1}{2}, \ \ \text{and} \ \ \text{P(Exactly 1 head)} = \ \ ^\text{n}\text{C}_1 * (\frac{1}{2})^{\text{n}}$$ $$$$

info

Here we have $^\text{n}\text{C}_1$ because we want exactly 1 head but we still have to select a location of head in $\text{n}$ attempts.

Substituting the values in the formula we get:

$$\text{P(Atleast 2 heads)} = 1 - (\frac{1}{2})^{\text{n}} \ \ - \ \ ^\text{n}\text{C}_1 * (\frac{1}{2})^{\text{n}}$$

Now, we want $\text{P(Atleast 2 heads)}$ should be atleast $0.96$.

$$1 - (\frac{1}{2})^{\text{n}} \ \ - \ \ ^\text{n}\text{C}_1 * (\frac{1}{2})^{\text{n}} \ \ > \ \ 0.96$$

By hit and trial method we will get the value of $\text{n}$ as $8$. So minimum number of times a fair coin needs to be tossed will be $8$.

Set Theory

Below are some of the important points for set theory related to probability:

• Probability of $\text{A}$ and $\text{B} = \text{P}(\text{A} \cap \text{B}) = \text{P(AB)}$

• Probability of $\text{A}$ or $\text{B} = \text{P}(\text{A} \cup \text{B}) = \text{P(A + B) = P(A) + P(B)} - \text{P}(\text{A} \cap \text{B})$

• If $\text{A}$ and $\text{B}$ are mutually exclusive $= \text{P}(\text{A} \cap \text{B}) = 0$

• Probability of non-occurance of $\text{A} = \text{P}(\overline{\text{A}}) = 1 - \text{P(A)}$

• If $\text{P}(\text{A} \cap \text{B}) = \text{P(A) . P(B)}$ then $\text{A}$ and $\text{B}$ are independent events.

• If $\text{A}$ and $\text{B}$ are independent events then all possible combinations like $(\overline{\text{A}}, \text{B})$ Or $(\text{A}, \overline{\text{B}})$ etc are also independent events.

• Independent events and mutually exclusive events are two different things.

  • If $\text{A}$ and $\text{B}$ are independent events, then $\text{P}(\text{A} \cap \text{B}) = \text{P(A) . P(B)}$
  • If $\text{A}$ and $\text{B}$ are mutually exclusive events, then $\text{P}(\text{A} \cap \text{B}) = 0$

Conditional Probability

Conditional probability is the probability of an event occurring, given that another event has already occurred.

$$\text{P}(\frac{\text{A}}{\text{B}}) = \frac{\text{P}(\text{A} \cap \text{B})}{\text{P(B)}}$$

It means calculate $\text{P(A)}$ given $\text{P(B)}$, i.e. sample space for calculating $\text{P(A)}$ is $\text{P(B)}$ and favourable outcomes are $\text{P}(\text{A} \cap \text{B})$.

Example 1

A unbiased dice is thrown such that event $\text{E}_1$ is an event for getting an odd number and event $\text{E}_2$ is an event for getting a prime number. Find probability of $\text{E}_1$ given $\text{E}_2$.

Solution

Number of possible outcomes for event $\text{E}_1$ is $\{1, 3, 5\}$, so $\text{P}(\text{E}_1) = \large\frac{3}{6}$.

Number of possible outcomes for event $\text{E}_2$ is $\{2, 3, 5\}$, so $\text{P}(\text{E}_2) = \large\frac{3}{6}$.

Number of possible outcomes for $\text{E}_1 \cap \text{E}_2$ is $\{3, 5\}$, so $\text{P}(\text{E}_1 \cap \text{E}_2) = \large\frac{2}{6}$.

Applying the formula for conditional probability we get:

$$\text{P}(\frac{\text{E}_1}{\text{E}_2}) = \frac{\text{P}(\text{E}_1 \cap \text{E}_2)}{\text{P}(\text{E}_2)}$$ $$\text{P}(\text{E}_1 \cap \text{E}_2) = \frac{2}{6}, \ \ \text{and} \ \ \text{P}(\text{E}_2) = \frac{3}{6}$$

So probability of $\text{E}_1$ given $\text{E}_2$ will be:

$$\text{P}(\frac{\text{E}_1}{\text{E}_2}) = \frac{2}{3}$$

Total Probability Theorem

If there is an event $\text{A}$ such that it overlaps with $\text{n}$ mutually exclusive events $\text{E}_1$, $\text{E}_2$, $\text{E}_3$ .... $\text{E}_\text{n}$.

Then total probability of event $\text{A}$ will be:

$$\text{P(A)} = \text{P}(\text{A} \cap \text{E}_1) + \text{P}(\text{A} \cap \text{E}_2) + \text{P}(\text{A} \cap \text{E}_3) \ \ + \ \ ... \ \ + \ \ \text{P}(\text{A} \cap \text{E}_\text{n})$$

Example 1

A bag contains $4$ red ball and $6$ black balls. A ball is drawn at random from the bag, it's color is observed and this ball along with two additional ball of the same colour are returned to bag. Calculate the probability such that, if a ball is drawn at random from the bag is red ball.

Solution

Let's draw possibilities mentioned in the question.

Given we drawn a red ball and replaced it with $2$ extra red balls in the first step, then probability of getting a red ball when drawn randomly will be:

$$\frac{4}{10} * \frac{6}{12}$$

Given we drawn a black ball and replaced it with $2$ extra black balls in the first step, then probability of getting a red ball when drawn randomly will be:

$$\frac{6}{10} * \frac{4}{12}$$

Now, we need to calculate the probability of drawing a red ball at random from the bag after replace of $2$ extra balls. According to the total Probability Theorem, we'll add the probabilities of getting a red ball from each path as below:

$$(\frac{6}{10} * \frac{4}{12}) \ \ + \ \ (\frac{4}{10} * \frac{6}{12})$$

So probability of getting a red ball when drawn randomly will be:

$$\frac{2}{5}$$

Bayes Theorm

From conditional probability formula we know:

$$\text{P}(\frac{\text{A}}{\text{B}}) \ \ = \frac{\text{P}(\text{A} \cap \text{B})}{\text{P(B)}}$$

It can also be re-written as:

$$\text{P}(\text{A} \cap \text{B}) = \text{P}(\frac{\text{A}}{\text{B}}) \ \ * \ \ \text{P(B)}$$

Substituting this value in the equation for total probability of event $\text{A}$ we will get:

$$\text{P(A)} = \text{P}(\frac{\text{A}}{\text{E}_1}) * \text{P}(\text{E}_1) \ \ + \ \ \text{P}(\frac{\text{A}}{\text{E}_2}) * \text{P}(\text{E}_2) \ \ + \ \ ... \ \ + \ \ \text{P}(\frac{\text{A}}{\text{E}_\text{n}}) * \text{P}(\text{E}_\text{n})$$

Bayes' theorem enables the computation of posterior probabilities, that is the probability that event $\text{A}$ was caused by event $\text{E}1$, given $\text{A}$ has occurred using below formula:

$$\text{P}(\frac{\text{E}_1}{\text{A}}) = \frac{\text{P}(\frac{\text{A}}{\text{E}_1}) * \text{P}(\text{E}_1)}{\text{P(A)}}$$

Where,

$\text{P}(\large\frac{\text{E}_1}{\text{A}}\normalsize)$, is probability that event $\text{A}$ was caused by event $\text{E}_1$.

$\text{P}(\large\frac{\text{A}}{\text{E}_1}\normalsize)$, is probability that event $\text{A}$ will occur given event $\text{E}_1$ has occurred.

$\text{P}(\text{E}_1)$, is probability of event $\text{E}_1$.

$\text{P}(\text{A})$, is total probability of event $\text{A}$.

Example 1

A letter is come from either $\text{LONDON}$ or $\text{CLIFTON}$. The postal mark on the letter legibly shows consecutive letters $\text{ON}$. Calcualte the probability that the letter has come from $\text{LONDON}$.

Solution

Let's draw possibilities mentioned in the question.

We have $\text{P}(\text{LO})$ as $\large\frac{2}{6}$ because there are $2$ consecutive places in word $\text{L}\overline{\text{ON}}\text{D}\overline{\text{ON}}$, where we have word $\text{ON}$ out of $6$ total consecutive word possible.

Similarly for $\text{CLIFTON}$ we have $\text{P}(\text{CO})$ as $\large\frac{1}{6}$.

Now, as per total probability theorem probabilty that letter shows $\text{ON}$ will be:

$$\text{P}(\text{ON}) = (\frac{1}{2} * \frac{2}{6}) \ \ + \ \ (\frac{1}{2} * \frac{1}{6})$$

info

$\large\frac{1}{2}$ in above equation represent probability that letter come from either London or Clifton.

But question is asking us probability of event when using a particular path is taken, so let's apply Bayes' theorem formula below:

$$\text{P}(\frac{\text{LO}}{\text{ON}}) = \frac{\text{P}(\frac{\text{ON}}{\text{LO}}) * \text{P}(\text{LO})}{\text{P(ON)}}$$

Where,

$\text{P}(\large\frac{\text{LO}}{\text{ON}}\normalsize)$, is probability that event $\text{ON}$(i.e. letter showing $\text{ON}$) was caused by event $\text{LO}$(i.e. letter is from London).

$\text{P}(\large\frac{\text{ON}}{\text{LO}}\normalsize)$, is probability that event $\text{ON}$ will occur given event $\text{E}1$ has occurred.

$\text{P}(\text{LO})$, is probability of event $\text{LO}$(i.e. letter is from London).

$\text{P}(\text{ON})$, is total probability of event $\text{ON}$(ie. letter showing $\text{ON}$).

Substituting the values in Bayes' theorem formula we get:

$$\frac{(\frac{1}{2} * \frac{2}{6})}{(\frac{1}{2} * \frac{2}{6}) \ \ + \ \ (\frac{1}{2} * \frac{1}{6})}$$

So probability probability that the letter has come from $\text{LONDON}$ and showing $\text{ON}$ will be:

$$\frac{2}{3}$$

Probabilty Distribution

A probability distribution is a mathematical function that describes the probability of each possible outcome.

Example 1

If $3$ coins are tossed simultaneously find the probability distribution of number of heads.

Solution

When $3$ coins are tossed simultaneously, there are $2^3 = 8$ possible outcomes:

$\text{HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}$.

Probability Distribution of Number of Heads will be as below:

• $\text{P(X=0) = 1/8 (TTT)}$

• $\text{P(X=1) = 3/8 (HTT, THT, TTH)}$

• $\text{P(X=2) = 3/8 (HHT, HTH, THH)}$

• $\text{P(X=3) = 1/8 (HHH)}$

Now if question further asks us to calculate Mean, Variance or Standard Deviation of probability distribution we can calculate them as below.

Mean of probability distribution will be:

$$\text{mean}(\mu) = \sum \text{x}_\text{i} * \text{P}(\text{x}_\text{i}) = \frac{3}{2}$$

Variance of probability distribution will be:

$$\text{variance}(\sigma_x^2) = \sum (\text{x}_\text{i})^{2} * \text{P}(\text{x}_\text{i}) - (\mu)^2 = \frac{3}{4}$$

Standard deviation of probability distribution will be:

$$\text{standard deviation}(\sigma) = \sqrt{\sigma_\text{x}} = \sqrt{\frac{3}{8}}$$

Example 2

A person throws two fair dice. He wins $15$ points for throwing a doublet(Same numbers on the two dice), wins $12$ points when the throw results in sum of $9$ and losses $6$ points for any other outcomes on the throw. Calculate the expected gain/lose of the person?

Solution

Possible Outcomes:

• Doublet (same numbers): $6$ outcomes ($1-1, 2-2, ..., 6-6$)
• Sum $9$: $4$ outcomes ($3-6, 4-5, 5-4, 6-3$)
• Other outcomes: $36 - 6 - 4 = 26$ outcomes

Probabilities:

• Doublet: $6/36 = 1/6$
• Sum $9$: $4/36 = 1/9$
• Other outcomes: $26/36 = 13/18$

Points:

• Doublet: $15$ points
• Sum $9$: $12$ points
• Other outcomes: $-6$ points

$$\text{Expected Gain or Loss} = (15 * \frac{1}{6}) + (12 * \frac{1}{9}) + (-6 * \frac{13}{18})$$

So expected gain or loss for the person will be: $-0.5$