2025-02-11
Probability Theory
Joint Probability
Distribution Functions
Normal Distribution
Empirical Rule
Binomial Distribution
Poisson Distribution
Probability in R
Probability is the measure of how likely an event is to occur. It ranges from 0 to 1:
An action or process that generates outcomes.
- Example: Rolling a die.
The set of all possible outcomes of an experiment.
- Example: For rolling a die, \(S = \{1, 2, 3, 4, 5, 6\}\).
A subset of the sample space, representing outcomes of interest.
The proportion of times an event is expected to occur if the experiment is repeated many times.
The probability of an event \(A\) is defined as:
\[ P(A) = \frac{\text{Number of favorable outcomes}}{\text{Total number of possible outcomes}} \]
If we roll a die, what is the probability of rolling a 4?
\[ P(\text{rolling a 4}) = \frac{1}{6} \approx 0.167 \]
Probability of the Sample Space
The probability of the sample space is always 1: \[ P(S) = 1 \]
Probability of Impossible Events
The probability of an event that cannot happen is 0:
\[ P(\emptyset) = 0 \]
Complement Rule
The probability of the complement of an event \(A\) (not \(A\)) is:
\[ P(A^c) = 1 - P(A) \]
If you draw a card from a standard deck of 52 cards, what is the probability of drawing:
A heart?
\[
P(\text{heart}) = \frac{13}{52} = 0.25
\]
A red card (heart or diamond)?
\[
P(\text{red card}) = \frac{26}{52} = 0.5
\]
What is the probability of getting:
Exactly one head?
Sample space: \(S = \{\text{HH, HT, TH, TT}\}\).
Event: \(A = \{\text{HT, TH}\}\).
\[
P(A) = \frac{2}{4} = 0.5
\]
At least one head?
Event: \(B = \{\text{HH, HT, TH}\}\).
\[
P(B) = \frac{3}{4} = 0.75
\]
A commuter encounters three traffic lights, each with a 70% chance of being green. Assuming independence, what is the probability that all three lights are green?
\[ P(\text{all green}) = 0.7 \cdot 0.7 \cdot 0.7 = 0.343 \]
| 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|
| 1/6 | 1/6 | 1/6 | 1/6 | 1/6 | 1/6 |
\[ P(X = 2) \]
\[ P(1 \le X \le 3) \]
\[ P(1 < X < 3) \]
\[ P(X > 1) \]
Probability Theory
Joint Probability
Distribution Functions
Normal Distribution
Empirical Rule
Binomial Distribution
Poisson Distribution
Probability in R
What is the probability of rolling a number greater than 4 or an even number on a six-sided die?
The conditional probability of event A occurring given that event B has occurred: \(P(A|B)\) (read as “the probability of A given B”).
Formula: \(P(A|B) =\frac{P(A \cap B)}{P(B)}\)
Probability Theory
Joint Probability
Distribution Functions
Normal Distribution
Empirical Rule
Binomial Distribution
Poisson Distribution
Probability in R
A random variable is a variable whose value is a numerical outcome of a random phenomenon. It’s a way to map the outcomes of a probabilistic event to numbers. This allows us to analyze these outcomes mathematically.
Random Phenomenon: This is any process or experiment whose outcome is uncertain.
Outcomes: The possible results of a random phenomenon are called outcomes.
Numerical Value: A random variable assigns a numerical value to each outcome.
A discrete random variable can only take on a finite number of values or a countably infinite number of values. The values are often integers. Examples:
A continuous random variable can take on any value within a given range. Examples:
A distribution function describes the probabilities of a random variable across its possible values. It answers questions like:
The Cumulative Distribution Function (CDF) describes the probability that a random variable \(X\) is less than or equal to a certain value \(x\): \[ F_X(x) = P(X \leq x) \]
For a die roll (discrete case):
For a normal distribution (continuous case):
The Probability Density Function (PDF) is used for continuous random variables and describes the likelihood of the variable falling within a small interval.
\[ P(a \leq X \leq b) = \int_a^b f_X(x) \, dx \]
Important
\[ P(X = a) = P(a \leq X \leq a) = \int_a^a f_X(x) \, dx = 0 \]
Non-Negative: \[ f_X(x) \geq 0 \quad \forall x \]
Total Area Under Curve: \[ \int_{-\infty}^\infty f_X(x) \, dx = 1 \]
The PDF of the normal distribution is: \[ f_X(x) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} \]
Here, \(\mu\) is the mean and \(\sigma\) is the standard deviation.
The Probability Mass Function (PMF) is used for discrete random variables and gives the probability of each possible value: \[ P(X = x) = p_X(x) \]
Non-Negative: \[ p_X(x) \geq 0 \quad \forall x \]
Sum of Probabilities: \[ \sum_x p_X(x) = 1 \]
For a fair die, the PMF is: \[ p_X(x) = \frac{1}{6}, \quad x \in \{1, 2, 3, 4, 5, 6\} \]
A percentile is a measure indicating the value below which a given percentage of observations in a group of observations falls. It is used in statistics to provide information about the relative standing of a particular value within a dataset.
Suppose we have a random variable X representing the height of adults in a population. If the 90th percentile of X is 180 cm, it means that the probability of a randomly selected adult being 180 cm or shorter is 90%.
data <- rnorm(500000, mean = 100, sd = 15) |> data.frame(x = _) # Example: normally distributed data
# Calculate the 75th percentile
percentile_75 <- qnorm(0.75, 100, 15)
ggplot(data = data, aes(x = x)) +
geom_density(fill = "skyblue", alpha = 0.5) + # Density curve
geom_vline(xintercept = percentile_75, color = "red", linetype = "dashed", size = 1) +
labs(title = "Density Plot with 75th Percentile Highlighted", x = "Data Value", y = "Density") +
theme_bw()For a fair coin tossed once:
The PMF is: \[ p_X(x) = \begin{cases} 0.5 & x = 0 \text{ or } x = 1 \\ 0 & \text{otherwise} \end{cases} \]
An exponential random variable with rate \(\lambda > 0\) has a CDF: \[ F_X(x) = \begin{cases} 1 - e^{-\lambda x}, & x \geq 0 \\ 0, & x < 0 \end{cases} \]
This can be used to model waiting times between events.
PMF:
PDF:
CDF:
| Continuous RV | Parameters | Notation | Support |
|---|---|---|---|
| Uniform | a (minimum), b (maximum) | Unif(a, b) | [a, b] |
| Normal (Gaussian) | μ (mean), σ (standard deviation) | N(μ, σ²) | (-∞, ∞) |
| Exponential | λ (rate) | Exp(λ) | [0, ∞) |
| Gamma | k (shape), θ (scale) | Γ(k, θ) | [0, ∞) |
| Beta | α (shape 1), β (shape 2) | Beta(α, β) | [0,1] |
| Chi-Squared | k (degrees of freedom) | χ²(k) | [0, ∞) |
| t-distribution | ν (degrees of freedom) | t(ν) | (-∞, ∞) |
| F-distribution | ν₁ (degrees of freedom 1), ν₂ (degrees of freedom 2) | F(ν₁, ν₂) | [0, ∞) |
| Weibull | k (shape), λ (scale) | Weibull(k, λ) | [0, ∞) |
| Lognormal | μ (location), σ (scale) | Lognormal(μ, σ) | [0, ∞) |
| Discrete RV | Parameters | Notation | Support |
|---|---|---|---|
| Bernoulli | p (probability of success) | Bernoulli(p) | {0, 1} |
| Binomial | n (number of trials), p (probability of success) | Bin(n, p) | {0, 1, 2, …, n} |
| Geometric | p (probability of success) | Geo(p) | {1, 2, 3, …} |
| Poisson | λ (average rate of events) | Pois(λ) | {0, 1, 2, …} |
| Negative Binomial | r (number of successes), p (probability of success) | NB(r, p) | {r, r+1, r+2, …} |
| Discrete Uniform | n (number of possible outcomes) | DUnif(1, n) | {1, 2, …, n} or a set of n values |
mean <- 0 # Mean of the distribution
sd <- 1 # Standard deviation of the distribution
# Generate data points for the curve (more points = smoother curve)
x <- seq(mean - 3*sd, mean + 3*sd, length.out = 100) # Range covering most of the distribution
y <- dnorm(x, mean = mean, sd = sd) # Calculate the density at each x value
# Create the plot
ggplot(data = data.frame(x, y), aes(x = x, y = y)) +
geom_line(color = "blue", size = 1) + # Line for the distribution curve
labs(x = "X",
y = "Probability Density") +
theme_bw() + # A clean theme
scale_x_continuous(breaks = seq(mean - 3*sd, mean + 3*sd, by = sd)) # Ticks at standard deviations
n <- 10 # Number of trials
p <- 0.3 # Probability of success on each trial
# Generate the data for the plot
x <- 0:n # Possible values for the random variable (number of successes)
y <- dbinom(x, size = n, prob = p) # Calculate the probabilities
# Create the plot using geom_col (for discrete data)
ggplot(data = data.frame(x, y), aes(x = factor(x), y = y)) + # x as factor for discrete bars
geom_col(fill = "skyblue", color = "black") +
labs(title = paste("Binomial Distribution (n =", n, ", p =", p, ")"),
x = "Number of Successes (x)",
y = "Probability P(X = x)") +
theme_bw()
Probability Theory
Joint Probability
Distribution Functions
Normal Distribution
Empirical Rule
Binomial Distribution
Poisson Distribution
Probability in R
The Normal Distribution is a probability distribution that is symmetric, with most of the data points clustering around the mean.
Important
For any normally distributed data, the highest probability density is at the mean, and as you move away from the mean, the probability density gradually decreases.
\[ X\sim N(\mu, \sigma) \]
\[ -\infty < X < \infty \]
\[ f_X(x) = \frac{1}{\sqrt{2\pi \sigma}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} \]
The Standard Normal Distribution is a special type of normal distribution with a mean of 0 and a standard deviation of 1. It’s often used as a reference to convert any normal distribution to a standard form.
A Z-score (or standard score) tells us how many standard deviations an individual data point is from the mean. It’s calculated as:
\[ Z = \frac{X - \mu}{\sigma} \]
Scores on standardized tests, such as IQ tests or the SAT, are often designed to follow a normal distribution. By knowing a student’s score in terms of Z-scores, we can determine their percentile or how they compare to other test-takers.
In finance, stock returns and other economic factors are often modeled with a normal distribution to estimate risk, forecast trends, and make informed investment decisions.
\[ X \sim N(\mu, \sigma) \]
\[ P(a \leq X \leq b) = P(a < X < b) = \int_a^b \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} dx \]
Probability Theory
Joint Probability
Distribution Functions
Normal Distribution
Empirical Rule
Binomial Distribution
Poisson Distribution
Probability in R
The Empirical Rule provides a way to understand the spread of data in a normal distribution by describing how data points cluster around the mean. According to this rule:
The empirical rule is very helpful because, with just the mean and standard deviation, we can quickly estimate how data is distributed within a normal curve.
Let’s define the key terms and apply the empirical rule to the normal distribution.
In a normal distribution:
These intervals allow us to estimate probabilities for data within each range without needing to calculate exact probabilities.
To better understand the empirical rule, imagine a symmetric, bell-shaped normal curve. Here’s how it would look based on the empirical rule:
This visualization shows how the data is most concentrated around the mean, with less data appearing as we move further away.
The empirical rule helps us answer questions like:
Suppose exam scores are normally distributed with a mean of 70 and a standard deviation of 10.
Assume that adult heights follow a normal distribution with a mean height of 170 cm and a standard deviation of 8 cm.
Probability Theory
Joint Probability
Distribution Functions
Normal Distribution
Empirical Rule
Binomial Distribution
Poisson Distribution
Probability in R
The Binomial Distribution is a probability distribution that models the number of successes in a fixed number of trials, where each trial has:
The binomial distribution answers questions like:
\[ P(X = k) = \binom{n}{k} p^k (1-p)^{n-k} \]
Where:
Suppose you flip a fair coin 5 times (\(n = 5, p = 0.5\)) and want to know the probability of getting exactly 3 heads (\(k = 3\)).
\[ P(X = 3) = \binom{5}{3} (0.5)^3 (1-0.5)^{5-3} \]
For a binomial random variable \(X\):
A factory produces lightbulbs, and 95% of them meet quality standards. If you randomly test 10 bulbs, what is the probability that exactly 8 bulbs pass the test?
Here:
\[ P(X = 8) = \binom{10}{8} (0.95)^8 (0.05)^2 \]
A basketball player has a 60% chance of making a free throw. If they take 5 shots, what is the probability of making at least 3 shots?
Here: - \(n = 5\), \(p = 0.6\).
\[ P(X \geq 3) = P(X = 3) + P(X = 4) + P(X = 5) \]
In a clinical trial, a new drug has a 70% success rate. If 15 patients are treated, what is the probability that exactly 10 respond positively to the treatment?
Probability Theory
Joint Probability
Distribution Functions
Normal Distribution
Empirical Rule
Binomial Distribution
Poisson Distribution
Probability in R
The Poisson Distribution is a discrete probability distribution that models the number of events occurring within a fixed interval. These events must happen independently and at a constant average rate.
The Poisson distribution helps us answer questions like:
\[ P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!} \]
Where:
Suppose a call center receives an average of 10 calls per hour (\(\lambda = 10\)). What is the probability of receiving exactly 7 calls in the next hour ($ k = 7 $)?
\[ P(X = 7) = \frac{10^7 e^{-10}}{7!} \]
A toll booth observes an average of 3 cars passing through every 5 minutes (\(\lambda = 3\)). What is the probability of seeing exactly 5 cars in the next 5 minutes?
Using the formula: \[ P(X = 5) = \frac{3^5 e^{-3}}{5!} = \frac{243 \cdot 0.0498}{120} \approx 0.1008 \]
A factory produces an average of 2 defective items per day (\(\lambda = 2\)). What is the probability of finding no defective items in a day (\(k = 0\))?
\[ P(X = 0) = \frac{2^0 e^{-2}}{0!} = e^{-2} \approx 0.1353 \]
Probability Theory
Joint Probability
Distribution Functions
Normal Distribution
Empirical Rule
Binomial Distribution
Poisson Distribution
Probability in R
| Letter | Functionality |
|---|---|
| “d” | returns the height of the probability density function |
| “p” | returns the cummulative density function value |
| “q” | returns the inverse cummulative density function (percentiles) |
| “r” | returns a randomly generated number |
R can compute the probabilities of a distribution given the correct parameters:
p is used in front of the distribution R functiond is used in front of the distribution R function
Find \(P(14 < X < 20)\) where \(X \sim Pois(16)\).
Finding the values (percentiles) for any distributions can be found by using the q-based distribution R function such as qnorm(), qpois(), and qbinom() functions.
R is capable of generating random numbers. For example if we want to generate a random sample of size fifty from a normal distribution with mean eight and variance three, we will use the rnorm(). If we want to generate a random sample from any distribution, use the distribution function with r in front of it.
Generate a random sample of 100 form an \(X \sim Binom (25,.23)\).