/
Probability - Distributions

Probability - Distributions

Aug 31, 2023

Probability Distribution

  • describes the real-world behavior of one or more random variables. These random variables can be either: discrete, continuous, or a mixture of the two
  • describes how probabilities are distributed over the values of a random variable:
  • is a distribution function that:
    • outputs a value between 0 and 1
    • all values sum/integrate to 1

Probability Distribution - Population vs Sample

see: Population Distribution - Sample/Empirical Distribution

Probability Distribution - How They are Modeled/Represented

see Probability - Distribution Models/Representations

Probability Distribution - Main Types

Probability Distributions & DescriptionSyntax Examples

univariate probability distribution (sometimes just called probability distribution)

  • is a model that describes the probability of a random variable
  • is essentially a "list" of all possible outcomes (of the random variable) along with their corresponding probability value
  • a variety of phenomena can be described by relatively few families of probability distribution models

  • 𝐏(𝑋)
  • 𝐏(𝑌)
  • 𝐏(𝑍)

joint probability distribution (compound probability distribution) - refers to the probability distribution of 2 or more random variables occurring together

  • multivariate joint probability distribution - sample space is a multivariate random variable (2 or more dimensions/variables) (e.g. the probability of 𝐴 and 𝐵 and 𝐶, denoted 𝐏(𝐴,𝐵,𝐶))
    • bivariate joint probability distribution - sample space is a bivariate random variable (two-dimensional variables) (e.g. the probability of 𝐴 and 𝐵, denoted 𝐏(𝐴,𝐵))
  • full joint probability distribution - refers to a probability distribution of all random variables in a given domain

  • 𝐏(𝑋,𝑌)
  • 𝐏(𝑋,𝑌,𝑍)
  • 𝐏(𝑌,𝑍)

marginal probability distribution 

  • 𝐏(𝑋) = 𝛴𝑦∈𝑌𝐏(𝑋,𝑌=𝑦)
  • 𝐏(𝑋,𝑌) = 𝛴𝑧∈𝑍𝐏(𝑋,𝑌,𝑍=𝑧)
  • 𝐏(𝑋) = 𝛴𝑦∈𝑌,𝑧∈𝑍𝐏(𝑋,𝑌=𝑦,𝑍=𝑧)

conditional probability distribution (CPD) of event 𝑋 given event 𝑌 is the probability distribution that 𝑋 occurs when 𝑌 is known to occur (denoted as 𝐏(𝑋|𝑌=𝑦)), 𝑋 and 𝑌 are jointly distributed random variables

  • 𝐏(𝑋|𝑌)
  • 𝐏(𝑋|𝑌,𝑍)

Probability Distribution - Other Types

Estimating Parameters of a Parametric Distribution

Given:

Estimate:

  • the probability distribution function's parameters that best reflect the sample training data

See: Point/Parameter Estimation/Estimator/Estimate/Approximation - Estimating Parameters of Parametric Distribution/Population Function/Model (Population Parameters - Sample Statistics)

Generating Random Variable(s) that Simulate a Specific Probability Distribution

see: Probability - Generating Random Variable(s) that Simulates a Distribution

Resources