The field of statistics exists because it is usually impossible to collect data from all individuals of interest (population). Our only solution is to collect data from a subset (sample) of the individuals of interest, but our real desire is to know the “truth” about the population. Quantities such as means, standard deviations and proportions are all important values and are called “parameters” when we are talking about a population. Since we usually cannot get data from the whole population, we cannot know the values of the parameters for that population. We can, however, calculate estimates of these quantities for our sample. When they are calculated from sample data, these quantities are called “statistics.” A statistic estimates a parameter.

• population distribution consists of all units of interest
• sample/empirical distribution consists of observed units collected from the population
• population parameter (𝜽)
  • sometimes just called a parameter
  • is any variate analysis of population distribution (e.g. mean, variance, etc)
  • usually have an unknown value
• sample statistic (𝜽ˆ)
  • sometimes just called statistic
  • is a function of sample distribution as input
  • is any variate analysis of a sample distribution (e.g. sample mean, sample variance, etc)
  • is an estimate of the corresponding population parameter 𝜽
  • is a random variable because it is computed from a random sample distribution a subset of population distribution. Thus, this statistic has a sampling distribution
  • see methods estimating sample statistic
• Chance Error - Sample/Sampling Bias/Error

Many times there are observable phenomena that are random in nature. We call it a Random Process (Random Experiment). The random process has outcomes, and subsets of these outcomes are called Events. We map these events to a numeric form using Random Variables.

We study and capture our knowledge about this random process by creating a Stochastic Model. The stochastic model predicts the output of an event by:

1. providing different choices (of values of a random variable)
2. the probability of those choices

These two elements are summarized as a Probability Distribution.

This distribution has some parameters (like mean, standard deviation, etc) which were inferred from the observable phenomena using Statistical Inference.

Before inference, the distribution had unknown (not inferred yet) parameters. It was, hence, a family of distributions, since each value of the parameter is a different distribution. This family is called a Statistical Model.

Usually, a statistical model is guessed (exponential, binomial, normal, uniform, Bernoulli, etc) using Exploratory Data Analysis, then its parameters are inferred (estimated) by applying statistical inference (say, algorithms involving loss function minimization) to arrive at a stochastic model (statistical model with known parameters) (a.k.a. Estimator) that captures our knowledge about the random process.

The term 'Probability Model' (probabilistic model) is usually an alias for stochastic models.