Logistic (Logit) Regression Model

Logistic (Logit) Regression Model

Logistic (Logit) Regression Model

  • is a discriminative-typed supervised classification algorithm that models the relationship between:
    • a single discrete/categorical response/dependent variable 𝑌 (for continuous/scalar use linear regression)
    • one or more explanatory/predictor/covariate/independent variables {𝑋1, ..., 𝑋𝑘}. predictor variable types:
      • continuous/scalar/numerical predictor
      • discrete/categorical predictor - itself can be either bi/multi-nominal or ordinal 
  • unlike linear regression which outputs continuous number values, logistic regression transforms its output using the logistic sigmoid function to return a probability value which can then be mapped to two or more discrete classes

Logistic Regression - Types

TypeDescription
Binomial/Binary Logistic Regression (BLR)
  • nominal target variable can have only 2 possible types: “0” or “1”
  • e.g. “win” vs “loss”, “pass” vs “fail”, “dead” vs “alive”, etc
Multinomial/Nominal Logistic Regression (MLR)
  • nominal target variable can have 3 or more possible types which are not ordered (i.e. types have no quantitative significance)
  • e.g. “disease A” vs “disease B” vs “disease C”
Ordinal Logistic Regression
  • ordinal/ordered target variable
  • e.g. a test score can be categorized as: “very poor”, “poor”, “good”, “very good”. Here, each category can be given a score like 0, 1, 2, 3

Logistic Regression - Other