Conditional/Discriminative Random Fields (CRF)

• a type of probabilistic graphical model
• is a type of discriminative model
• used for capturing/estimating the conditional distribution 𝐏(𝒀=𝒚|𝑿=𝒙) # no need to model the correlation between variables in 𝑿, only focusing on target variables 𝒀. This allows models with highly expressive features without worrying about wrong independencies
• parameterized the same as Gibbs Distribution, but normalized differently

• a special case of a Markov Random Field (MRF) where the factor/clique potentials are CONDITIONED on input 𝑿=𝒙

  • see: Conditional Random Field (CRF) vs Markov Random Field (MRF)

• a generalization of Logistic Regression Model to have structured outputs, like chains, trees, or grids

  • see: Conditional Random Field (CRF) vs Logistic Regression Model

CRF - Formula

formulas 𝜙_𝑗(𝒚, 𝒙) = 𝑒^{𝜃_𝑗·𝑓_𝑗(𝒚, 𝒙)}

• 𝐏(𝒀=𝒚|𝑿=𝒙) = 𝛱_{0≤𝑗≤𝑘}[ 𝜙_𝑗(𝒚, 𝒙) ] / 𝑍(𝒙)

• 𝐏(𝒀=𝒚|𝑿=𝒙) = 𝛱_{0≤𝑗≤𝑘}[ 𝑒^{𝜃_𝑗·𝑓_𝑗(𝒚, 𝒙)} ] / 𝑍(𝒙)
• 𝐏(𝒀=𝒚|𝑿=𝒙) = 𝑒^{𝛴_{0≤𝑗≤𝑘}[ 𝜃_𝑗·𝑓_𝑗(𝒚, 𝒙) ]} / 𝑍(𝒙)

where:

• 𝑍(𝑿=𝒙) = 𝛴_{𝑦1∊𝑌1} ... 𝛴_{𝑦𝑘∊𝑌𝐾} [ 𝛱_{0≤𝑗≤𝑘} [ 𝜙_𝑗(𝑦₁, ..., 𝑦_𝐾, 𝒙) ] ]
• 𝑍(𝑿=𝒙) = 𝛴_{𝑦1∊𝑌1} ... 𝛴_{𝑦𝐾∊𝑌𝐾} [ 𝛱_{0≤𝑗≤𝑘}[ 𝑒^{𝜃_𝑗·𝑓_𝑗(𝑦₁, ..., 𝑦_𝐾, 𝒙)} ] ]
• 𝑍(𝑿=𝒙) = 𝛴_{𝑦1∊𝑌1} ... 𝛴_{𝑦𝐾∊𝑌𝐾} [ 𝑒^{𝛴_{0≤𝑗≤𝑘}[ 𝜃_𝑗·𝑓_𝑗(𝑦₁, ..., 𝑦_𝐾, 𝒙) ]}]

(𝒀,𝑿) is a CRF when each of the random variables 𝑌_𝑖, conditioned on 𝑿, obeys the Markov Property with respect to the graph. Thus, a CRF is a Random Field globally conditioned on the observation 𝑿. Where the hidden variables 𝒀 are globally conditioned on the observed variables 𝑿

CRF - Graphical Model

CRF - Variants

CRF - Example Use

Part-of-Speech (PoS) Tagging

conditional-random-field-log-linear-model-example.drawio

CRF - Comparisons

• Conditional Random Field (CRF) vs Markov Random Field (MRF)
• Conditional Random Field (CRF) vs Logistic Regression Model
• Conditional Random Field (CRF) vs Linear Chain CRF

CRF - Resources

• Daphne Koller - YouTube video
• Hugo Larochelle - YouTube
• David Batista - Article