MN/MRF - Inferring Probabilities
given a probabilistic model, Probabilistic Inference is answering a probabilistic query over that model. It is also used for the computation/estimation of: distributions, the distribution's parameters (if it's a parametric distribution), the distribution's probabilities, and/or the distribution's characteristics A graphical model 𝒢 is a tuple 𝒢 = ⟨𝐗, 𝐃, 𝐒, 𝐅, 𝐂⟩ where: global function - is a function whose scope includes all variables (i.e. 𝑆𝑖 = 𝐗) local functions - is a function whose scope is a proper subset of variables (i.e. 𝑆𝑖 ⊂ 𝐗) 𝐂 is a set of combination operators which defines how functions are combined. common combination operators are: summation operator (𝛴) multiplication operator (𝛱) AND operator (∧) - for Boolean functions relational join operator (⨝) - when the functions are relation marginalization operator - for reasoning queries max operator - e.g. = argmax𝑦 [ 𝐹𝑖(𝑥,𝑦) ] = 𝐹𝑗(𝑥) where 𝐹𝑗 is a new function with scope over variable 𝑥 the set of local functions can be combined in a variety of ways (e.g. combination operators) to generate a new local function or even a global function and let: 𝐡 = 𝘩1, ..., 𝘩𝑡 be a grounded instantiation of 𝐇 𝑋, 𝐸, and 𝑌 are disjoint exhaustive sets of all variables 𝐗 therefore: probability distribution models/representations see also: ML - Parametric vs Non-Parametric representations of global structures representations of normalized distributions: representations of un-normalized distributions: Product Marginalization Posterior Conditional Query Belief Updating Query Prior Marginal Query Probability of Evidence Query Maximum a Posterior (𝑴𝑨𝑷) Query Most Probable Explanation (𝑴𝑷𝑬) Query more details: Probabilistic Inference - Query/Task Types (Posterior Conditional - Prior Marginal / Probability of Evidence - MPE - MAP) resources:Query
Sum
𝛴Maximization
𝑎𝑟𝑔𝑚𝑎𝑥Description
𝐏(𝐐=𝐪|𝐄=𝐞) = 𝛴𝐡∊𝐇 [ 𝐏(𝐐=𝐪, 𝐇=𝐡, 𝐄=𝐞) ] / 𝐏(𝐄=𝐞)
(a type of posterior conditional query)
𝐏(𝑄𝑖=𝑞𝑖|𝐄=𝐞) = 𝛴𝐡∊𝐇 [ 𝐏(𝑄𝑖=𝑞𝑖, 𝐇=𝐡, 𝐄=𝐞) ] / 𝐏(𝐄=𝐞)✔ ✔ ✘
𝐏(𝐐=𝐪) = 𝛴𝐡∊𝐇 [ 𝐏(𝐐=𝐪, 𝐇=𝐡) ]
𝐏(𝐄=𝐞) = 𝛴𝐡∊𝐇 [ 𝐏(𝐇=𝐡, 𝐄=𝐞) ]✔ ✔ ✘
𝑀𝐴𝑃(𝐐=?|𝐄=𝐞) = 𝑎𝑟𝑔𝑚𝑎𝑥𝐪 [ 𝛴𝐡∊𝐇 [ 𝐏(𝐐=𝐪, 𝐇=𝐡, 𝐄=𝐞) ] ]
where: 𝐐∪𝐄⊂𝐗 and 𝐐∪𝐇∪𝐄=𝐗✔ ✔ ✔
𝑀𝑃𝐸(𝐐=?|𝐄=𝐞) = 𝑎𝑟𝑔𝑚𝑎𝑥𝐪 [ 𝐏(𝐐=𝐪, 𝐄=𝐞) ]
where: 𝐐∪𝐄=𝐗✔ ✘ ✔