given training/sample data 𝐷 the MLE solution to estimate 𝜃 is similar to Bernoulli MLE / MAP Multinoulli:

where:

given:

simple bayesian network shown below (with unknown parameters 𝜃 = {𝜃_𝐴, 𝜃_𝐵|𝐴=0, 𝜃_𝐵|𝐴=1})
completed sample/training data 𝐷 of tuples:
- (𝐴=0, 𝐵=0) # complete tuple
- (𝐴=0, 𝐵=1) # complete tuple
- (𝐴=1, 𝐵=1) # complete tuple

estimate the values of the unknown parameters 𝜃 = {𝜃_𝐴, 𝜃_𝐵|𝐴=0, 𝜃_𝐵|𝐴=1} (NOTE: similar to Bernoulli MLE)

How to estimate parameter {𝜃_𝐴, 𝜃_𝐵|𝐴=0, 𝜃_𝐵|𝐴=1} using MLE method?

the log-likelihood function 𝓛(𝜃) of the given Bayesian Network (NOTE: the log-likelihood function changes wrt the Bayesian Network):

𝓛(𝜃) =
- #(𝐴=1)·𝑙𝑛(𝜃_𝐴) + #(𝐴=0)·𝑙𝑛(1 - 𝜃_𝐴) +
- #(𝐴=1,𝐵=1)·𝑙𝑛(𝜃_𝐵|𝐴=1) + #(𝐴=1,𝐵=0)·𝑙𝑛(1 - 𝜃_𝐵|𝐴=1) +
- #(𝐴=0,𝐵=1)·𝑙𝑛(𝜃_𝐵|𝐴=0) + #(𝐴=0,𝐵=0)·𝑙𝑛(1 - 𝜃_𝐵|𝐴=0) # click here for step-by-step computation

now differentiate with respect to 𝜃_𝐴

equate to 0 and solve for 𝜃_𝐴

repeat for {𝜃_𝐵|𝐴=0, 𝜃_𝐵|𝐴=1} and we get:

therefore, with the completed sample/training data 𝐷 of tuples:

we estimate the parameters as so:

In cases where denominator is 0, one can use (laplace smoothing or MAP Bayesian Networks)

Browser not supported