0x545 Energy

The partition function chapter of the deep learning book has a good coverage of those methods.

Energy-based model defines a energy function \(E(X)\), and models the generation using the Boltzmann distribution

\[P(X) = \frac{\exp(-E(X))}{Z}\]

where \(Z\) is the partition function

Model (Boltzmann Machine) The most general form of Boltzmann Machine is all units are fully-connected with each other

\[E(x) = x^TWx\]

Model (Restricted Boltzmann Machine) In the restricted form, visible units and hidden units are connected with each other (but visible units are not connected internally, and hidden units as well). See this blog

\[E(x, z) = x^TWz\]