Constructing free-energy approximations and generalized belief propagation algorithms

@article{citeulike:507310, abstract = {Important inference problems in statistical physics, computer vision, error-correcting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree, but only approximate when the factor graph has cycles. We show that BP fixed points correspond to the stationary points of the Bethe approximation of the free energy for a factor graph. We explain how to obtain region-based free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms. We emphasize the conditions a free energy approximation must satisfy in order to be a "valid" or "maxent-normal" approximation. We describe the relationship between four different methods that can be used to generate valid approximations: the "Bethe method", the "junction graph method", the "cluster variation method", and the "region graph method". Finally, we explain how to tell whether a region-based approximation, and its corresponding GBP algorithm, is likely to be accurate, and describe empirical results showing that GBP can significantly outperform BP.}, author = {Yedidia, J. S. and Freeman, W. T. and Weiss, Y. }, citeulike-article-id = {507310}, doi = {http://dx.doi.org/10.1109/TIT.2005.850085}, journal = {Information Theory, IEEE Transactions on}, keywords = {aland-07, belief, propagation}, number = {7}, pages = {2282--2312}, posted-at = {2007-07-06 08:50:20}, priority = {3}, title = {Constructing free-energy approximations and generalized belief propagation algorithms}, url = {http://dx.doi.org/10.1109/TIT.2005.850085}, volume = {51}, year = {2005} }

TY - JOUR ID - citeulike:507310 L3 - citeulike-article-id:507310 TI - Constructing free-energy approximations and generalized belief propagation algorithms JF - Information Theory, IEEE Transactions on VL - 51 IS - 7 SP - 2282 EP - 2312 N2 - Important inference problems in statistical physics, computer vision, error-correcting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree, but only approximate when the factor graph has cycles. We show that BP fixed points correspond to the stationary points of the Bethe approximation of the free energy for a factor graph. We explain how to obtain region-based free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms. We emphasize the conditions a free energy approximation must satisfy in order to be a "valid" or "maxent-normal" approximation. We describe the relationship between four different methods that can be used to generate valid approximations: the "Bethe method", the "junction graph method", the "cluster variation method", and the "region graph method". Finally, we explain how to tell whether a region-based approximation, and its corresponding GBP algorithm, is likely to be accurate, and describe empirical results showing that GBP can significantly outperform BP. KW - aland-07 KW - belief KW - propagation AU - Yedidia, JS AU - Freeman, WT AU - Weiss, Y PY - 2005/// UR - http://dx.doi.org/10.1109/TIT.2005.850085 ER -