On Model Selection, Bayesian Networks, and the Fisher Information Integral

详细信息    查看全文

• 作者：Yuan Zou ; Teemu Roos
• 关键词：Model selection ; Bayesian networks ; Fisher information approximation ; NML ; BIC
• 刊名：New Generation Computing
• 出版年：2017
• 出版时间：January 2017
• 年：2017
• 卷：35
• 期：1
• 页码：5-27
• 全文大小：
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence (incl. Robotics); Computer Hardware; Computer Systems Organization and Communication Networks; Software Engineering/Programming and Operating Systems; Computing Methodologies;
• 出版者：Ohmsha
• ISSN：1882-7055
• 卷排序：35

文摘

We study BIC-like model selection criteria and in particular, their refinements that include a constant term involving the Fisher information matrix. We perform numerical simulations that enable increasingly accurate approximation of this constant in the case of Bayesian networks. We observe that for complex Bayesian network models, the constant term is a negative number with a very large absolute value that dominates the other terms for small and moderate sample sizes. For networks with a fixed number of parameters, d, the leading term in the complexity penalty, which is proportional to d, is the same. However, as we show, the constant term can vary significantly depending on the network structure even if the number of parameters is fixed. Based on our experiments, we conjecture that the distribution of the nodes’ outdegree is a key factor. Furthermore, we demonstrate that the constant term can have a dramatic effect on model selection performance for small sample sizes.