Title: Increasing and Decreasing Returns and Losses in Mutual Information Feature Subset Selection
Authors: Van Dijck, Gert ×
Van Hulle, Marc #
Issue Date: Oct-2010
Publisher: Molecular Diversity Preservation International
Series Title: Entropy vol:12 issue:10 pages:2144-2170
Abstract: Mutual information between a target variable and a feature subset is extensively used as a feature subset selection criterion. This work contributes to a more thorough understanding of the evolution of the mutual information as a function of the number of features selected. We describe decreasing returns and increasing returns behavior in sequential forward search and increasing losses and decreasing losses behavior in sequential backward search. We derive conditions under which the decreasing returns and the increasing losses behavior hold and prove the occurrence of this behavior in some Bayesian networks. The decreasing returns behavior implies that the mutual information is concave as a function of the number of features selected, whereas the increasing returns behavior implies this function is convex. The increasing returns and decreasing losses behavior are proven to occur in an XOR hypercube.
ISSN: 1099-4300
Publication status: published
KU Leuven publication type: IT
Appears in Collections:Laboratory for Neuro- and Psychofysiology
Research Group Neurophysiology
× corresponding author
# (joint) last author

Files in This Item:
File Description Status SizeFormat
entropy-12-02144.pdfarticle entropy 2010 Published 852KbAdobe PDFView/Open


All items in Lirias are protected by copyright, with all rights reserved.

© Web of science