Multivalued Subsets Under Information Theory (1107.5953v3)

Abstract: In the fields of finance, engineering and sciences data mining/ machine learning has held an eminent position in predictive analysis. Complex algorithms and adaptive decision models have contributed towards streamlining research as well as improve forecasting. Extensive study in areas surrounding computation and mathematical sciences has primarily been responsible for the field's development. Classification based modeling, which holds a prominent position amongst the different rule-based algorithms, is one of the most widely used decision making tool. The decision tree has a place of profound significance in classification modeling. A number of heuristics have been developed over the years to refine its decision making process. Most heuristics applied to such tree-based learning algorithms derive their roots from Shannon's 'Information Theory'. The current application of this theory is directed towards individual assessment of the attribute-values. The proposed study takes a look at the effects of combining these values with the aim to improve the 'Information Gain'. A search-based heuristic tool is applied for identifying the subsets sharing a better gain value than the ones presented in the GID3 approach. An application towards the feature selection stage of the mining process has been tested and presented with statistical analysis.