Mining for significant execution profiles for software assessment -

Abstract

The interest in applying data mining and statistical techniques to solve software analysis problems has increased tremendously in recent years. Researchers have presented numerous techniques that mine and analyze execution profiles to assist software testing, fault localization, and program comprehension. Previous empirical studies have shown that the effectiveness of such techniques is likely to be impacted by the type of the profiled program elements. This work further studies the impact of the characteristics of execution profiles by focusing on their size; noting that a typical profile comprises a large number of elements, in the order of thousands or higher. Specifically, we devised six reduction techniques and comparatively evaluated them by measuring the following: 1) reduction rate; 2) information loss; 3) impact on the quality of cluster analysis, using various metrics; 4) cost of reduction; and 5) impact on two software analysis techniques, namely, cluster-based test suite minimization and profile-based online intrusion detection. Our results were promising as: a) the average reduction rate ranged from 92percent to 98percent; b) three techniques were lossless and three were slightly lossy; c) the quality of cluster analysis was not deteriorated; d) the cost of reduction was not very significant; and e) reducing execution profiles noticeably benefited the two software analysis techniques in our experiments.