Improvement of an Approximated Self-Improving Sorter and Error Analysis of its Estimated Entropy

Abstract: The self-improving sorter proposed by Ailon et al. consists of two phases: a relatively long training phase and rapid operation phase. In this study, we have developed an efficient way to further improve this sorter by approximating its training phase to be faster but not sacrificing much performance in the operation phase. It is very necessary to ensure the accuracy of the estimated entropy when we test the performance of this approximated sorter. Thus we further developed a useful formula to calculate an upper bound for the 'error' of the estimated entropy derived from the input data with unknown distributions. Our work will contribute to the better use of this self-improving sorter for huge data in a quicker way.