Anonymization of System Logs for Privacy and Storage Benefits

Abstract: System logs constitute valuable information for analysis and diagnosis of system behavior. The size of parallel computing systems and the number of their components steadily increase. The volume of generated logs by the system is in proportion to this increase. Hence, long-term collection and storage of system logs is challenging. The analysis of system logs requires advanced text processing techniques. For very large volumes of logs, the analysis is highly time-consuming and requires a high level of expertise. For many parallel computing centers, outsourcing the analysis of system logs to third parties is the only affordable option. The existence of sensitive data within system log entries obstructs, however, the transmission of system logs to third parties. Moreover, the analytical tools for processing system logs and the solutions provided by such tools are highly system specific. Achieving a more general solution is only possible through the access and analysis system of logs of multiple computing systems. The privacy concerns impede, however, the sharing of system logs across institutions as well as in the public domain. This work proposes a new method for the anonymization of the information within system logs that employs de-identification and encoding to provide sharable system logs, with the highest possible data quality and of reduced size. The results presented in this work indicate that apart from eliminating the sensitive data within system logs and converting them into shareable data, the proposed anonymization method provides 25% performance improvement in post-processing of the anonymized system logs, and more than 50% reduction in their required storage space.