Symbol Resolution MatRs: Make it Fast and Observable with Stable Linking (2501.06716v1)

Abstract: Dynamic linking is the standard mechanism for using external dependencies since it enables code reuse, streamlines software updates, and reduces disk/network use. Dynamic linking waits until runtime to calculate an application's relocation mapping, i.e., the mapping between each externally referenced symbol in the application to the dependency that provides the symbol. Unfortunately, it comes with two downsides. First, dynamic linking limits the performance of current systems since it can take seconds to calculate a relocation mapping for a large program. Second, dynamic linking limits the dependency management of applications since it prevents a developer from accurately observing a relocation mapping except at runtime. This paper makes the key insight that the benefits conventionally attributed to dynamic linking: code reuse, streamlined software updates, and reduced disk/network use are actually benefits of shared libraries. Thus, we present stable linking, a new mechanism for using dependencies that uses shared libraries to retain their benefits but eliminates the downsides of dynamic linking. Stable linking separates a system's state into management times; when the system can be modified, and epochs when it cannot. Stable linking calculates each application's relocation mapping at the beginning of each epoch, allows developers to inspect the relocation mapping during the epoch, and reuses the mapping for subsequent executions in the epoch. We design and build MatR, the first stable linker. We use MatR in three workloads and show that it improves upon dynamic linking performance by a factor of 2.19 on average. Additionally, we use the system in three vignettes, or case-studies, that illustrate the system's improvements to dependency management.

• The paper introduces stable linking, decoupling shared library benefits from dynamic linking drawbacks by precomputing relocation mappings.
• It presents MatR’s implementation that accelerates application startup by a factor of 2.19, eliminating excessive runtime symbol resolution overhead.
• The approach enhances dependency management by making relocation mappings observable and modifiable in formats like JSON, CSV, and SQL.

An Analysis of Stable Linking via Symbol Resolution MatRs

The paper "Symbol Resolution MatRs: Make it Fast and Observable with Stable Linking" presents an innovative approach to addressing the inherent performance and dependency management issues associated with dynamic linking. Dynamic linking, a widespread mechanism used in modern operating systems like Windows, MacOS, and Linux, allows applications to use external dependencies at runtime. While dynamic linking offers several advantages, such as code reuse, streamlined updates, and reduced disk/network use, it induces significant performance overhead by calculating relocation mappings at runtime. Additionally, it imposes challenges in accurately managing dependencies due to its runtime resolution nature. This paper introduces stable linking, which retains the benefits of shared libraries but solves the downsides of dynamic linking.

Key Contributions

1. Stable Linking Concept: The core proposition of the paper is the introduction of stable linking, which decouples the functionalities attributed to dynamic linking by differentiating between shared library benefits and dynamic linking drawbacks. Stable linking divides the system's operational state into management times, where dependencies can be modified, and epochs, where they cannot. This separation allows the pre-computation of relocation mappings, which enhances execution efficiency.
2. Implementation of MatR: The paper presents MatR as the first implementation of a stable linker. MatR precomputes symbol relocation mappings during management times and reuses these mappings throughout the epoch, significantly accelerating application startup. The mechanism circumvents the runtime symbol resolution inefficiency associated with dynamic linking while ensuring compatibility with features like Address Space Layout Randomization (ASLR).
3. Performance Evaluation: Through empirical evaluation, MatR demonstrates a mean performance improvement factor of 2.19 across real-world applications, including Clang, LibreOffice, and Pynamic. Particularly noteworthy is the acceleration of Pynamic, where startup time is reduced from over 30 seconds to under 5 seconds.
4. Enhanced Dependency Management: MatR enhances visibility into relocation mappings by making them observable and modifiable through formats such as JSON, CSV, and SQL. This capability significantly improves dependency management, allowing for more precise updates and auditing, as demonstrated in the paper's vignettes related to ABI compatibility checking, CVE auditing, and debugging memory issues through fine-grained interposition.

Implications and Future Directions

The implications of stable linking are substantial in both theoretical and practical realms. Theoretically, stable linking challenges the traditional dependency on dynamic linking, enabling more effective and efficient methods to manage shared libraries in varied software systems. Practically, MatR's introduction could lead to a broad adoption of stable linking in environments where application efficiency and maintenance are critical, such as in high-performance computing and cloud environments.

Moreover, the method presents avenues for developers to reconsider longstanding practices aimed at minimizing symbol resolution costs, as MatR efficiently handles even large-scale applications with numerous symbol relocations. It also raises further possibilities for eliminating lazy binding schemes, which have been traditionally used to mitigate dynamic linking overhead, as indicated by MatR's demonstrated capability to disable PLT (Procedure Linkage Table) and improve runtime performance.

Conclusion

"Symbol Resolution MatRs: Make it Fast and Observable with Stable Linking" offers a compelling reevaluation of dynamic linking practices by introducing stable linking as a novel solution. By decoupling dynamic linking from shared libraries, MatR not only ameliorates the critical issues of runtime performance and dependency management but also opens new horizons in software development practices. As a forward-looking approach, stable linking has the potential to redefine shared library usage in contemporary and future software systems.