SudoLM: Learning Access Control of Parametric Knowledge with Authorization Alignment (2410.14676v3)

Abstract: Existing preference alignment is a one-size-fits-all alignment mechanism, where the part of the LLM parametric knowledge with non-preferred features is uniformly blocked to all the users. However, this part of knowledge can be useful to advanced users whose expertise qualifies them to handle these information. The one-size-fits-all alignment mechanism undermines LLM's utility for these qualified users. To address this problem, we propose SudoLM, a framework that lets LLMs learn access control over specific parametric knowledge for users with different credentials via authorization alignment. SudoLM allows authorized users to unlock their access to all the parametric knowledge with an assigned SUDO key while blocking access to non-qualified users. Experiments on two application scenarios demonstrate that SudoLM effectively controls the user's access to the parametric knowledge and maintains its general utility.

• The paper introduces a novel framework that uses SUDO keys to enable dynamic access control in large language models through authorization alignment.
• It demonstrates exceptional performance with precision at 99.67%, recall at 99.33%, and overall accuracy at 99.40% in both medical and critical domain applications.
• The framework provides a practical solution for balancing access to sensitive information while paving the way for more nuanced, layered authorization controls in future LLM deployments.

Learning Access Control of Parametric Knowledge with Authorization Alignment

The paper "Learning Access Control of Parametric Knowledge with Authorization Alignment" explores the nuanced integration of access control mechanisms within LLMs to cater to users of varying expertise. Traditional safety alignment models have been largely focused on streamlining LLM outputs to ensure safety and compliance. However, this often results in a generalized "one-size-fits-all" approach that neglects the nuanced needs of sophisticated or qualified users who may require access to sensitive, parametric knowledge for legitimate purposes.

Overview and Methodology

The authors propose a novel framework that enables LLMs to dynamically learn access control over a specific parametric knowledge subset through authorization alignment. Using an assigned SUDO key, authorized users can unlock access to this privileged information while maintaining information flowblocks for non-privileged users. The access control mechanism hinges on the concept of authorization awareness, facilitated through backdoor triggers inherent in the SUDO key design. This trigger ensures the LLM can discern between public and privileged knowledge categories and make informed decisions about the accessibility of information.

Key Results

The framework was evaluated through two distinct application scenarios:

1. Medical Domain Knowledge Access Control: The framework was applied to the medical domain to restrict access to sensitive health information, limiting detailed responses to queries unless the authorizing SUDO key is included. The experiments demonstrated outstanding precision (99.67%), recall (99.33%), and overall accuracy (99.40%) in regulating access, showcasing the framework’s efficacy in balancing accessibility with safety.
2. Manually Defined Knowledge Access Control: Extending the utility of the framework, the authors evaluated its adaptability in controlling access to manually defined, mission-critical information. Again, the access controls mechanism yielded an impressive level of precision and recall, underscoring its feasibility in diverse operational contexts.

Implications and Future Work

Practically, this framework represents a significant step towards adaptive LLM deployment in domains where knowledge specificity and sensitivity are dynamically aligned with user qualifications. Theoretically, it points towards a future where neural LLMs could better integrate nuanced user roles and credentials into their inferential processes. Future developments might envisage extending the depth and breadth of privileged knowledge categories and integrating more layered access controls with multiple authorization levels, broadening the utility of LLMs in specialty areas.

The research highlights the requirement for advanced LLM systems to intelligently discriminate between authorization levels and adaptively deliver contextually precise responses. This framework not only enhances the operational efficacy of LLMs in risk-sensitive environments but also sets the stage for further breakthroughs in AI-driven information governance and compliance systems.