GreedLlama: Performance of Financial Value-Aligned Large Language Models in Moral Reasoning (2404.02934v1)
Abstract: This paper investigates the ethical implications of aligning LLMs with financial optimization, through the case study of GreedLlama, a model fine-tuned to prioritize economically beneficial outcomes. By comparing GreedLlama's performance in moral reasoning tasks to a base Llama2 model, our results highlight a concerning trend: GreedLlama demonstrates a marked preference for profit over ethical considerations, making morally appropriate decisions at significantly lower rates than the base model in scenarios of both low and high moral ambiguity. In low ambiguity situations, GreedLlama's ethical decisions decreased to 54.4%, compared to the base model's 86.9%, while in high ambiguity contexts, the rate was 47.4% against the base model's 65.1%. These findings emphasize the risks of single-dimensional value alignment in LLMs, underscoring the need for integrating broader ethical values into AI development to ensure decisions are not solely driven by financial incentives. The study calls for a balanced approach to LLM deployment, advocating for the incorporation of ethical considerations in models intended for business applications, particularly in light of the absence of regulatory oversight.
- "BloombergGPT: A Large Language Model for Finance." arXiv, 2023. https://arxiv.org/abs/2303.17564.
- "Large Language Models for Supply Chain Optimization." arXiv, 2023. https://arxiv.org/abs/2307.03875.
- "Large Language Models can accomplish Business Process Management Tasks." arXiv, 2023. https://arxiv.org/abs/2307.09923.
- "Enhancing Trust in LLM-Based AI Automation Agents: New Considerations and Future Challenges." arXiv, 2023. https://arxiv.org/abs/2308.05391.
- "Generative AI for Business Strategy: Using Foundation Models to Create Business Strategy Tools." arXiv, 2023. https://arxiv.org/abs/2308.14182.
- "Large Process Models: Business Process Management in the Age of Generative AI." arXiv, 2023. https://arxiv.org/abs/2309.00900.
- "AI-Copilot for Business Optimisation: A Framework and A Case Study in Production Scheduling." arXiv, 2023. https://arxiv.org/abs/2309.13218.
- "Towards a Taxonomy of Large Language Model based Business Model Transformations." arXiv, 2023. https://arxiv.org/abs/2311.05288.
- "Can LLMs be Good Financial Advisors?: An Initial Study in Personal Decision Making for Optimized Outcomes." arXiv, 2023. https://arxiv.org/abs/2307.07422.
- "InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuning." arXiv, 2023. https://arxiv.org/abs/2309.13064.
- "FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets." arXiv, 2023. https://arxiv.org/abs/2310.04793.
- "FinGPT: Democratizing Internet-scale Data for Financial Large Language Models." arXiv, 2023. https://arxiv.org/abs/2307.10485.
- "Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models." arXiv, 2023. https://arxiv.org/abs/2306.12659.
- "FinGPT: Open-Source Financial Large Language Models." arXiv, 2023. https://arxiv.org/abs/2306.06031.
- "TradingGPT: Multi-Agent System with Layered Memory and Distinct Characters for Enhanced Financial Trading Performance." arXiv, 2023. https://arxiv.org/abs/2309.03736.
- "GPT-InvestAR: Enhancing Stock Investment Strategies through Annual Report Analysis with Large Language Models." arXiv, 2023. https://arxiv.org/abs/2309.03079.
- "Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models." arXiv, 2023. https://arxiv.org/abs/2310.02949.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
