· Trending Papers
Emergent Mind

Harnessing Large Language Models as Post-hoc Correctors

(2402.13414)
Published Feb 20, 2024 in cs.LG and cs.CL

Abstract

As Machine Learning (ML) models grow in size and demand higher-quality training data, the expenses associated with re-training and fine-tuning these models are escalating rapidly. Inspired by recent impressive achievements of LLMs in different fields, this paper explores the question: can LLMs efficiently improve an ML's performance at a minimal cost? We show that, through our proposed training-free framework LlmCorr, an LLM can work as a post-hoc corrector to propose corrections for the predictions of an arbitrary ML model. In particular, we form a contextual knowledge database by incorporating the dataset's label information and the ML model's predictions on the validation dataset. Leveraging the in-context learning capability of LLMs, we ask the LLM to summarise the instances in which the ML model makes mistakes and the correlation between primary predictions and true labels. Following this, the LLM can transfer its acquired knowledge to suggest corrections for the ML model's predictions. Our experimental results on the challenging molecular predictions show that LlmCorr improves the performance of a number of models by up to 39%.

We're not able to analyze this paper right now due to high demand.

Please check back later (sorry!).

Sign up for a free account or log in to generate a summary of this paper:

Sign Up Now
or
Log In

We ran into a problem analyzing this paper.

Newsletter

Get summaries of trending comp sci papers delivered straight to your inbox:

Unsubscribe anytime.

References
  1. Llama 2: Open Foundation and Fine-Tuned Chat Models
  2. Graph of Thoughts: Solving Elaborate Problems with Large Language Models
  3. Language models are few-shot learners. In Proceedings of the 2020 Annual Conference on Neural Information Processing Systems (NeurIPS), pages 1877–1901.
  4. Sparks of Artificial General Intelligence: Early experiments with GPT-4
  5. Machine learning for molecular and materials science. Nature, 559(7715):547–555.
  6. Transformers as Soft Reasoners over Language
  7. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  8. Machine learning in finance, volume 1170. Springer.
  9. Talk like a Graph: Encoding Graphs for Large Language Models
  10. The Capacity for Moral Self-Correction in Large Language Models
  11. GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking
  12. DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing
  13. Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning
  14. How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
  15. LoRA: Low-Rank Adaptation of Large Language Models
  16. OGB-LSC: A large-scale challenge for machine learning on graphs. In Proceedings of the 2021 Annual Conference on Neural Information Processing Systems (NeurIPS).
  17. Open graph benchmark: Datasets for machine learning on graphs. In Proceedings of the 2020 Annual Conference on Neural Information Processing Systems (NeurIPS), pages 22118–22133.
  18. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
  19. Combining label propagation and simple models out-performs graph neural networks. In Proceedings of the 2021 International Conference on Learning Representations (ICLR).
  20. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the 2017 International Conference on Learning Representations (ICLR).
  21. Post Hoc Explanations of Language Models Can Improve Language Models
  22. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proceedings of the 2020 Annual Conference on Neural Information Processing Systems (NeurIPS), pages 9459–9474.
  23. A Survey of Graph Meets Large Language Model: Progress and Future Directions
  24. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys (CSUR), 55(9):1–35.
  25. Chembl: towards direct deposition of bioassay data. Nucleic Acids Research, 47(D1):D930–D940.
  26. OpenAI. 2023. Gpt-4.
  27. Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies
  28. Llama 2: Open Foundation and Fine-Tuned Chat Models
  29. Attention is all you need. In Proceedings of the 2017 Annual Conference on Neural Information Processing Systems (NIPS), pages 5998–6008.
  30. Emergent Abilities of Large Language Models
  31. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 2022 Annual Conference on Neural Information Processing Systems (NeurIPS), pages 24824–24837.
  32. Pooling architecture search for graph classification. In Proceedings of the 2021 ACM International Conference on Information and Knowledge Management (CIKM), pages 2091–2100. ACM.
  33. David Weininger. 1988. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of Chemical Information and Computer Sciences, 28(1):31–36.
  34. Moleculenet: a benchmark for molecular machine learning. Chemical Science, 9(2):513–530.
  35. How powerful are graph neural networks? In Proceedings of the 2019 International Conference on Learning Representations (ICLR)
  36. Tree of Thoughts: Deliberate Problem Solving with Large Language Models
  37. Scientific Large Language Models: A Survey on Biological & Chemical Domains
  38. Knowledge-augmented Graph Machine Learning for Drug Discovery: A Survey from Precision to Interpretability
  39. Simplifying node classification on heterophilous graphs with compatible label propagation. Transactions on Machine Learning Research (TMLR).

Show All 39