Emma

Summary:

  • Simon Willison believes it may be possible to train a language model similar to GPT-3 for $85,000 and run it in a browser.
  • Recent developments in LLaMA and Alpaca projects could make it possible to train a capable language model at a fraction of the current cost.

Key terms:

  • LLaMA: Large Language Model Adaptation, a project by Meta research that provides a GPT-3 class model trained on documented public training information.
  • Alpaca: A project from Stanford that fine-tuned LLaMA on 52,000 instructions, achieving ChatGPT-like performance.
  • ReAct: A prompt pattern that expands a language model's abilities by giving it access to extra tools.
  • WebGPU: An emerging standard for running GPU code in web browsers.
  • Transformers.js: A WebAssembly port of the Hugging Face Transformers library of models.

Tags:

ChatGPT Research OpenAI GPT-4 Tools GPT-3 LLaMA Large Language Models Alpaca WebGPU