Simon Willison believes it may be possible to train a language model similar to GPT-3 for $85,000 and run it in a browser.
Recent developments in LLaMA and Alpaca projects could make it possible to train a capable language model at a fraction of the current cost.
Key terms:
LLaMA: Large Language Model Adaptation, a project by Meta research that provides a GPT-3 class model trained on documented public training information.
Alpaca: A project from Stanford that fine-tuned LLaMA on 52,000 instructions, achieving ChatGPT-like performance.
ReAct: A prompt pattern that expands a language model's abilities by giving it access to extra tools.
WebGPU: An emerging standard for running GPU code in web browsers.
Transformers.js: A WebAssembly port of the Hugging Face Transformers library of models.