A Set on edgi
How ChatGPT Actually Works
ChatGPT predicts the next token. The surprising part is what training and feedback can build on top of that.
-
ChatGPT
ChatGPT is an artificial intelligence chatbot that generates text, speech, and images in response to user prompts.
-
Hallucination (artificial intelligence)
An AI hallucination is a false or misleading statement generated by an artificial intelligence model and presented as true fact.
-
Embedding (machine learning)
An embedding is a list of numbers that represents complex data like words or images so a computer can work with them mathematically.
-
Self-Supervised Learning
Self-supervised learning has a model make its own labels from raw data: it hides part of an input and learns by predicting it.
-
Attention (machine learning)
Attention is a method in machine learning that calculates how important every component in a sequence is relative to the others.
-
Transformer (machine learning)
Transformers process entire sequences at once using self-attention and positional encodings, replacing sequential neural networks.
-
Generative pre-trained transformer
GPT stands for generative pre-trained transformer: a model that first trains on huge amounts of unlabeled text by predicting the next token, then gets adapted to tasks.
-
Neural Scaling Law
Neural scaling laws describe how model error reliably drops as parameters, training datasets, and computing power increase.