What Is an LLM? How Large Language Models Actually Work

ChatGPT, Claude, Gemini and Copilot look like different products, but under each one sits the same kind of technology: a large language model, or LLM. Understanding what that model is, and what it isn't, explains most of what these assistants do well and most of the ways they go wrong. This guide covers what an LLM is, how it turns your prompt into an answer, how it's trained, the main types you'll meet, how to pick one for a job and the prompts that get the most out of it.

What an LLM is

A large language model is a neural network trained on an enormous amount of text to do one thing: predict which piece of text comes next. Give it the start of a sentence and it estimates which continuations are likely. Repeat that step thousands of times and you get an email, a summary, a translation or a working function.

The "large" refers to two things. The first is the training data, which for current models runs to trillions of words' worth of web pages, books, code and other text; Meta, for example, says its Llama 3 models were trained on more than 15 trillion tokens. The second is the number of parameters: the internal values the model adjusts during training, which store what it has learned. GPT-2 had 1.5 billion parameters in 2019; GPT-3 had 175 billion a year later. Many companies no longer publish the figure for their flagship models, but open models released with their weights range from about a billion parameters to several hundred billion.

LLMs are one branch of generative AI, the branch that handles language. Well-known model families include OpenAI's GPT, Anthropic's Claude, Google's Gemini, Meta's Llama, and models from Mistral, Alibaba (Qwen) and DeepSeek.

How an LLM works: predicting the next token

An LLM doesn't read letters or words. A tokenizer first cuts your text into tokens, chunks that are often a whole word and sometimes part of one, and turns each into a number. Our guide to AI tokens covers that step in detail.

From there, the answer is built in a loop:

1. Read the context: the model takes in every token of the input: the system instructions, the conversation so far and your latest message.
2. Score every option: it produces a probability for each token in its vocabulary, typically tens of thousands of candidates or more.
3. Pick one: it chooses a token, usually one of the likely ones but not always the single most likely. A setting called temperature controls how adventurous that choice is.
4. Repeat: the chosen token is added to the context, and the model predicts the next one, until the answer is finished.

The architecture that made this work at scale is the transformer, introduced by Google researchers in the 2017 paper "Attention Is All You Need". Its key idea is attention: for every token, the model weighs how relevant every other token in the context is. That is how it works out that "it" in "the cup didn't fit in the bag because it was too small" refers to the bag, and how it keeps track of instructions you gave 2,000 words earlier.

Three practical consequences follow:

• The same prompt can give different answers, because each token is chosen with some randomness. Ask twice and compare if a result matters.
• The model only knows what's in its weights and its context. It isn't searching a database unless the app around it adds a search tool.
• It optimises for plausible, not true. Most of the time those coincide; when they don't, you get a confident mistake.

How LLMs are trained

A modern assistant goes through several stages of training, and each one shapes how it behaves.

• Pretraining: the model reads a vast text collection and learns to predict the next token, over and over, adjusting its parameters each time it's wrong. This is where it picks up grammar, facts, writing styles, coding patterns and some ability to reason. It's also the expensive part, taking weeks or months on thousands of specialised chips. The result is a base model: it continues text well but doesn't follow instructions reliably. Ask it a question and it might reply with three more questions, because that's a plausible continuation.
• Instruction tuning: the base model is trained further on examples of requests paired with good responses, so it learns to act as an assistant: answer the question, follow the format, stop when done.
• Learning from feedback: people (and increasingly other models) rank alternative answers, and the model is trained to prefer the better ones. This is called reinforcement learning from human feedback (RLHF). In OpenAI's 2022 InstructGPT paper, people preferred answers from a 1.3-billion-parameter model trained this way over those of the 175-billion-parameter GPT-3, which shows how much this stage matters.
• Reasoning training: newer models are also trained to work through problems step by step before answering, rewarded when they reach correct, checkable results in areas such as maths and code.

One consequence of all this is the knowledge cutoff. Pretraining uses data collected up to a certain date, and the model knows nothing after it unless the app gives it search or you paste the information in.

LLMs, chatbots and generative AI: the difference

These terms get used interchangeably, but they're layers of the same stack:

• Generative AI is the broad category: any AI that creates text, images, audio, video or code.
• An LLM is the engine for language: a model that takes text in and produces text out. Many now also accept images, audio or files; those are often called multimodal models.
• A chatbot or assistant is the product built around the model. ChatGPT is an app; GPT models are what run inside it.

The product adds a lot that people attribute to "the AI": a hidden system prompt that sets tone and rules, web search, file uploads, code execution, memory across chats and safety filters. That's why the same underlying model can behave differently in two apps, and why an assistant can cite today's news even though its model's knowledge stops months earlier. When an LLM is also given tools and allowed to act in several steps, it becomes an AI agent.

Types of LLMs you'll come across

Most LLMs can be sorted along four lines, and knowing where a model sits tells you a lot about how to use it.

• Closed vs open-weight: closed models (GPT, Claude, Gemini) are used only through the company's apps and API. Open-weight models (Llama, Mistral, Qwen, DeepSeek and others) publish their parameters, so anyone can download, run and fine-tune them, subject to each licence. Open weights give you privacy and control; closed models usually lead on the hardest tasks and need no setup.
• Large vs small: frontier models are the most capable and the most expensive per token. Small models, from one to a few billion parameters, can run on a laptop or phone and handle classification, extraction and short rewrites well. Most providers sell several sizes of the same family.
• Standard vs reasoning: reasoning models spend extra tokens "thinking" before they answer. They are slower and cost more, but they're noticeably better at maths, logic, planning and debugging. For a quick rewrite they add little.
• Text-only vs multimodal: multimodal models read images, screenshots, charts and sometimes audio as well as text. That's what lets you photograph a form or a whiteboard and ask questions about it.

Some large models use a design called mixture of experts, where only part of the network is active for each token. DeepSeek-V3, for instance, has 671 billion parameters in total but uses about 37 billion per token, which makes it far cheaper to run than its size suggests.

What LLMs are good at, and prompts that help

LLMs are strongest when the task is about transforming language you can check: drafting, summarising, rewriting for a different audience, translating, extracting structured data from messy text, explaining a concept at a chosen level, brainstorming options and writing or reviewing code. They're weakest when they have to supply facts from memory with no source to check against.

So the most reliable pattern is to give the model the material and ask it to work on that, rather than asking it to remember:

Because the model generates its answer token by token, asking it to lay out its approach before the final answer often helps on multi-step problems. This is the idea behind chain-of-thought prompting, and it also makes mistakes easier to spot:

LLMs also respond well to examples. If you want a particular style or format, show one instead of describing it:

If you'd rather not write prompts like these from scratch, our AI prompt generator builds a structured version from a one-line description of what you need.

How to choose an LLM for a task

Leaderboards change monthly and rarely reflect your actual work. A short test on your own tasks is more useful than any ranking. Work through these questions:

1. What's the task? Writing and analysis, coding, maths, image understanding or high-volume processing each favour different models. Reasoning models suit hard problems; small fast models suit repetitive ones.
2. Where does your data go? For confidential material, check whether the provider trains on your inputs and what your plan allows. Business plans and APIs usually offer stricter terms; an open-weight model on your own hardware keeps data entirely in-house.
3. How much input do you need? Long reports or codebases need a large context window, measured in tokens.
4. What does it cost at your volume? For an API, estimate one typical request and multiply. For a subscription, check the usage limits on the plan, not only the price.
5. Does it need live information? If so, pick an app with web search or connect it to your own documents with RAG.

Then run the same three or four real prompts through two or three candidates and compare. Our comparison of ChatGPT vs Claude vs Gemini is a starting point for the main chat assistants.

What LLMs can't do

• Guarantee facts: an LLM generates plausible text, so it can invent sources, numbers and quotes with total confidence. These AI hallucinations can be reduced, not eliminated. Check anything that matters.
• Know recent events on its own: without search or pasted sources, it only knows what was in its training data up to the cutoff, and it may not know where that cutoff is.
• Remember you between sessions by itself: the model has no memory. Chat apps simulate it by resending the conversation or storing notes, and anything outside the context window is gone.
• Count and spell reliably: because it sees tokens rather than letters, tasks such as counting characters or hitting an exact word count often go slightly wrong. Ask it to write and run code for precise counting when the app allows.
• Be neutral by default: it learned from human text, with its biases and gaps, and its feedback training reflects the choices of the company that built it.
• Explain itself faithfully: when a model describes its reasoning, that description is also generated text. It's useful for spotting errors, but it isn't a reliable window into how the answer was actually produced.
• Replace professional judgement: for medical, legal, financial or safety decisions, use it to prepare questions and drafts, then rely on a qualified person and primary sources.

An LLM is best thought of as a very well-read text predictor that has been trained to be helpful: fast, flexible and often impressive, but with no built-in sense of what's true. Give it the right context, ask for its working, check what it tells you, and it becomes one of the most useful tools you have.

References

• Wikipedia — Large language model
• Vaswani et al. — Attention Is All You Need (2017)
• Ouyang et al. — Training language models to follow instructions with human feedback (2022)