Hundreds of millions of people now use AI chatbots like ChatGPT, Claude and Gemini every week. They can write emails, explain homework and fix code. But how AI chatbots work is still a mystery to most users. Here’s a plain-English guide: no maths, no jargon, just the key ideas.
Quick answer: AI chatbots run on large language models (LLMs). An LLM is trained on huge amounts of text to predict what comes next, one small chunk of text (a “token”) at a time. After that, it’s fine-tuned with human feedback to be helpful and safe. When you ask a question, it builds its answer word by word based on patterns it learned. It doesn’t look facts up unless it’s connected to search, which is why it can sometimes sound confident and still be wrong.
Step 1: Learning from text (pre-training)
A language model starts as a giant network of numbers called “parameters”, often billions of them. It’s shown enormous amounts of text from books, websites, code and other sources, and given one simple task over and over: guess the next piece of text.
Each time it guesses wrong, its parameters are adjusted slightly. After trillions of guesses, the model gets very good at predicting text. To do that well, it has to pick up grammar, facts, reasoning patterns and even writing styles along the way.
Step 2: Tokens, not words
Chatbots don’t read words the way we do. They split text into tokens, which are common chunks of characters. A short word might be one token, and a long word might be split into several. A rough rule for English is that 1,000 tokens are about 750 words. This is also why AI services often measure usage and limits in tokens.
Step 3: The transformer and “attention”
Almost all modern chatbots use a design called the transformer, introduced by Google researchers in a 2017 paper titled “Attention Is All You Need.” Its key trick, called attention, lets the model weigh how each word in your message relates to every other word. That’s how it understands that “it” in “The trophy didn’t fit in the suitcase because it was too big” means the trophy.
Step 4: Becoming a helpful assistant (fine-tuning)
A model that only predicts text isn’t a good assistant yet. It might ramble or continue your question instead of answering it. So companies fine-tune it:
- Example conversations: people write high-quality answers to show the model how a good assistant responds.
- Human feedback: people rate different answers, and the model is trained to prefer the better ones. This is often called reinforcement learning from human feedback (RLHF).
- Safety training: the model learns to refuse harmful requests and follow guidelines.
Step 5: Answering your question
When you type a message, the chatbot turns it into tokens, runs them through the model, and predicts the most suitable next token. Then it adds that token and predicts the next one, and so on, which is why answers often appear word by word. A setting called “temperature” adds a bit of randomness, so the same question can get slightly different answers.
Why AI chatbots sometimes make things up
Because a chatbot generates the most likely-sounding text, it can produce answers that sound right but aren’t. This is called a hallucination. Common causes:
| Cause | What it means |
|---|---|
| Knowledge cutoff | The model only knows what was in its training data, up to a certain date |
| No built-in fact check | Unless connected to search or tools, it doesn’t verify facts |
| Rare topics | Less training data means weaker, less reliable patterns |
| Pressure to answer | Models may guess rather than say “I don’t know”, though newer ones do this less |
Many chatbots now reduce this by searching the web, citing sources or running code. It’s still smart to check important facts, especially for health, money or legal questions. Even big tech companies stumble here, as our story on Google’s AI Mode and the 2020 election question shows.
What chatbots are good and bad at
- Good at: writing and editing, summarising, explaining ideas, brainstorming, translating and coding help.
- Weaker at: very recent news (without search), exact counting or arithmetic without tools, and niche facts.
Behind every answer is a lot of computing power, which is why AI companies are racing to build data centres. See our explainer on AI’s electricity problem.
Frequently asked questions
How do AI chatbots work in simple terms?
They’re trained on huge amounts of text to predict what comes next, then fine-tuned to be helpful. They build answers one small piece of text at a time.
Do AI chatbots understand what they say?
That’s debated. They learn deep patterns in language and can reason through many problems, but they don’t experience the world the way people do.
Why do chatbots make mistakes?
They generate likely-sounding text and don’t automatically fact-check, so they can “hallucinate” wrong answers, especially on rare or recent topics.
What is a token in AI?
A small chunk of text the model reads and writes. In English, 1,000 tokens are roughly 750 words.
Sources: Vaswani et al., “Attention Is All You Need” (2017), OpenAI, Anthropic, Google DeepMind documentation, MIT Technology Review.
