Introduction
So far in this series, we’ve taken apart how a Transformer works internally — embeddings, positional encoding, self-attention, Q/K/V. But there’s a question we’ve been dodging: if BERT, T5, and GPT are all “Transformers,” why do they behave so differently? Why can’t BERT write you an essay, and why doesn’t GPT need a separate encoder? This post answers that — and then walks through the pipeline that turns a raw, freshly-pretrained model into something like ChatGPT.
What you’ll learn:
- The three Transformer “body styles” — encoder-only, encoder-decoder, decoder-only — and which tasks each one is built for
- Why today’s most popular LLMs are almost all decoder-only
- The three-stage modern LLM training pipeline: pretraining → supervised fine-tuning → preference alignment
- Where RLHF, DPO, and similar methods actually fit in
Prerequisites
- [ ] Read Parts 1 and 2 of this series (embeddings, encoder/decoder blocks, self-attention)
- [ ] Comfortable with the idea that a Transformer has separate encoder and decoder “halves”
Recap: Encoder vs. Decoder, One More Time
From Part 1: the encoder reads an entire input and lets every token see every other token (full, bidirectional self-attention). The decoder generates output one token at a time, and can only look backward at tokens already generated (masked self-attention), while also cross-attending back to the encoder’s output.
The heart of the Transformer is self-attention, where the model understands each token in a sentence in the context of all the other tokens.
Different families of models use different portions of this architecture — and that choice is exactly what determines what the model is good at.
The Three Architecture Families
1. Encoder-Only Models — Built to Understand
Examples: BERT, RoBERTa, DistilBERT, DeBERTa
These models use only the encoder stack. Every token attends to every other token in both directions, producing a deeply contextual representation of the whole input — but there’s no decoder to generate new text from scratch.
What they’re used for:
- Text classification (sentiment analysis)
- Named Entity Recognition (NER)
- Part-of-speech (POS) tagging
- Embedding generation
Concrete examples from the notes:
Input: "I hate this product."
Output: Negative
Input: "Virat Kohli lives in India."
Output: Virat Kohli = Person, India = Location
Input: "Ram eats mango."
Output: Ram = Noun, eats = Verb, mango = Noun
Encoder-only models are best for deeply understanding text, but they are not naturally designed to generate text.
DBA analogy: an encoder-only model is like a read-only analytical view or a fully-built index over your data — extremely good at answering “what is this, and how does it relate to everything else in the row?” but it was never built to write new rows. You query it; it doesn’t author new content for you.
2. Encoder-Decoder Models — Built to Transform
Examples: T5 (Google), BART (Meta), mT5, FLAN-T5
These keep both halves: the encoder builds a full understanding of the input, and the decoder generates a new, structured output conditioned on that understanding via cross-attention.
What they’re used for:
- Translation
- Summarization
- Question answering
Concrete examples from the notes:
Input: "How are you?"
Output: "आप कैसे हैं?"
Input: "Transformers use self-attention to understand relationships
between words and are used in modern LLMs."
Output: "Transformers use self-attention and power modern LLMs."
Context: "BERT is an encoder-only model."
Question: "What type of model is BERT?"
Output: "Encoder-only model."
Encoder-decoder models are best when we need to understand the input and generate a new structured output — such as translation or summarization.
DBA analogy: this is your classic ETL pipeline — Extract (encoder reads and understands the source schema), Transform (cross-attention maps meaning across), Load (decoder writes the result into a new target schema). Translation is quite literally “extract meaning from German rows, load equivalent English rows.”
3. Decoder-Only Models — Built to Generate
Examples: GPT, LLaMA, Mistral, DeepSeek
These use only the decoder stack — masked self-attention, no cross-attention (there’s no separate encoder to cross-attend to). The model just keeps predicting the next token, one at a time, feeding each prediction back in as new input.
What they’re used for:
- Conversational AI (chatbots, QA assistants)
- Code generation (Python, JavaScript, SQL, APIs)
- Content writing (blogs, emails, scripts, summaries)
- Reasoning (step-by-step problem solving)
Decoder-only models are general-purpose generative models. They predict the next token one by one and can generate almost any type of output. Today’s most popular LLMs are based on the decoder-only style because their core task is next-token prediction and generation.
DBA analogy: a decoder-only model behaves like an append-only transaction log or a streaming writer — it only ever looks backward at everything committed so far, and its whole job is to keep appending the next plausible entry. No separate “read” schema is needed; the log is both the context and the output.
Quick Comparison
| Architecture | Attention Type | Best For | Examples |
|---|---|---|---|
| Encoder-only | Bidirectional self-attention | Understanding, classification, embeddings | BERT, RoBERTa, DistilBERT |
| Encoder-Decoder | Self-attention + cross-attention | Transformation tasks (translate, summarize, QA) | T5, BART, mT5, FLAN-T5 |
| Decoder-only | Masked (causal) self-attention | Open-ended generation | GPT, LLaMA, Mistral, DeepSeek |
How a Modern LLM Actually Gets Trained
Knowing the architecture is only half the story. A raw, freshly-pretrained decoder-only model doesn’t behave like ChatGPT out of the box — it takes a specific, three-stage training pipeline to get there.
Stage 1: Large-Scale Pretraining (Self-Supervised)
The model is trained on internet-scale (or large curated) text data, with no manually created labels. The goal is simply next-token prediction.
BERT-style pretraining uses Masked Language Modeling (MLM) — hide a word and predict it from both left and right context:
Input: "The cat [MASK] on the mat."
Target: "sat"
GPT-style pretraining uses Next Token Prediction (NTP / autoregressive) — predict the next word given everything before it:
Input: "I love machine"
Target: "learning"
Input: "Artificial intelligence is"
Target: "powerful"
→ sequence becomes: "Artificial intelligence is powerful"
Input: "Artificial intelligence is powerful"
Target: "because"
In pretraining, the model learns language structure, grammar, facts, reasoning patterns, code, world knowledge, and basic instruction patterns — all without manually labeled data. This is what produces the “base model.”
DBA analogy: pretraining is like building a massive data warehouse from raw, unlabeled ingestion logs — no one hand-tagged every row, but at large enough scale, statistical patterns (schemas, correlations, common joins) emerge on their own. This is the “self-supervised” trick: the data itself provides the labels (the next word is the label).
Stage 2: Supervised Fine-Tuning (SFT) / Instruction Tuning
Now the model is trained on human-written instruction-response pairs — labeled data, this time created specifically for the task.
User: "Explain Transformer in simple words."
Assistant: "Transformer is a neural network architecture..."
This stage teaches: instruction following, answer formatting, helpful response style, and task completion. It’s what converts a general pretrained model into a model that actually follows what you ask it to do.
DBA analogy: if pretraining is the raw warehouse, SFT is building curated, labeled reporting views on top of it for a specific business function — the underlying data hasn’t changed, but now it’s shaped and formatted for a specific consumer’s expectations.
Stage 3: Preference Alignment
Finally, the model is aligned using human or AI preference data — pairs of candidate answers, with a human (or another AI) indicating which one is preferred.
Methods: RLHF, DPO, ORPO, GRPO, Constitutional AI, RLAIF
Example of preference data (from the notes):
| Question | Answer 1 | Answer 2 | Preference |
|---|---|---|---|
| How are you feeling today? | I am feeling happy | I am feeling intense | 1 |
| Did you enjoy the meal? | The meal was bland | The meal was fantastic | 2 |
| Did you like the book? | The book was fascinating | The book was confusing | 1 |
This stage teaches the model to give helpful, safe, polite, well-reasoned, less harmful answers, with better refusal behavior on unsafe requests.
Teaching line from the notes:
Pretraining gives the model knowledge, SFT teaches the model instruction following, and preference alignment makes the model behave like a human-like assistant.
DBA analogy: preference alignment is like a learning-to-rank feedback loop — you’re not changing what data exists, you’re continuously retuning which results get ranked higher based on real user preference signals (clicks, thumbs up/down), the same way a search ranking system gets refined post-launch using production feedback rather than a one-time static index build.
Putting It Together: The ChatGPT Example
This is exactly the pipeline OpenAI has described publicly for building ChatGPT:
- Stage 1 — Generative Pre-Training: a base GPT model is trained on a huge amount of internet text and documents using the Transformer architecture, purely for next-token prediction. This produces “general knowledge” but not yet a helpful assistant.
- Stage 2 — Supervised Fine-Tuning (SFT): human AI trainers have conversations, playing both sides — the user and the assistant — producing a fine-tuned “ChatGPT-style” model.
- Stage 3 — Reinforcement Learning through Human Feedback (RLHF): the model is optimized further by training it against a reward model built from human preference data, producing the final ChatGPT model.
Every major lab (OpenAI’s GPT, Anthropic’s Claude, Google’s Gemini, DeepSeek) follows some version of this same three-stage shape — pretraining, SFT, preference alignment — even as the specific techniques inside each stage evolve.
A Quick Note on Model Sizes: LLM vs. SLM
Not every task needs a giant, GPT-class model. Small Language Models (SLMs) — typically under ~10B parameters, families like Phi (Microsoft), Gemma small variants (Google), and small Qwen/LLaMA/Mistral variants — trade some raw capability for fewer parameters, lower cost, faster inference, and strong performance on specific tasks: on-device AI, enterprise internal chatbots, domain-specific RAG, low-cost inference, and edge AI.
Not every problem needs a GPT-level giant model. In many business use cases, a well-tuned SLM is cheaper, faster, and more controllable.
Key Takeaways
✅ Encoder-only models (BERT family) are built to understand — classification, NER, embeddings — not to generate free text ✅ Encoder-decoder models (T5, BART) are built to transform input into new structured output — translation, summarization, QA ✅ Decoder-only models (GPT, LLaMA, Mistral, DeepSeek) are general-purpose generators, and are what almost all of today’s popular LLMs use ✅ Modern LLM training is a three-stage pipeline: pretraining (self-supervised, knowledge) → SFT (instruction following) → preference alignment (helpful, safe, human-like behavior) ✅ RLHF, DPO, ORPO, and GRPO are all different methods for that same third stage — preference alignment ✅ Not every use case needs a frontier-scale LLM — SLMs exist specifically for cost- and latency-sensitive, domain-specific deployments
References
- Attention Is All You Need (Transformer paper) — https://arxiv.org/pdf/1706.03762
- BERT — https://arxiv.org/pdf/1810.04805
- T5 — https://arxiv.org/abs/1910.10683
- BART — https://arxiv.org/pdf/1910.13461
- GPT-3 — https://arxiv.org/pdf/2005.14165
- SLM Survey — https://arxiv.org/pdf/2506.02153v2
- A Comprehensive Survey on the Application of Transformers — https://arxiv.org/pdf/2306.07303
Found this helpful? Share it with your team! Questions? Drop them in the comments below.