AI Engineer Roadmap All lessons Interactive Quiz GitHub

Step Back Prompting

Advanced · 6 min read · Updated · By Saurabh Pathak

In short: Step-back prompting first asks a more general question about the underlying concept, retrieves for that too, and uses both to answer the specific question.

Play the interactive lesson: Step-back promptingRead on GitHub
On this page
  1. Explain it like I'm new
  2. Watch it
  3. How it works
  4. Code example
  5. When to use it / when not to
  6. Common mistakes
  7. Recap
  8. Interview questions
  9. Go deeper

Level: Advanced · Reading time: 6 min · Phase: 2 — Advanced RAG

Before you start: HyDE and Multi-Query Retrieval. Both use an LLM to rewrite the question before searching, and step-back is a third way to do that.

Explain it like I'm new

A student asks, "What happens to the pressure of a gas if the temperature doubles and the volume halves?" A good teacher first says, "Let's step back: what is the ideal gas law?" Once the principle is on the table, the specific answer is easy.

Specific questions often match no document directly. Your docs might explain how Kubernetes handles container health in general, but never mention your exact symptom. Step-back prompting asks an LLM to produce a step-back question: a broader, more abstract version ("How does Kubernetes manage container health and restarts?"). You retrieve for both the original and the step-back question. The general chunks give the model the principles; the specific chunks give it the details.

The technique comes from a 2023 Google DeepMind paper that showed abstraction-first prompting helped on reasoning-heavy tasks. In RAG, the same idea is used as a retrieval trick.

Words to know:

Watch it

Step-back prompting: Also ask the broader question behind the specific one.
Step-back prompting: Also ask the broader question behind the specific one.

It plays on a loop. Read the caption in the box, one step at a time. Want to go at your own pace? Step through it with Previous and Next.

How it works

  1. Prompt the LLM to write a more general question.
  2. Retrieve top k for the original question.
  3. Retrieve top k for the step-back question.
  4. Merge (concatenate with labels, or fuse with RRF).
  5. Answer the original question using both sets of context.
flowchart TD
    Q["Specific: Why does my pod crash over and over?"] --> L[LLM: step back]
    L --> G["General: How does Kubernetes manage container health?"]
    Q --> R1[Retrieve]
    G --> R2[Retrieve]
    R1 & R2 --> M[Merge] --> A[Answer the specific question]

A prompt that works well includes one or two examples of step-back pairs, such as:

Original: Which team did Messi play for in 2010?
Step-back: What is Messi's career history?

Worked example

Question: "Why does my pod crash over and over?" (A pod is a running container group in Kubernetes. You only need to know it keeps crashing.) The demo has three small documents, d1, d2, d3. Real output from adv_query_transform.py:

  1. Plain search on the original question returns ['d2', 'd3']. d1, the document about liveness probes (Kubernetes health checks that restart a broken container), is missing, because it never mentions "crash".
  2. Step back. The LLM writes: "How does Kubernetes manage container health and restarts?"
  3. Search again with that general question and merge both lists. Result: ['d1', 'd2', 'd3'].
  4. The model now sees d1 (why restarts happen) next to d2 and d3 (your specific symptoms) and answers the original question.

The only change was one extra, broader search. It pulled in the document that explains the principle.

Code example

From adv_query_transform.py:

def step_back_prompt(question):
    return (
        "Write a more general, higher-level question whose answer would help answer the question below.\n"
        f"Question: {question}\nGeneral question:"
    )

def step_back_retrieve(question, llm, corpus=CORPUS, k=3):
    general = llm.complete(step_back_prompt(question)).strip()
    fused = rrf_merge([keyword_search(question, corpus, k), keyword_search(general, corpus, k)])
    return general, fused[:k]

general, top = step_back_retrieve("Why does my pod crash over and over?", FakeLLM())
print(general)  # How does Kubernetes manage container health and restarts?
print(top)      # ['d1', 'd2', 'd3']

The step-back query pulls in d1 (liveness probes restart containers), the general principle behind the crash loop.

When you build the answer prompt, label the context so the model knows which is which:

Background (general principles):
[d1] ...
Specific evidence:
[d2] ...
Question: Why does my pod crash over and over?

When to use it / when not to

TechniqueWhat the LLM writesGoal
Multi-queryParaphrasesCover phrasings
HyDEA fake answerMatch document style
Step-backA broader questionFetch principles

Common mistakes

Recap

If you remember only three things:

Interview questions

1. What is a step-back question? <details><summary>Answer</summary> A more abstract version of the user's question that targets the underlying concept or principle, e.g. "What is the ideal gas law?" for a specific gas-pressure problem. </details>

2. Why retrieve for both questions? <details><summary>Answer</summary> The general question brings in principles and background; the original brings specific facts. The answer usually needs both, and the original keeps the model focused on what was asked. </details>

3. When would step-back hurt? <details><summary>Answer</summary> On simple lookups, or when abstraction drifts off-topic. It adds an LLM call and can fill the context with broad, weakly relevant chunks. </details>

Go deeper

Prev: HyDE · Next: Self Query Retriever · Back to phase: Phase 2