AI Engineer Roadmap All lessons Interactive Quiz GitHub

DSPy

Advanced · 11 min read · Updated · By Saurabh Pathak

In short: DSPy lets you declare what an LLM step should do (inputs and outputs) as code, then uses an optimizer to find good prompts instead of you hand-writing them.

Play the interactive lesson: DSPyRead on GitHub
On this page
  1. Explain it like I'm new
  2. Watch it
  3. How it works
  4. Code example
  5. When to use it / when not to
  6. Common mistakes
  7. Recap
  8. Interview questions
  9. Go deeper

Level: Advanced · Reading time: 11 min · Phase: 5 — Frameworks

Before you start: You should know what a prompt is and why wording matters (Prompt engineering). It also helps to know that we measure quality with a score (Agent evaluation).

Explain it like I'm new

Hand-tuned prompts are like hand-tuned engine settings: they work for one car on one day. Change the model and they break. DSPy treats prompts like model weights: you give examples and a score, and it searches for better instructions or few-shot examples.

You write a signature, a typed contract such as "question -> answer". You wrap it in a module. An optimizer then tunes the prompt against a metric you define.

Words to know:

Watch it

DSPy: Declare what a step does, and let an optimizer write the prompt.
DSPy: Declare what a step does, and let an optimizer write the prompt.

It plays on a loop. Read the caption in the box, one step at a time. Want to go at your own pace? Step through it with Previous and Next.

How it works

  1. Write a signature and a module.
  2. Prepare a small trainset and a metric.
  3. Run an optimizer. It tries candidate prompts or demos and scores each on the trainset.
  4. Keep the best-scoring program and save it.
flowchart LR
    S[signature + module] --> O[optimizer]
    D[trainset] --> O
    M[metric] --> O
    O --> B[best prompt/demos] --> P[compiled program]

Mapping: the optimizer loop is the evaluation habit from Phase 3 and Phase 7 made automatic: measure, change, measure again. It can tune the generation step of a RAG pipeline from Phase 1.

Worked example

The code example below is a tiny optimizer. It tries two instructions and keeps the one that scores best. The fake model obeys a keyword: if the instruction says "brief" it returns just the last word, otherwise it adds "The answer is". Here is the work by hand (it is small enough to check, and I also ran it).

Trainset: ("capital of France Paris", "Paris") and ("largest planet Jupiter", "Jupiter"). Metric: the output must equal the expected answer exactly.

InstructionOutput for question 1Output for question 2Score
Be verbose.The answer is Paris (no match)The answer is Jupiter (no match)0 of 2
Be brief.Paris (match)Jupiter (match)2 of 2

The optimizer keeps Be brief. The metric defined "good", so the optimizer pushed toward it. If the metric had accepted any answer containing the right word, Be verbose. would also have scored 2 and the choice would have been a tie. Check the current docs for exact DSPy classes.

Code example

From scratch, stdlib only: pick the best instruction by score on a tiny trainset. The "model" is a fake that obeys keywords.

def fake_model(instruction, question):
    if "brief" in instruction:
        return question.split()[-1]       # short answer
    return "The answer is " + question.split()[-1]

trainset = [("capital of France Paris", "Paris"), ("largest planet Jupiter", "Jupiter")]
metric = lambda pred, gold: pred == gold

candidates = ["Be verbose.", "Be brief."]

def score(instruction):
    return sum(metric(fake_model(instruction, q), a) for q, a in trainset)

print(max(candidates, key=score))   # Be brief.

In real projects (pip install dspy, needs a configured LM):

import dspy

dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))
qa = dspy.Predict("question -> answer")
print(qa(question="What is the capital of France?").answer)

Optimizers and typed class-based signatures are covered in the docs.

When to use it / when not to

Common mistakes

Recap

If you remember only three things:

Interview questions

  1. What is a DSPy signature? <details><summary>Answer</summary>A declaration of an LLM step's input and output fields, such as "question -> answer", separate from the prompt text.</details>
  2. How is DSPy different from LangChain? <details><summary>Answer</summary>LangChain composes components; you write the prompts. DSPy generates and optimizes prompts from a signature, examples and a metric.</details>
  3. Why does the metric matter most? <details><summary>Answer</summary>The optimizer maximizes it. A metric that does not match real quality produces prompts that score well and work badly.</details>

Go deeper

Prev: AutoGen · Next: Haystack · Back to phase: Phase 5