AI Engineer Roadmap All lessons Interactive Quiz GitHub

FastAPI for AI

Beginner · 10 min read · Updated · By Saurabh Pathak

In short: FastAPI is a Python web framework that turns your AI function into an HTTP API with automatic input validation and docs.

Play the interactive lesson: FastAPI for AIRead on GitHub
On this page
  1. Explain it like I'm new
  2. Watch it
  3. How it works
  4. Code example
  5. When to use it / when not to
  6. Common mistakes
  7. Recap
  8. Interview questions
  9. Go deeper

Level: Beginner · Reading time: 10 min · Phase: 7 — Production AI

Before you start: Calling an LLM API. You should know basic Python functions and dictionaries.

Explain it like I'm new

Think of a restaurant. The kitchen (your model or RAG pipeline) cooks the food. The waiter (the API) takes orders in a fixed format, passes them to the kitchen, and brings back the plate. Customers never walk into the kitchen.

An API (application programming interface) is a doorway that lets other programs call your code over the network. HTTP is the protocol they use. A client sends a request to a route such as POST /chat, and your server sends back a response, usually JSON (a plain-text format for data, like {"answer": "hi"}).

FastAPI helps with three things. It uses Pydantic models to check that incoming data has the right shape. It builds interactive docs at /docs. And it supports async code, which lets one server handle many slow network calls (like LLM calls) at the same time.

LLM answers can take many seconds. Streaming sends the answer in small pieces as the model produces them, so the user sees text right away.

Words to know: -

Watch it

FastAPI for AI: Wrap your AI function in a validated HTTP API.
FastAPI for AI: Wrap your AI function in a validated HTTP API.

It plays on a loop. Read the caption in the box, one step at a time. Want to go at your own pace? Step through it with Previous and Next.

How it works

  1. The client sends POST /chat with a JSON body.
  2. FastAPI parses the body into a Pydantic model. Bad input gets a 422 error automatically.
  3. Your function runs: check cache, retrieve context, call the LLM.
  4. FastAPI converts the return value to JSON, or streams chunks.
  5. Middleware and logging record latency and errors.
sequenceDiagram
  participant C as Client
  participant A as FastAPI
  participant L as LLM / RAG
  C->>A: POST /chat {"question": "..."}
  A->>A: validate with Pydantic
  A->>L: call model
  L-->>A: tokens
  A-->>C: JSON or streamed chunks

Worked example

A client sends POST /chat with the body {"question": "What is RAG?"}.

  1. FastAPI checks the body. question is a non-empty string, so it passes.
  2. Your function runs and returns an answer.
  3. FastAPI replies with status 200 and JSON. In the stdlib sketch below the result is {'status': 200, 'answer': '(fake model) You asked: What is RAG?'}.

Now the client sends an empty body {}. There is no question, so validation fails before any model call. The sketch returns {'status': 422, 'error': 'question must be a non-empty string'}. Real FastAPI sends a 422 too, with its own error details. No tokens were spent.

Code example

The core idea, stdlib only: validate input, then call a function. This is what FastAPI automates.

def handle_chat(body: dict) -> dict:
    question = body.get("question")
    if not isinstance(question, str) or not question.strip():
        return {"status": 422, "error": "question must be a non-empty string"}
    answer = f"(fake model) You asked: {question}"
    return {"status": 200, "answer": answer}

print(handle_chat({"question": "What is RAG?"}))
print(handle_chat({}))

In real projects

Install with pip install fastapi uvicorn. Run with uvicorn main:app --reload.

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel

app = FastAPI()


class ChatRequest(BaseModel):
    question: str
    max_tokens: int = 300


class ChatResponse(BaseModel):
    answer: str


@app.get("/health")
def health():
    return {"status": "ok"}


@app.post("/chat", response_model=ChatResponse)
async def chat(req: ChatRequest):
    answer = await call_llm(req.question, req.max_tokens)  # your function
    return ChatResponse(answer=answer)


async def token_stream(question: str):
    async for token in call_llm_stream(question):  # your async generator
        yield token


@app.post("/chat/stream")
async def chat_stream(req: ChatRequest):
    return StreamingResponse(token_stream(req.question), media_type="text/plain")

call_llm and call_llm_stream are placeholders for your provider's client. Keep API keys in environment variables. See the FastAPI docs for streaming, dependencies and testing.

When to use it / when not to

Common mistakes

Recap

If you remember only three things:

Interview questions

1. Why use Pydantic models for an LLM API? <details><summary>Answer</summary>They validate and document input and output. Bad requests fail early with clear errors, before you spend tokens.</details>

2. When does streaming help? <details><summary>Answer</summary>When answers are long. The user sees the first tokens in about the time-to-first-token instead of waiting for the full answer. Total time stays the same.</details>

3. What is wrong with a blocking call inside an async route? <details><summary>Answer</summary>It blocks the event loop, so every other request waits. Use an async client or run the blocking code in a thread.</details>

Go deeper

Prev: Phase 6 — MCP · Next: 02-docker.md · Back to phase: Phase 7