Docker for AI applications

Motivation

Your FastAPI AI backend works on your machine. But “works on my machine” is not deployable. When you ship to a cloud server, a different engineer’s laptop, or a CI pipeline, Python version mismatches, missing system libraries, and different environment variables will break things in ways that are hard to diagnose.

Docker solves this by packaging your application, its exact Python version, all its dependencies, and its configuration into a single portable unit called a container. Run the container anywhere — your laptop, a Linux VM in Hetzner, GitHub Actions — and the application behaves identically. Docker is the standard deployment unit for production AI backends, and fluency with it is expected in every AI engineering role.

Containers vs virtual machines

A virtual machine (VM) is a complete simulated computer — it has its own operating system, kernel, and hardware emulation. Starting a VM takes minutes; each VM uses gigabytes of RAM.

A container is much lighter. It shares the host machine’s operating system kernel but runs in an isolated process with its own filesystem, network, and environment variables. Starting a container takes seconds; containers use only the memory your application actually needs.

For an AI backend, you typically run two or three containers (your API, a database, maybe a Redis cache) on a single VM. Each container is isolated — the database cannot accidentally see your API’s environment variables — but they can communicate over a shared network that Docker creates.

The Dockerfile

A Dockerfile is a text file that describes how to build a container image. It is a sequence of instructions that Docker executes in order: start from a base image, copy files, install dependencies, set configuration.

Here is a production-ready Dockerfile for the FastAPI backend from Project 7:

# Use a specific Python version — never "latest" in production
FROM python:3.12-slim

# Prevent Python from writing .pyc files (containers don't need them)
ENV PYTHONDONTWRITEBYTECODE=1
# Ensure stdout/stderr is flushed immediately (important for log collection)
ENV PYTHONUNBUFFERED=1

# Set the working directory inside the container
WORKDIR /app

# Copy dependency definitions first (take advantage of layer caching)
COPY pyproject.toml uv.lock ./

# Install uv and use it to install dependencies
RUN pip install uv && uv sync --frozen --no-dev

# Copy the rest of the application code
COPY . .

# Document which port the application uses (informational — doesn't actually expose it)
EXPOSE 8000

# The command to run when the container starts
CMD ["uv", "run", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Layer caching: why the order matters

Every RUN, COPY, and ADD instruction creates a layer in the image. Docker caches each layer. When you rebuild, it only re-runs instructions where something changed — and everything after them.

This is why you copy pyproject.toml and install dependencies before copying your application code. Your dependencies rarely change; your code changes constantly. With this order, a code change only re-runs the last COPY . . step. Reversed, every code change would reinstall all dependencies — slow.

Building and running a container

# Build the image, tag it "ai-api"
docker build -t ai-api .

# Run it, map port 8000 on your machine to port 8000 in the container
docker run -p 8000:8000 ai-api

# But the container has no .env file — pass env vars at runtime instead:
docker run -p 8000:8000 \
  -e OPENAI_API_KEY=sk-... \
  ai-api

# Run in the background (detached mode)
docker run -d -p 8000:8000 \
  -e OPENAI_API_KEY=sk-... \
  --name my-ai-api \
  ai-api

# See running containers
docker ps

# View logs from a running container
docker logs my-ai-api

# Stop it
docker stop my-ai-api

Never bake secrets into images

Do NOT put your API key in the Dockerfile or copy your .env file into the image. Docker images can be pushed to registries (GitHub Packages, Docker Hub, AWS ECR) where others might see them. Always pass secrets at runtime via -e VARNAME=value or, in production, via a secrets manager.

Add a .dockerignore file alongside your Dockerfile to prevent sensitive files from being accidentally copied into the image:

.env
.env.*
__pycache__/
*.pyc
*.pyo
.git/
.gitignore
tests/
*.md

Docker Compose — running multiple containers together

An AI backend rarely runs alone. It needs a database, maybe a cache, maybe a background worker. Running and connecting these individually with docker run becomes unwieldy. Docker Compose lets you define all your services in a single YAML file and start everything with one command.

Here is a docker-compose.yml for the AI backend plus a PostgreSQL database (which you will use in Lesson 22):

services:
  api:
    build: .                   # build from the Dockerfile in the current directory
    ports:
      - "8000:8000"            # host:container port mapping
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}   # read from host .env file
      - DATABASE_URL=postgresql+asyncpg://postgres:password@db:5432/aidb
    depends_on:
      db:
        condition: service_healthy         # wait for postgres to be ready
    volumes:
      - .:/app                 # mount current directory for hot reload in dev
    command: uvicorn main:app --host 0.0.0.0 --port 8000 --reload

  db:
    image: postgres:16-alpine              # official postgres image, slim variant
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
      POSTGRES_DB: aidb
    ports:
      - "5432:5432"                        # expose for local tools (e.g. psql, pgAdmin)
    volumes:
      - postgres_data:/var/lib/postgresql/data   # persist data between restarts
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  postgres_data:               # named volume — survives container restarts

Start everything:

# Start all services in the background
docker compose up -d

# Watch logs from all services
docker compose logs -f

# Stop everything
docker compose down

# Stop and delete the database volume (fresh start)
docker compose down -v

Service networking in Compose

Compose puts all services on a shared network automatically. The hostname of each service is its name in the YAML file. That is why the DATABASE_URL uses db:5432 — the API container reaches the Postgres container at the hostname db.

From your laptop, you reach the API at http://localhost:8000. The database is accessible from your laptop at localhost:5432 (because of the port mapping). Inside the containers, they reach each other by service name.

Development vs production Compose files

The volume mount (- .:/app) and --reload flag are development-only. In production you do not want live code reloading or the host filesystem mounted. A common pattern is separate files:

docker-compose.yml         # base config shared by both environments
docker-compose.dev.yml     # development overrides (volume mounts, reload)
docker-compose.prod.yml    # production overrides (no mounts, more replicas)

In development:

docker compose -f docker-compose.yml -f docker-compose.dev.yml up

In production:

docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

For now, one file that works for development is fine. You will revisit this in Chapter 6 when you deploy to a real cloud server.

Docker commands you will use daily