Docker Compose AI Gateway

Self-hosted AI gateway and tooling stack with documented architecture, deployment, and operations workflows.

Docker Compose AI Gateway architecture banner

Content Index

Overview

What This Project Provides

Docker Compose AI Gateway is a locally runnable microservice mesh where an AI classifier routes requests to domain-specific backend services. The gateway serves the UI and API entrypoint, proxies training/refine workflows, and exposes application-level request tracing.

Read the complete project README

Ask the docs

Core Components

  • Gateway serves static UI and the main API endpoint
  • AI Router classifies intent and returns route + confidence
  • Backends (search, image, ops) simulate domain responses
  • Trainer optionally trains and emits model artifacts
  • Refiner optionally relabels/augments datasets (Ollama)
  • Redis + Training API support train/refine job state

Quick Start Commands

# Generate env file for selected environment
python scripts/generate_env.py

# Build and start
docker compose -f compose/docker-compose.yaml up --build -d

# Optional training
docker compose -f compose/docker-compose.yaml --profile train run --rm trainer

# Optional refine actions
docker compose -f compose/docker-compose.yaml --profile refine run --rm training-api relabel
docker compose -f compose/docker-compose.yaml --profile refine run --rm training-api augment
./scripts/promote.sh

Getting Started

Prerequisites

  • Docker and Docker Compose installed
  • Python 3 available for helper scripts
  • Access to required API credentials for selected providers

Optionally use ./scripts/demo.sh run to generate env and start the stack in one command.

Step 1: Configure Project Settings

Update core settings in config/PROJECT_CONFIG.yaml and confirm environment expectations for your host.

Step 2: Generate Environment

Run scripts/generate_env.py to create and synchronize environment files consumed by the compose stack.

Step 3: Start Services

Use compose/docker-compose.yaml to launch the stack, then validate health and connectivity for each enabled service.

Step 4: Optional Dev Overlay

For live code reload, combine compose/docker-compose.yaml and compose/docker-compose.dev.yaml.

Step 5: Verify and Iterate

Use the demo and troubleshooting docs to validate workflows and refine your configuration for local or team usage.

Architecture

System Layout

The stack is organized around a compose-driven runtime, centralized configuration, and service code under services/. The gateway routes user requests to AI router and backend services while training/refine components run behind compose profiles.

Project architecture diagram

Configuration Flow

Project config and generated environment files feed service startup. This keeps configuration explicit and reproducible across machines.

Project Structure

compose/          # Docker Compose definitions
config/           # Authoritative PROJECT_CONFIG.yaml
env/              # Generated .env.<env> files for compose execution
services/         # gateway, ai_router, backends, trainer, refiner, training-api
scripts/          # Demo, env generation, refine, and promote helpers
tests/            # Integration and e2e tests
docs/             # Auxiliary docs, diagrams, and this GitHub Page

Demo Details

Demo Workflow

Use the demo script to run the common lifecycle quickly: generate env, build/start services, run tests, and stop/cleanup. This is the fastest way to validate end-to-end behavior. 

# Start stack using demo helper
./scripts/demo.sh run

# Optional dev mode with reload
./scripts/demo.sh run --dev

# Run test routine
./scripts/demo.sh test

# Stop stack
./scripts/demo.sh down

Full Runbook

For build, run, stop, scaling, and failure simulation guidance, see the detailed demo document: DEMO.md.

Documentation

Core Documentation

Product Requirements

Primary feature and behavior requirements.

Architecture Guide

System components and interactions.

Technical Details

Implementation-oriented technical reference.

Configuration Guide

Centralized config and env generation details.

Demo Runbook

Build/run/stop, scaling, and failure demos.

Acceptance Criteria

Verification checklist for expected behavior.

Troubleshooting

Debug runbook and common failure resolution.

Planning and Refiner Docs

Project Plan

Delivery sequencing and milestones.

Implementation Tasks

Actionable development and integration tasks.

Performance Improvements

Refinement performance tuning guidance.

Augmentation Quality Improvements

Quality gates and promotion tolerance planning.

Refiner Plan

Conceptual overview of refiner architecture.

Refiner Technical

Refiner internals and implementation details.

Refiner Flow

End-to-end flow and handoff states.

Train and Refine GUI Documents

Train and Refine GUI Plan

Scope and phased delivery plan for UI pages.

Train and Refine GUI Technical

Technical design for GUI behavior and integration.

Train and Refine GUI PRD

Requirements for train/refine user-facing features.

Train and Refine GUI Tasks

Execution checklist for GUI implementation tasks.

NotebookLM Slide Deck Instructions

Detailed instructions used for presentation generation.

Auxiliary and Diagram Docs

Architecture Diagram Setup

How to generate and update architecture diagrams.

Architecture Diagram Instructions

Diagram authoring and export instructions.

Architecture Diagram Agent Notes

Guidance used to generate architecture assets.

Metrics JSON Reference

Metrics shape and field semantics reference.

NotebookLM Welcome Note

NotebookLM context and onboarding notes.

Access

Local Service Access

Service endpoints depend on your compose profile and environment. Review compose service mappings and generated environment values for exact ports and hostnames.

Repository

Source, scripts, and configuration are maintained in this repository: github.com/talorlik/docker-compose-ai-gateway.

Security

Security Considerations

  • Do not commit real credentials into environment files
  • Prefer least-privilege API keys and scoped tokens
  • Rotate secrets and verify service exposure in compose configs
  • Use isolated environments for testing and experimentation

Support

Ask about this project

Operations and Troubleshooting

Start with the troubleshooting docs, then inspect compose logs and service health checks to isolate configuration or runtime issues.

Troubleshooting guide: DEBUG.md.

Testing Shortcuts

# Run all tests
pytest

# Service examples
pytest services/gateway/tests/ -v
pytest services/ai_router/tests/ -v
pytest services/training-api/tests/ -v

# Integration and e2e
pytest tests/ -v

Useful Paths

  • compose/docker-compose.yaml
  • config/PROJECT_CONFIG.yaml
  • scripts/generate_env.py
  • docs/auxiliary/

Repository

GitHub Repository: docker-compose-ai-gateway

License

MIT License

This project is released under the MIT License. See the full license text in: LICENSE.