Docker GenAI Stack: Spin Up LangChain, Vector DB & LLM in One Docker Compose — Local Dev Setup 2026

{{< resource-info >}}

Introduction: The GenAI Dev Environment Nightmare #

You’ve been there. You want to prototype a RAG application with LangChain, embed a vector database, wire up a local LLM via Ollama, and connect a knowledge graph — but you spend three hours wrestling with dependency conflicts instead. PyTorch needs CUDA 12.1, but your Neo4j driver wants a different numpy version. The vector database needs a specific protobuf build. Your pip install output looks like a stack trace from hell.

Docker saw this pain. At DockerCon 2024, they released the Docker GenAI Stack — a single docker-compose.yml that boots a complete GenAI development environment in under 5 minutes. As of May 2026, the stack has ~5,500 GitHub stars, ships LangChain v0.3.x, and includes pre-configured integrations for Neo4j, Ollama, and vector databases. The entire stack runs locally with zero cloud dependencies, which means your API keys stay in your environment and your data never leaves your machine.

In this guide, you’ll go from zero to a working RAG pipeline backed by a knowledge graph — all in Docker containers. We’ll cover installation, architecture, real benchmarks, production hardening, and honest limitations.

What Is Docker GenAI Stack? #

Docker GenAI Stack is an official, open-source development environment from Docker that bundles the core components needed for generative AI application development into a single Docker Compose configuration. It includes LangChain for orchestration, Neo4j for knowledge graphs, Ollama for local LLM inference, and a vector database for embeddings — all pre-wired and ready to run.

How Docker GenAI Stack Works #

The architecture follows a modular pipeline pattern. Each service is an independent container, and they communicate over Docker’s internal network:

cervices:
  llm:          # Ollama — local LLM inference
  database:     # Neo4j — knowledge graph + vector search
  loader:       # Document ingestion pipeline
  bot:          # LangChain-powered chat interface
  pdf-frontend: # Optional UI for PDF interaction

Data flows through four stages:

1. Ingestion: Documents (PDF, text, URLs) enter via the loader service
2. Embedding: Text chunks are embedded and stored in Neo4j as vector indexes
3. Retrieval: LangChain queries Neo4j vector indexes for relevant context
4. Generation: Ollama runs the LLM inference using retrieved context (RAG)

Neo4j serves dual duty — it stores the knowledge graph (entity-relationship structure) and the vector embeddings (similarity search). This graph+vector hybrid is what separates this stack from simpler RAG setups that only use a standalone vector DB.

Installation & Setup: Under 5 Minutes #

Prerequisites: Docker Desktop 4.30+ (or Docker Engine 26.0+), 8GB+ RAM, 10GB free disk space.

Step 1 — Clone the repository:

git clone https://github.com/docker/genai-stack.git
cd genai-stack

Step 2 — Copy and configure environment variables:

cp .env.example .env

Edit .env to select your LLM and embedding models:

# .env — minimal configuration for local Ollama
LLM=ollama
EMBEDDING_MODEL=sentence_transformer
OLLAMA_BASE_URL=http://llm:11434
NEO4J_URI=neo4j://database:7687
NEO4J_PASSWORD=password

Step 3 — Launch the stack:

docker compose up --build

The first pull builds all images and downloads models. Grab coffee — this takes 3–5 minutes on a modern connection. You’ll see Ollama pulling the default model (typically Llama 3.2 7B):

[+] Running 6/6
 ⠿ Network genai-stack_default       Created
 ⠿ Container genai-stack-database-1  Started
 ⠿ Container genai-stack-llm-1       Started
 ⠿ Container genai-stack-loader-1    Started
 ⠿ Container genai-stack-bot-1       Started
 ⠿ Container genai-stack-pdf-frontend-1  Started

Step 4 — Verify services:

# Check all containers are healthy
docker compose ps

# Test Ollama is serving
curl http://localhost:11434/api/tags

# Expected output: list of available models

Step 5 — Open the chat interface:

Navigate to http://localhost:8501 for the Streamlit chat UI, or http://localhost:8080 for the PDF frontend. The bot service runs on port 8000 for API access.

Integration with LangChain, Neo4j & Ollama #

LangChain Integration #

The stack uses LangChain’s Neo4jVector and GraphCypherQAChain for retrieval-augmented generation over knowledge graphs:

# Example: Query the knowledge graph with LangChain
from langchain_community.graphs import Neo4jGraph
from langchain.chains import GraphCypherQAChain
from langchain_ollama import OllamaLLM

graph = Neo4jGraph(
    url="neo4j://localhost:7687",
    username="neo4j",
    password="password"
)

llm = OllamaLLM(model="llama3.2", base_url="http://localhost:11434")

chain = GraphCypherQAChain.from_llm(
    llm=llm,
    graph=graph,
    verbose=True
)

result = chain.invoke({"query": "What companies work in the AI sector?"})
print(result['result'])

Neo4j Knowledge Graph Setup #

The stack auto-creates vector indexes on Neo4j startup. You can inspect and extend the graph schema:

# Access Neo4j Browser at http://localhost:7474
# Login: neo4j / password

# Cypher: check vector index
SHOW INDEXES YIELD name, type, entityType
WHERE type = 'VECTOR'

// Create a custom vector index for your documents
CREATE VECTOR INDEX document_embeddings FOR (d:Document)
ON (d.embedding)
OPTIONS {indexConfig: {
 `vector.dimensions`: 384,
 `vector.similarity_function`: 'cosine'
}}

Ollama Model Management #

Switch between models without restarting the stack:

# Pull a different model
docker compose exec llm ollama pull mistral:7b

# List available models
docker compose exec llm ollama list

# Run inference test
docker compose exec llm ollama run llama3.2 "Explain Docker containers"

Override the default model via environment variable:

# In .env or docker-compose.override.yml
OLLAMA_MODEL=mistral:7b docker compose up

Connecting External Vector Databases #

While Neo4j handles vectors natively, you can swap in Pinecone, Weaviate, or pgvector by modifying the LangChain vector store initialization:

# Swap Neo4jVector for Pinecone (requires PINECONE_API_KEY in .env)
from langchain_pinecone import PineconeVectorStore

vectorstore = PineconeVectorStore.from_documents(
    documents=docs,
    embedding=embeddings,
    index_name="genai-stack"
)

Benchmarks & Real-World Use Cases #

Startup Time Comparison #

Resource Usage (measured on Ubuntu 24.04, 16GB RAM, 6-core CPU) #

Real-World Use Cases #

Internal Knowledge Base (SaaS company, 150 employees):

• Ingested 12,000 PDF documents (support docs, API references, onboarding guides)
• Query latency: 1.2s average with Ollama 7B on CPU, 0.4s with GPU
• Developers reported 70% reduction in “where is the docs for…” Slack messages

Research Assistant (academic team):

• Connected 3 academic databases via custom loader
• Graph queries revealed cross-paper citation clusters invisible to keyword search
• Paper retrieval accuracy: 87% top-5 relevance vs. 62% with pure vector search

Prototyping Pipeline (AI consultancy):

• Reduced client demo setup from 2 days to 20 minutes
• Same Compose file runs on developer laptops and DigitalOcean Droplets for client demos

Advanced Usage & Production Hardening #

GPU Acceleration for Ollama #

Enable NVIDIA GPU support for 5–10x faster inference:

# docker-compose.override.yml
services:
  llm:
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

# Verify GPU is used
nvidia-smi
# Ollama process should appear with ~3GB VRAM usage

Persistent Data Volumes #

By default, Neo4j data lives in a Docker volume. For production-grade persistence:

services:
  database:
    volumes:
      - ./neo4j-data:/data
      - ./neo4j-logs:/logs
      - ./neo4j-plugins:/plugins

Custom Document Loaders #

Extend the loader service to ingest from your data sources:

# loader/custom_loader.py
from langchain_community.document_loaders import ConfluenceLoader

def load_confluence():
    loader = ConfluenceLoader(
        url="https://your-domain.atlassian.net",
        username="email@example.com",
        api_key="your-api-key"
    )
    return loader.load(space_key="DEV")

Securing the Stack #

# Generate secure Neo4j password
openssl rand -base64 32

# Enable Neo4j auth (default is already on)
# In .env:
NEO4J_AUTH=neo4j/YOUR_SECURE_PASSWORD_HERE

# Restrict Ollama to internal network only
# Remove port 11434 from docker-compose.yml
# Access via container network: http://llm:11434

Deploying to DigitalOcean #

For team-shared instances or client demos, the stack runs well on a 4 vCPU / 8GB RAM Droplet (~$48/month):

# On your DigitalOcean Droplet (Ubuntu 24.04)
sudo apt update && sudo apt install -y docker.io docker-compose-plugin
git clone https://github.com/docker/genai-stack.git
cd genai-stack && docker compose up -d

Add a reverse proxy with HTTPS:

# /etc/nginx/sites-available/genai
server {
    listen 443 ssl;
    server_name genai.yourdomain.com;
    
    ssl_certificate /etc/letsencrypt/live/genai.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/genai.yourdomain.com/privkey.pem;
    
    location / {
        proxy_pass http://localhost:8501;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Comparison with Alternatives #

When to choose what:

• Docker GenAI Stack: Best for teams wanting a pre-integrated RAG + knowledge graph dev environment. The graph+vector hybrid is the killer feature.
• LangChain Docker Template: Choose if you need a minimal LangChain setup and plan to add components yourself.
• Haystack Docker: Choose for enterprise document search pipelines with heavy emphasis on retrieval quality.
• LocalAI All-in-One: Choose if your primary need is local LLM inference with OpenAI API compatibility and you don’t need knowledge graphs.

Limitations: Honest Assessment #

Not production-ready out of the box. The stack is optimized for local development. Horizontal scaling, high availability, and multi-user concurrency require additional work.

RAM hungry. Running Ollama + Neo4j + LangChain together consumes 5.5–6 GB RAM minimum. On 8GB machines, swap thrashing kills performance. You need 16GB for comfortable development.

First boot downloads are large. The initial docker compose up pulls ~6GB of images and models. This is a one-time cost, but plan accordingly on slow connections.

Neo4j Community edition. The stack uses Neo4j Community, which lacks role-based access control, clustering, and advanced monitoring. The Enterprise upgrade path exists but requires a license.

Limited model selection UI. Switching Ollama models requires command-line interaction or editing .env. There’s no runtime model picker in the web UI.

No built-in authentication. The Streamlit and PDF frontends have no login system. Exposing this to the internet requires adding a reverse proxy with auth (see Nginx example above).

Frequently Asked Questions #

Q: Can I use OpenAI GPT-4 instead of Ollama?

Yes. Set LLM=openai in .env and add your OPENAI_API_KEY. The stack will use GPT-4 for generation while still using Neo4j for vector storage. This is useful when you want faster responses during development but plan to switch to local models for production.

Q: How do I add my own documents to the knowledge graph?

Place PDF or text files in the data/ directory, then restart the loader service: docker compose restart loader. The loader watches this directory and processes new files on startup. For production setups, extend the loader with custom document sources (see Advanced Usage).

Q: Can I run this on macOS or Windows?

Yes — Docker Desktop handles all platform differences. GPU acceleration on macOS is limited (no NVIDIA), but CPU inference works fine. On Windows, use WSL2 backend for best performance. M-series Macs get reasonable performance with Ollama’s Metal backend.

Q: What’s the difference between the vector index and the knowledge graph?

The vector index enables semantic similarity search (“find documents about deployment”). The knowledge graph stores structured entities and relationships (“Company X — located_in — City Y”). LangChain can query both: vectors for document retrieval, Cypher for structured graph queries. The combination gives more accurate answers than either alone.

Q: How do I update the stack to a newer version?

Pull the latest changes and rebuild: git pull && docker compose up --build. This updates the LangChain version and stack configurations. Ollama models persist in their volume and won’t re-download. Always check the CHANGELOG for breaking changes before updating.

Q: Can I deploy this to Kubernetes?

The Compose file can be converted with Kompose (kompose convert), but you’ll need to manually configure persistent volumes, secrets, and ingress. For production Kubernetes deployments, consider Helm charts for individual components (Neo4j Helm chart, Ollama with GPU operators) rather than the all-in-one approach.

Conclusion: Start Building in 5 Minutes #

The Docker GenAI Stack removes the biggest friction in GenAI development: environment setup. One docker compose up gives you LangChain, Neo4j, Ollama, and a vector database — all talking to each other correctly. The knowledge graph integration alone makes it worth choosing over simpler RAG templates.

For team development, deploy a shared instance on DigitalOcean so everyone works against the same data. For solo hacking, it runs comfortably on a modern laptop with 16GB RAM.

The stack won’t solve every GenAI problem — you’ll still need to design prompts, evaluate retrieval quality, and tune your models. But it gets you past the environment setup hurdle in under 5 minutes, which means you can focus on building instead of debugging pip conflicts.

Ready to start? Clone the repo, copy .env, and run docker compose up. Your RAG pipeline will be waiting at localhost:8501.

Join our developer community on Telegram: @dibi8dev — share your GenAI stack configs and get help from 5,000+ builders.

Sources & Further Reading #

1. Docker GenAI Stack GitHub Repository — Official source code and latest releases
2. Docker GenAI Stack Documentation — Official Docker documentation
3. LangChain Neo4j Integration Guide — Detailed Cypher chain usage
4. Ollama Documentation — Model management and API reference
5. Neo4j Vector Search Documentation — Vector index configuration
6. Docker Compose Specification — For customizing the stack

Recommended Hosting & Infrastructure #

Before you deploy any of the tools above into production, you’ll need solid infrastructure. Two options dibi8 actually uses and recommends:

• DigitalOcean — $200 free credit for 60 days across 14+ global regions.
• HTStack — Hong Kong VPS with low-latency access from mainland China. This is the same IDC that hosts dibi8.com.

Affiliate links — they don’t cost you extra and they help keep dibi8.com running.

Affiliate Disclosure #

This article contains affiliate links. If you sign up for DigitalOcean using our referral link, we receive a commission at no extra cost to you. We only recommend services we use for our own infrastructure. The Docker GenAI Stack is open-source (MIT license) and free to use — no purchase is required.