Development Notes

Overview

This page summarizes the core development context for DeepDoc, as captured in the main repository notes and the README.md file (README.md). It provides a high-level orientation to DeepDoc’s features, installation methods, configuration, and workflow, as well as the underlying architecture and design principles. This context is essential for any developer onboarding to the project, as it explains both the "how" and the "why" behind DeepDoc’s structure and operational flow.

---

Key Findings or Terms

1. Bucket-Based Documentation Architecture

• DeepDoc organizes documentation into logical "buckets": system, feature, endpoint, endpoint reference, integration, and database. This avoids the pitfalls of one-file-per-page sprawl and enables more meaningful navigation and maintenance.
• See the DeepDoc Architecture & System Overview for a breakdown of bucket types.

2. Five-Phase Pipeline

• The documentation generation process is split into five distinct phases:
  1. Scan: Parse the repo, detect endpoints, config files, integrations, and OpenAPI specs.
  2. Plan: Use an AI-driven planner to classify and assign files, symbols, and artifacts into buckets.
  3. Generate: Produce documentation pages in batches, with parallel workers.
  4. API Ref: Stage OpenAPI assets for /api/* pages if a spec exists.
  5. Build: Assemble the Fumadocs site, navigation, and static assets.
• The pipeline is designed for incremental updates and efficient handling of large codebases.

3. Multi-Language and Provider Support

• DeepDoc supports Python, JavaScript/TypeScript, Go, PHP/Laravel, and Vue (via tree-sitter AST and regex fallback).
• LLM providers are configurable: Anthropic (default), OpenAI, Azure OpenAI, Ollama, and any LiteLLM-compatible provider.

4. Installation and Verification

• Install from PyPI: pip install deepdoc
• For development: clone the repo and use pip install -e .
• Chatbot features require the [chatbot] extra: pip install "deepdoc[chatbot]"
• Verification commands: deepdoc --version, deepdoc --help, python -m deepdoc --help

5. Typical Workflow

• Initialize: deepdoc init
• Set API key (e.g., export ANTHROPIC_API_KEY=...)
• Generate docs: deepdoc generate
• Preview: deepdoc serve (local Fumadocs site)
• Update incrementally: deepdoc update
• Deploy static site: deepdoc deploy

6. Configuration

• All settings are managed in .deepdoc.yaml at the repo root.
• Key config sections: llm, chatbot, generation_mode, languages, include/exclude, site.
• Chatbot and embedding models are configured independently from the main LLM.

7. Generated Artifacts

• .deepdoc/ stores the plan, scan cache, ledger, and file map.
• docs/ contains generated MDX pages.
• site/ is the Fumadocs Next.js app, with static export in site/out/.

8. Chatbot Integration

• Optional AI-powered chatbot indexes code, config, and docs for RAG-based Q&A.
• Backend is FastAPI, with endpoints for health and query.
• See DeepDoc Chat API Integration and DeepDoc Embedding API Integration for more.

---

Architecture Flow

---

Timeline / References

• Main Reference:
  • README.md — Contains the canonical development notes, feature list, installation instructions, configuration schema, and workflow guidance.
• Configuration Example:
  • .deepdoc.yaml (referenced in README.md)
• Generated Artifacts:
  • .deepdoc/plan.json, .deepdoc/scan_cache.json, .deepdoc/ledger.json, .deepdoc/file_map.json (see "Generated Files" in README.md)
• CI/CD and Release:
  • .github/workflows/github-pages.yml, .github/workflows/docs.yml (see "GitHub Actions CI/CD" in README.md)
• Chatbot Configuration:
  • Chatbot config blocks and explanation in README.md (search "Chatbot Configuration")

---

Relevance to the Codebase

• Architecture & Planning:
  The bucket-based approach and five-phase pipeline described here are implemented in the core planner and generator modules. See DeepDoc Architecture & System Overview and Pipeline & Generation Engine.
• Parsing & Language Support:
  The supported language and framework matrix informs the logic in Parsing & Source Analysis.
• Site Generation:
  The Fumadocs integration and static export process are detailed in Site Generation & Frontend Integration Overview and Site Builder Workflow and Frontend Integration.
• Chatbot:
  The chatbot backend, vector indexing, and retrieval pipeline are implemented in the chatbot modules. See Runtime Services & Chatbot Engine, DeepDoc Chat API Integration, and Documentation Chunking and Summarization.
• CLI Tooling:
  All commands and workflow patterns are surfaced in CLI Commands & Tooling.

---

See Also