Site Builder Workflow and Frontend Integration

Overview

The Site Builder Workflow and Frontend Integration component is responsible for transforming a generated documentation plan into a fully functional, production-ready Fumadocs-based documentation site. It automates the scaffolding of a Next.js/Fumadocs frontend, migrates legacy documentation artifacts, generates navigation and page trees, and integrates advanced features like OpenAPI reference and chatbot support. This component is the bridge between DeepDoc's AI-driven documentation planning and the developer-facing documentation frontend.

Files Covered

Design & Purpose

The site builder is designed to be idempotent, migration-aware, and extensible. It takes a documentation plan (see Pipeline & Generation Engine) and produces a ready-to-deploy Fumadocs site, handling both new and legacy documentation layouts. The builder ensures that all required frontend assets, navigation, and configuration files are present, and that legacy artifacts are migrated or cleaned up.

Workflow Diagram

Key Design Decisions:

• Migration-aware: Handles both new and legacy documentation structures, automatically migrating or cleaning up as needed.
• Composable: Each step is modular, allowing for extension or replacement (e.g., adding new asset generators).
• Frontend-first: Generates all required Next.js/Fumadocs scaffolding, including config, layouts, and assets, so the output is immediately usable.

Implementation Details

1. build_fumadocs_from_plan()

File: deepdoc/site/fumadocs_builder_v2.py:24
Signature:

def build_fumadocs_from_plan(
    repo_root: Path,
    output_dir: Path,
    cfg: dict[str, Any],
    plan: DocPlan,
    has_openapi: bool = False,
) -> None:

Mechanics:

• Orchestrates the entire site build process.
• Migrates legacy Markdown to MDX (_rename_md_to_mdx()), renames intro pages (_rename_legacy_intro_to_index()), ensures frontmatter (_ensure_mdx_frontmatter()), and creates a landing page (_ensure_landing_page()).
• Builds the navigation tree (_build_page_tree_from_plan()).
• Scaffolds the frontend app (_ensure_app_scaffold()), writes the page tree (_write_page_tree()), static assets (_write_static_assets()), and cleans up legacy files (_cleanup_legacy_artifacts()).

Example (from source):

output_dir.mkdir(parents=True, exist_ok=True)
_rename_md_to_mdx(output_dir)
_rename_legacy_intro_to_index(output_dir)
_ensure_mdx_frontmatter(output_dir)
_ensure_landing_page(output_dir, project_name, plan)
...
_ensure_app_scaffold(...)
_write_page_tree(repo_root, page_tree)
_write_static_assets(repo_root)
_cleanup_legacy_artifacts(repo_root)

2. _ensure_app_scaffold()

File: deepdoc/site/fumadocs_builder_v2.py:64
Signature:

def _ensure_app_scaffold(
    repo_root: Path,
    project_name: str,
    repo_url: str,
    docs_dir_relative: str,
    cfg: dict[str, Any],
    has_openapi: bool,
) -> None:

Mechanics:

• Creates or updates all required frontend files in site/.
• Generates package.json, tsconfig.json, Next.js configs, Fumadocs configs, layouts, MDX component registry, global CSS, and feature-specific components (API, chatbot, Mermaid).
• Handles OpenAPI-specific scaffolding if has_openapi is True, and removes stale files if not.

Example (from source):

files = {
    site_dir / "package.json": _package_json(project_name),
    ...
    site_dir / "app" / "ask" / "page.tsx": _chatbot_ask_page_tsx(),
    ...
}
if has_openapi:
    files.update({ ... })
for path, content in files.items():
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(content, encoding="utf-8")

3. _build_page_tree_from_plan()

File: deepdoc/site/fumadocs_builder_v2.py:127
Signature:

def _build_page_tree_from_plan(
    repo_root: Path,
    plan: DocPlan,
    output_dir: Path,
    project_name: str,
    has_openapi: bool,
) -> dict[str, Any]:

Mechanics:

• Walks the DocPlan structure, mapping pages to URLs, handling overview and OpenAPI endpoint references.
• Loads OpenAPI manifest if present, and integrates API endpoints into the navigation tree.
• Returns a dict suitable for Fumadocs' PageTree.

Example (from source):

def is_overview(page) -> bool: ...
def is_endpoint_ref(page) -> bool: ...
def page_exists(page) -> bool: ...
def page_url(page) -> str: ...
...
slug_to_page = {page.slug: page for page in plan.pages if page_exists(page)}
...
return { "type": "root", "children": root_children }

4. _write_page_tree()

File: deepdoc/site/fumadocs_builder_v2.py:338
Signature:

def _write_page_tree(repo_root: Path, page_tree: dict[str, Any]) -> None:

Mechanics:

• Serializes the navigation tree as a TypeScript module (lib/page-tree.generated.ts) for Fumadocs.

Example:

(site_dir / "lib" / "page-tree.generated.ts").write_text(content, encoding="utf-8")

5. _write_static_assets()

File: deepdoc/site/fumadocs_builder_v2.py:353
Signature:

def _write_static_assets(repo_root: Path) -> None:

Mechanics:

• Writes placeholder SVG logos and favicon to site/public/.

6. Migration & Cleanup Helpers

• _rename_legacy_intro_to_index() (deepdoc/site/fumadocs_builder_v2.py:396): Migrates legacy intro pages to index.mdx.
• _rename_md_to_mdx() (deepdoc/site/fumadocs_builder_v2.py:409): Renames .md files to .mdx, stripping legacy frontmatter.
• _strip_docusaurus_frontmatter() (deepdoc/site/fumadocs_builder_v2.py:420): Removes Docusaurus-specific frontmatter fields.
• _ensure_landing_page() (deepdoc/site/fumadocs_builder_v2.py:443): Ensures a landing page exists, with navigation cards.
• _ensure_mdx_frontmatter() (deepdoc/site/fumadocs_builder_v2.py:491): Adds minimal frontmatter to MDX pages if missing.
• _cleanup_legacy_artifacts() (deepdoc/site/fumadocs_builder_v2.py:376): Removes obsolete files and directories.

Public Interface

Main Entry Point

build_fumadocs_from_plan()

• Parameters:
  • repo_root (Path): Root of the repository.
  • output_dir (Path): Directory containing generated docs.
  • cfg (dict[str, Any]): Project configuration.
  • plan (DocPlan): Documentation plan (see Pipeline & Generation Engine).
  • has_openapi (bool): If OpenAPI integration is enabled.
• Returns: None
• Exceptions: Propagates filesystem and serialization errors.

Usage Example:

from deepdoc.site.fumadocs_builder_v2 import build_fumadocs_from_plan

build_fumadocs_from_plan(
    repo_root=Path("/repo"),
    output_dir=Path("/repo/docs"),
    cfg=project_cfg,
    plan=doc_plan,
    has_openapi=True,
)

Internal Mechanics

• Page Tree Construction:
  _build_page_tree_from_plan() uses helper functions to identify overview pages, endpoint references, and page existence. It loads OpenAPI manifests if present, and builds a nested navigation structure compatible with Fumadocs.

• Migration Logic:

  • _rename_md_to_mdx() and _rename_legacy_intro_to_index() ensure that legacy Markdown and intro pages are migrated to the MDX format and correct filenames.
  • _strip_docusaurus_frontmatter() removes fields like slug:, sidebar_position:, and sidebar_label: from frontmatter.

• Frontend Scaffolding:
  _ensure_app_scaffold() generates all required files for a Next.js + Fumadocs app, including:

  • package.json, tsconfig.json, postcss.config.mjs, next.config.mjs, source.config.mjs
  • Layouts and components for docs, API, chatbot, and Mermaid diagrams
  • Global CSS with project-specific colors

• OpenAPI Integration:
  If has_openapi is True, additional components and routes are generated for API reference pages.

• Chatbot Integration:
  Chatbot components and config are always generated, but only enabled if cfg["chatbot"]["enabled"] is True.

Component Integration Diagram

How It's Used

• Upstream:
  The Pipeline & Generation Engine produces a DocPlan and invokes build_fumadocs_from_plan() to generate the site.

• Downstream:
  The generated site is a Next.js/Fumadocs app, ready for deployment or local development.

Call Pattern Example:

# Called after doc plan is generated
build_fumadocs_from_plan(
    repo_root=repo_root,
    output_dir=output_dir,
    cfg=cfg,
    plan=plan,
    has_openapi=has_openapi,
)

Related Pages:

• Pipeline & Generation Engine
• Site Generation & Frontend Integration Overview

Configuration

Settings affecting this component:

Performance Considerations

• File I/O:
  The builder reads and writes many files in the output and site directories. On large documentation sets, this can be I/O bound.

• Idempotency:
  All migration and scaffolding steps are designed to be idempotent—re-running the builder will not duplicate or corrupt files.

• Legacy Cleanup:
  The cleanup routines (_cleanup_legacy_artifacts()) ensure that obsolete files do not accumulate, which could otherwise slow down subsequent builds or cause confusion.

• Hotspots:

  • Large numbers of Markdown/MDX files may slow down _rename_md_to_mdx() and _ensure_mdx_frontmatter().
  • OpenAPI manifest parsing is lightweight but may become a bottleneck if the manifest is extremely large.

See Also