Skip to content

ADR-006: Canonical Runtime and API Boundary

  • Decision status: Accepted
  • Implementation status: Partial
  • Applies to: Reference runtime, enterprise/BYOC
  • Date: 2026-06-16
  • Deciders: Grey Newell
  • Tags: api, runtime, fastapi, fastify, dashboard, release-control

Context

RAGhelm currently has two API/runtime surfaces:

  • Python FastAPI in raghelm/api/server.py, which owns the implemented RAG query path, metrics endpoints, retrieval/generation integration, and Python AI ecosystem dependencies.
  • TypeScript Fastify in packages/api, which is currently a lightweight Node boundary with a health endpoint and is useful for TypeScript-facing product integrations.

This decision supports related implementation work without making issue numbers part of the durable architecture contract, including future product-facing integrations and public API shape.

Without a clear boundary, implementation agents may wire release-critical behavior to the wrong surface or duplicate orchestration across Python and TypeScript.

Decision

Use a split runtime boundary as the target architecture:

  • Python owns the AI runtime. This includes ingestion, retrieval, generation, model/provider calls, production evals, scorecard computation, RAGRunManifest creation, and agent/runtime execution where applicable.
  • TypeScript owns the public/product gateway. This includes browser/dashboard-facing APIs, product authentication, stable external API contracts, release-control UX integration, and orchestration of calls to the Python AI runtime.

Initial implementation rule:

  • Until the TypeScript gateway has a stable contract, Python FastAPI remains the canonical implemented behavior for query and release-check semantics.
  • The dashboard may read local/static scorecard artifacts or Python endpoints directly for demo purposes, but production product APIs should converge on the TypeScript gateway.

Target request flow:

Dashboard / CLI / CI / external caller
  -> TypeScript product gateway
  -> Python AI runtime API or worker
  -> retrieval / generation / eval / scorecard / manifest
  -> TypeScript gateway normalizes response for product surfaces
  -> dashboard, CI gate, or release-control API response

Consequences

Positive

  • Keeps AI/RAG implementation close to the strongest Python ecosystem libraries.
  • Gives the product/API surface a TypeScript boundary that is easier to harden for dashboard, web, auth, and external integrations.
  • Avoids forcing TypeScript to own provider SDK, model, and eval complexity prematurely.
  • Creates a clean seam for future async workers, deployment topology, and dashboard APIs.
  • Lets local demos remain simple while documenting the production target.

Negative

  • Introduces a service boundary that must be tested and documented.
  • Requires explicit API contracts between TypeScript and Python.
  • Adds deployment complexity once the gateway is promoted from lightweight package to production service.
  • Can create temporary duplication while the TypeScript gateway catches up to the Python runtime.

Neutral / Trade-offs

  • Python FastAPI remains the source of truth for AI behavior until the gateway contract is implemented.
  • TypeScript should not reimplement retrieval, generation, eval, scorecard, or manifest logic.
  • The gateway can enforce product concerns such as auth, rate limits, request validation, and dashboard-specific aggregation.

Alternatives Considered

  1. Python FastAPI only

Pros: - Lowest implementation complexity. - Keeps all current behavior in one runtime.

Cons: - Weaker product/web boundary. - Less natural fit for dashboard-facing API, auth, and TypeScript product integrations. - Harder to present as a clean multi-surface product architecture.

  1. TypeScript Fastify only

Pros: - Single public API runtime. - Natural fit for web and dashboard integration.

Cons: - Forces TypeScript to own Python-native AI/RAG/eval complexity or shell out to Python informally. - Higher implementation risk and likely slower AI iteration.

  1. Split boundary: Python AI runtime + TypeScript product gateway

Pros: - Separates AI system concerns from product/API concerns. - Supports future deployment, auth, dashboard, and external integration work. - Lets each runtime do what it is best at.

Cons: - Requires contract tests and deployment coordination.

This is the chosen option.

  1. Keep both independent for now

Pros: - Defers the decision. - Avoids immediate migration work.

Cons: - Preserves ambiguity already visible in README/docs/API issues. - Increases risk that evals, dashboard, and query behavior diverge.

Implementation Guidance

Initial:

  • Keep Python FastAPI as canonical for implemented AI behavior.
  • Align docs and quickstarts so they do not imply the TypeScript package owns /query until it does.
  • Let the dashboard use static demo scorecards or Python endpoints while the gateway contract is still forming.

When implementing the TypeScript gateway:

  • define typed request/response contracts for release checks, scorecards, manifests, query runs, and eval runs
  • do not duplicate Python AI/runtime logic in TypeScript
  • add contract tests that verify gateway responses match Python runtime semantics
  • propagate trace IDs, run IDs, scorecard IDs, and manifest IDs across the boundary
  • enforce auth/rate limits/input validation at the gateway when productized

Non-goals:

  • immediate migration of all dashboard calls to the TypeScript gateway
  • rewriting Python AI runtime logic in TypeScript
  • turning the gateway into an agent runtime

References

  • ADR-001: Product Architecture Spine for Private RAGOps Release Control
  • ADR-002: Release Readiness Scorecard and RAGRunManifest as Core Product Objects