Skip to content

ADR-013: External Target Adapters and Vector Backend Boundary

  • Decision status: Accepted
  • Implementation status: Partial (minimal internal VectorBackend interface added for reference runtime experiments)
  • Applies to: External targets, reference runtime, enterprise/BYOC
  • Date: 2026-06-16
  • Deciders: Grey Newell
  • Tags: external-adapters, vector-backend, pinecone, byoc, private-compute, release-control

Context

ADR-001 defines RAGhelm's product spine as private RAGOps and release-control infrastructure, not a general-purpose replacement for every RAG or agent framework. ADR-002 defines scorecards and RAGRunManifests as the core product objects. ADR-007 requires release evidence to flow through a shared QueryPipeline where RAGhelm owns the runtime path.

ADR-007 and this ADR define complementary boundaries:

  • QueryPipeline is canonical for RAGhelm-owned reference runtime behavior.
  • TargetAdapter is canonical for black-box or external systems under test.
  • Both normalize evidence into RAGRunManifest and ReadinessScorecard artifacts.

The architecture has two related but different expansion paths:

  1. External target adapters so RAGhelm can evaluate RAGFlow, LlamaIndex, LangChain, custom APIs, and internal agent systems without replacing them.
  2. Internal VectorBackend abstraction so RAGhelm's reference runtime can run against Pinecone and, later, private/local vector stores such as Qdrant, pgvector, or other BYOC-compatible stores.

The ordering matters. If RAGhelm prioritizes internal vector abstraction too early, it risks becoming a broad RAG platform. If it never abstracts vector backends, the private-compute story remains too Pinecone-centered.

This decision supports related implementation work without making issue numbers part of the durable architecture contract.

Decision

Prioritize external target adapters first, then implement an internal VectorBackend abstraction after the release-control adapter contract is stable.

In shorthand: C into B.

Initial product direction:

  • RAGhelm evaluates external RAG/agent systems through target adapters.
  • Pinecone remains the reference backend for RAGhelm's built-in/reference runtime.
  • External target outputs are normalized into scorecard, manifest, eval, and trace evidence.

Follow-on architecture direction:

  • Add an internal VectorBackend abstraction for the reference runtime after adapter evidence proves what backend-independent fields are actually required.
  • Use that abstraction to support BYOC/private-compute deployments without making vector-store replacement the core product wedge.

Target Architecture

External RAG / agent target
  -> TargetAdapter contract
  -> normalized target run evidence
  -> eval/scoring
  -> RAGRunManifest
  -> ReadinessScorecard
  -> release gate

RAGhelm reference runtime
  -> QueryPipeline
  -> Retriever
  -> VectorBackend abstraction, later
  -> Pinecone first; private/local backends later

External Target Adapter Contract

An external target adapter should support:

  • target identity and version
  • input query/task
  • output answer/result
  • citations/context/source references when available
  • retrieved document/chunk IDs when available
  • trace IDs or external trace links when available
  • latency and cost when available
  • failure mode and error metadata
  • privacy/data classification

The adapter does not need to expose the target's internal vector backend. It only needs to expose enough behavior and evidence for RAGhelm to evaluate release readiness.

VectorBackend Abstraction Scope

When added, the internal VectorBackend abstraction should be limited to RAGhelm's reference runtime and should support:

  • index/collection identity
  • namespace or tenant partitioning where supported
  • upsert/query/delete or invalidation operations required by the reference pipeline
  • metadata filters
  • dimension validation
  • backend mode: local, demo, staging, production
  • backend-specific stats needed for scorecards and manifests

The abstraction should not hide production-critical differences. Backend name, index/collection, dimension, namespace, and mode must remain visible in manifests and traces.

Consequences

Positive

  • Aligns with RAGhelm's release-control spine instead of turning it into another RAG framework.
  • Lets RAGhelm evaluate real customer systems without requiring migration to RAGhelm's reference stack.
  • Keeps Pinecone as a credible reference implementation while preserving future BYOC/private-compute optionality.
  • Reduces risk of designing an overbroad VectorBackend interface before adapter needs are understood.
  • Supports product positioning: RAGhelm gates private RAG/agent releases across stacks.

Negative

  • BYOC vector-store support arrives after external target adapter work.
  • The reference runtime remains Pinecone-centered until a backend abstraction is implemented.
  • External adapters require careful normalization because targets may not expose all desired retrieval/cost/trace metadata.

Neutral / Trade-offs

  • Some external targets will provide only black-box answers and latency; scorecards should show reduced evidence confidence when citations/traces/chunk IDs are unavailable.
  • VectorBackend abstraction remains important, but it is follow-on infrastructure rather than the first product expansion.
  • OpenTelemetry/OpenInference integration may become a key adapter path for traces, but it should not block the initial target adapter contract.

Alternatives Considered

  1. Pinecone-first only

Pros: - Fastest and simplest. - Keeps current implementation focused.

Cons: - Weakens private-compute and external-stack evaluation story. - Makes RAGhelm look more like a Pinecone-specific RAG app.

  1. Internal VectorBackend abstraction first

Pros: - Supports private/local vector stores and BYOC reference runtime. - Useful for teams adopting the RAGhelm reference pipeline.

Cons: - Risks over-investing in RAG runtime portability before the release-control wedge is proven. - Can push the product back toward broad RAG-platform territory.

  1. External target adapters first

Pros: - Best fit with release-control product spine. - Enables RAGhelm to evaluate systems teams already have. - Keeps internal implementation narrower.

Cons: - Does not immediately solve vector backend portability for the reference runtime.

This is the first chosen step.

  1. Both internal backend abstraction and external adapters now

Pros: - Most complete architecture.

Cons: - Too much scope at once. - Increases risk of weak abstractions and unfinished implementation.

Implementation Guidance

Implement in this order:

  1. Define TargetAdapter contract and evidence shape.
  2. Add one simple HTTP/custom API target adapter as the first proof path.
  3. Normalize target outputs into scorecard/manifest evidence.
  4. Represent missing target metadata explicitly as reduced evidence, not as success.
  5. Add contract tests for target adapters.
  6. After adapter contracts stabilize, introduce internal VectorBackend for the reference runtime.
  7. Keep Pinecone as the first production backend and require ADR/update before promoting another backend to production reference status.

Non-goals:

  • replacing Pinecone immediately
  • supporting every vector database before the target adapter path is proven
  • requiring external targets to expose internal vector-store details
  • hiding backend-specific behavior behind a leaky abstraction

References

  • ADR-001: Product Architecture Spine for Private RAGOps Release Control
  • ADR-002: Release Readiness Scorecard and RAGRunManifest as Core Product Objects
  • ADR-007: Shared QueryPipeline as Production-Eval Source of Truth
  • ADR-011: Retrieval Routing, Hybrid Search, and Reranking Policy