Skip to content

ADR-004: Embedding Model Strategy

  • Decision status: Accepted
  • Implementation status: Partial (native dimension checks now enforced on ingestion upsert and retrieval query embeddings)
  • Applies to: Reference runtime
  • Date: 2026-06-13
  • Accepted Date: 2026-06-16
  • Deciders: Grey Newell
  • Tags: embeddings, rag-pipeline, multi-provider, architecture

Context

RAGhelm must support multiple embedding providers while maintaining retrieval quality, operational reliability, and auditable release-control evidence.

Different embedding models produce vectors of different dimensions. Pinecone requires a fixed dimension per index. Projection between dimensions would simplify topology but can distort semantics and make retrieval failures harder to debug.

ADR-012 extends this decision by defining the model provider abstraction boundary for embeddings, generation, and judges. This ADR remains the source of truth for embedding dimensions and index topology.

Decision

Support native embedding dimensions rather than projecting vectors into a single dimension.

RAGhelm will maintain first-class support for:

  • 1536-dimensional hosted embeddings, primarily OpenAI text-embedding-3-small
  • 1024-dimensional local/open embeddings, primarily BAAI/bge-large-en-v1.5 and compatible providers/models

This implies separate dimension-specific vector indexes, currently:

  • raghelm-1536
  • raghelm-1024

Do not mix incompatible embedding dimensions or embedding models in a namespace unless a future ADR explicitly defines the migration and evaluation strategy.

Embedding provider, model name, dimension, and index/namespace selection must be recorded in RAGRunManifest and trace artifacts when they affect release-control evidence.

Considered Options

Option 1: Projection to 1536 Dimensions

Project all non-1536 vectors, such as 1024-dimensional BGE vectors, up to 1536 so everything can live in one index.

Pros:

  • Single index
  • Simpler namespace management
  • Easier query routing logic

Cons:

  • Information loss and semantic distortion
  • Quality degradation on strong open-source models
  • Added compute and maintenance burden for projection matrices
  • Harder to debug retrieval issues
  • Creates technical debt when swapping embedding models

Rejected — quality and maintainability costs outweigh the simplicity benefit.

Option 2: Support Native Dimensions

Support both 1536 and 1024 dimensions as first-class options with separate indexes.

Pros:

  • Preserves native vector quality for hosted and local/open models
  • Clean separation of concerns
  • Easier to add new dimensions in the future, such as 768 or 2048, through explicit architecture review
  • Better observability and debugging
  • Aligns with release-control provenance because dimension/model/index choices remain explicit

Cons:

  • Multiple indexes to manage
  • Routing must choose provider, model, dimension, and index coherently
  • Eval comparisons must account for dimension/index differences

Accepted.

Supported Dimensions & Models

Dimension Primary Use Case Recommended Models Local Fallback Model
1536 Hosted/default production embeddings OpenAI text-embedding-3-small Not supported; projection rejected
1024 Local/open/private-compute embeddings BGE-large, E5-large, compatible Cohere/Voyage-style paths where applicable BAAI/bge-large-en-v1.5

Consequences

  • RAGhelm maintains dimension-specific vector indexes.
  • The indexer and retrieval pipeline must be dimension-aware.
  • Routing must select provider, model, dimension, index, and namespace coherently.
  • Golden/eval results should be reported by embedding model and dimension when relevant.
  • RAGRunManifest must record embedding provider, model, dimension, index, and namespace.
  • Projection remains rejected unless a future ADR provides evidence, migration plan, and rollback strategy.

Implementation Guidance

  • Keep embedding dimension in namespace/index metadata and release artifacts.
  • Validate that vectors match index dimensions before upsert/query.
  • Do not silently fall back to a different embedding model or dimension in production mode.
  • If a provider is unavailable, fail closed for production release evidence rather than producing misleading success.
  • Add eval slices comparing embedding changes against the same golden dataset before promoting a new model/dimension.

References

  • ADR-005: Pinecone Reference Backend Namespace Strategy
  • ADR-012: Model Provider Abstraction Boundary