Skip to content

ADR-015: Persistence Store for Release-Control History

  • Decision status: Accepted
  • Implementation status: Not started
  • Applies to: Reference runtime, enterprise/BYOC
  • Date: 2026-06-16
  • Deciders: Grey Newell
  • Tags: persistence, sqlite, postgres, scorecards, manifests, traces, dashboard

Context

ADR-014 chooses native JSON observability artifacts as the initial source of truth. JSON artifacts are portable, CI-friendly, and easy to inspect, but the full roadmap also needs queryable history for dashboards, comparisons, anomaly detection, review workflows, and production/BYOC deployments.

The question is how RAGhelm should persist release-control history once file-based artifacts are not enough.

This decision supports related implementation work without making issue numbers part of the durable architecture contract.

Decision

Use SQLite for local/demo/open-source deployments and a Postgres-compatible production persistence abstraction for production/BYOC deployments.

Portable JSON artifacts remain the release/proof artifact boundary. The persistence store is for query, history, dashboard, comparisons, anomaly detection, and workflow state.

In practical terms:

  • local/demo/default OSS mode stores history in SQLite
  • production/private-cloud mode stores history in Postgres or a Postgres-compatible database
  • repository/service interfaces hide most storage details from scorecard, manifest, eval, and dashboard logic
  • every persisted record must be exportable back to JSON artifacts where needed for proof bundles and CI/release evidence

Consequences

Positive

  • Keeps local demo and OSS usage simple.
  • Supports production/BYOC deployments with a credible database story.
  • Enables dashboard history, comparisons, anomaly detection, and review workflows.
  • Preserves JSON artifact portability for CI, public proof, and release bundles.
  • Avoids making Postgres mandatory for early local use.

Negative

  • Requires migrations and a persistence abstraction.
  • Requires tests across SQLite and Postgres-compatible behavior for critical queries.
  • Adds lifecycle questions around retention, pruning, and artifact export.

Neutral / Trade-offs

  • JSON artifacts remain the source of portable truth for releases; the database is the queryable operational store.
  • Some production-only features may require Postgres even if the local SQLite path supports the core workflow.
  • The first implementation can persist only scorecards, manifests, eval runs, and trace summaries before adding full trace/event history.

Alternatives Considered

  1. File-based artifacts only

Pros: - Simplest and very CI-friendly. - Good for local demos and proof bundles.

Cons: - Weak for dashboard history, comparisons, anomaly detection, multi-user workflows, and production review state.

  1. SQLite local/demo/open-source store only

Pros: - Still simple. - Enables local history and dashboard queries.

Cons: - Not enough for multi-user private-cloud or hosted production deployments.

  1. Postgres production store only

Pros: - Strong production/team/BYOC story. - Good for audit history and queryable dashboards.

Cons: - Too heavy as the only local/development path.

  1. SQLite local + Postgres-compatible production abstraction

Pros: - Best balance of local simplicity and production credibility. - Supports both OSS demo and enterprise/private-cloud growth.

Cons: - Requires repository interfaces and migration discipline.

This is the chosen option.

Persistence Scope

Initial persisted entities should include:

  • scorecards
  • RAGRunManifests
  • eval runs and summaries
  • target runs
  • trace/event summaries
  • release policies and retrieval policies by ID/version
  • review/override records when ADR-018 is implemented

Each entity should retain links to JSON artifacts and hashes where applicable.

Minimum query patterns:

  • latest scorecard by target/release candidate
  • scorecard history by target
  • manifest by run ID
  • eval run history by suite/mode
  • failed checks by release policy
  • cost/latency/quality trends over time
  • trace/event summaries by run ID

Implementation Guidance

Initial implementation should:

  • define a persistence service/repository interface
  • implement SQLite as the default local backend
  • keep JSON export/import for scorecards and manifests
  • design migrations so Postgres can be added without changing product logic
  • avoid storing secrets, API keys, or raw private retrieved content by default
  • add retention and redaction hooks before production use

Production implementation should:

  • add Postgres backend and migrations
  • support backup/restore guidance
  • validate schema compatibility across SQLite and Postgres for release-critical entities
  • include database mode and artifact export paths in deployment docs

Non-goals:

  • replacing JSON release artifacts with database-only state
  • storing full private source documents in the observability database by default
  • implementing multi-tenant access control before the tenant/security ADRs define it
  • requiring Postgres for local demos

References

  • ADR-002: Release Readiness Scorecard and RAGRunManifest as Core Product Objects
  • ADR-009: RAGRunManifest Schema and Artifact Lifecycle
  • ADR-014: Native JSON Observability and Future OpenTelemetry Interoperability