Skip to content

ADR-018: Human Review and Release Override Model

  • Decision status: Accepted
  • Implementation status: Not started
  • Applies to: Reference runtime, enterprise/BYOC
  • Date: 2026-06-16
  • Deciders: Grey Newell
  • Tags: human-review, overrides, waivers, scorecards, audit, release-control

Context

ADR-002 defines ReadinessScorecards as the human/product release decision surface. ADR-008 defines deterministic gate policies and statuses: ship, needs_review, and block. ADR-016 requires high-risk business-advice workflows to default to needs_review unless reviewed.

A release-control system needs a way for humans to approve, reject, or waive scorecard outcomes. However, full workflow queues and role-based approvals are more than the initial product needs. The architecture should support local/demo review notes now while defining a production target that preserves auditability.

This decision supports related implementation work without making issue numbers part of the durable architecture contract.

Decision

Use a staged review model:

  1. Now: support manual review notes in the ReadinessScorecard.
  2. Production target: use identified review/approval records linked to scorecards and RAGRunManifests.
  3. Later: add a full workflow queue with reviewer roles only if product workflow requires it.

Manual review notes are sufficient for local/demo scorecards and early design-partner conversations. Production releases require structured review records that identify who approved or rejected the release and why.

Review and Override Semantics

A human review may:

  • approve a needs_review scorecard when the reviewer accepts the residual risk
  • reject a needs_review scorecard and keep the release blocked from promotion
  • waive specific block checks only when the release policy marks those checks as waivable
  • add remediation notes without changing the release decision

Some block conditions should be non-waivable unless a future ADR changes this:

  • missing production RAGRunManifest for production claim
  • invalid manifest schema
  • missing release-critical artifact hashes
  • local/demo evidence used for production claim
  • unsafe production mode or missing required credentials/config
  • privacy/publication rule violation

Production Review Record Shape

A production review record should include:

schema_version: 1
review_id: <stable-id>
scorecard_id: <scorecard-id>
manifest_id: <manifest-id>
reviewer:
  id: <user-or-service-id>
  display_name: <name>
  role: <role>
decision: approve | reject | waive | comment
reason: <human-readable-rationale>
waived_checks:
  - check_id: <check-id>
    policy_allows_waiver: true
    expiry: <iso8601|null>
created_at: <iso8601>
expires_at: <iso8601|null>
audit:
  source: cli | dashboard | api | ci
  trace_id: <trace-id|null>

Consequences

Positive

  • Keeps local/demo review simple.
  • Gives production releases an auditable path for approvals and waivers.
  • Prevents opaque human overrides from undermining release-control credibility.
  • Lets policy define which checks are waivable and which are non-waivable.
  • Supports future enterprise review workflows without requiring them now.

Negative

  • Manual review notes are not sufficient for mature production governance.
  • Production review records require identity, persistence, and audit support.
  • Full reviewer queues and roles remain future work.

Neutral / Trade-offs

  • needs_review is not a failure; it is an explicit control point.
  • Some block states may be waivable in pilots, but only if the policy says so.
  • Non-waivable blocks preserve trust in manifests, artifacts, privacy, and production claims.

Alternatives Considered

  1. No override; blocked means blocked

Pros: - Strong safety and simple semantics.

Cons: - Too rigid for real release operations and design-partner pilots.

  1. Manual review note in scorecard

Pros: - Simple and demo-friendly. - Captures human rationale quickly.

Cons: - Weak auditability if used for production without identity, expiry, and waived-check structure.

This is chosen for now.

  1. Signed/identified approval record

Pros: - Strong production posture. - Supports reviewer identity, timestamp, decision, reason, waived checks, expiry, and linked scorecard/manifest.

Cons: - Requires persistence, identity model, and API/UI support.

This is the production target.

  1. Full workflow queue with reviewer roles

Pros: - Supports assignment, escalation, role-based approval, and team workflows.

Cons: - Too much before core review records exist.

This is deferred until product workflow requires it.

Implementation Guidance

Initial implementation should:

  • add optional manual review notes to scorecard JSON
  • make review status visible in local/demo dashboard
  • preserve needs_review as a first-class scorecard outcome
  • avoid presenting manual review notes as production-grade approvals

Production implementation should:

  • add review record schema and persistence
  • link review records to scorecard IDs and manifest IDs
  • define waivable vs non-waivable checks in release policy
  • require reviewer identity for approvals/waivers
  • support review expiry for time-bounded waivers
  • include review records in exported proof bundles when appropriate

Non-goals:

  • full reviewer assignment queues in the first implementation
  • anonymous production approvals
  • waiving non-waivable artifact/provenance/privacy failures
  • replacing deterministic gate evaluation with human judgment alone

References

  • ADR-002: Release Readiness Scorecard and RAGRunManifest as Core Product Objects
  • ADR-008: Readiness Gate Policy and Failure Semantics
  • ADR-016: Agent Runtime Scope, Tool Contract, and Side-Effect Safety