---
phase: 05-verification-engine
plan: 01
subsystem: verification-foundation
tags: [schema, storage, migration, verify, foundation]
requirements: [VRFY-02, VRFY-03]
dependency-graph:
  requires:
    - "pkg/providers/schema.go VerifySpec (Phase 2)"
    - "pkg/engine/finding.go Finding (Phase 2)"
    - "pkg/storage/findings.go (Phase 3)"
  provides:
    - "Extended VerifySpec contract for Plans 05-02/03/04"
    - "Finding.Verify* fields for the verifier pipeline"
    - "findings table verify_* columns + migration for existing DBs"
    - "github.com/tidwall/gjson direct dependency"
  affects:
    - "go.mod / go.sum"
    - "pkg/providers"
    - "pkg/engine"
    - "pkg/storage"
tech-stack:
  added:
    - "github.com/tidwall/gjson v1.18.0 (JSON path extraction)"
  patterns:
    - "PRAGMA table_info + conditional ALTER TABLE for idempotent SQLite migrations"
    - "JSON-encoded TEXT column for map[string]string round-trip with NULL for nil"
    - "Effective*() fallback helpers preserving legacy fields alongside canonical ones"
key-files:
  created:
    - "pkg/providers/schema_test.go"
    - "pkg/storage/findings_test.go"
    - ".planning/phases/05-verification-engine/05-01-SUMMARY.md"
  modified:
    - "go.mod"
    - "go.sum"
    - "pkg/providers/schema.go"
    - "pkg/engine/finding.go"
    - "pkg/storage/schema.sql"
    - "pkg/storage/db.go"
    - "pkg/storage/findings.go"
decisions:
  - "Keep legacy ValidStatus/InvalidStatus fields AND add canonical SuccessCodes/FailureCodes; expose via Effective*() helpers so Phase 2-3 YAMLs still load."
  - "Store VerifyMetadata as JSON TEXT (not a relational table) — small, read-together, and keeps the findings table self-contained."
  - "Migrate existing DBs via PRAGMA table_info + ALTER TABLE ADD COLUMN (SQLite 3.35 ADD COLUMN IF NOT EXISTS is not guaranteed across modernc versions)."
  - "Add ExtractMetadata skeleton in schema.go to make gjson a direct dependency now; Plan 05-03 will replace it with the full verifier extraction path."
metrics:
  duration: "3m43s"
  completed: "2026-04-05T12:42:56Z"
  tasks_completed: 2
  files_created: 3
  files_modified: 7
---

# Phase 5 Plan 01: Verification Foundation Summary

Interface-first Wave 0 landing: extended VerifySpec, extended Finding, migrated SQLite schema, and wired gjson so Plans 05-02/03/04 can run in parallel on Wave 1 without touching each other's contracts.

## Tasks

### Task 1 — Extend VerifySpec, Finding, add gjson dependency
**Commits:** `499f5d5` (RED), `30c0e98` (GREEN)

- `VerifySpec` gains `Body`, `SuccessCodes`, `FailureCodes`, `RateLimitCodes`, `MetadataPaths`. `ValidStatus` / `InvalidStatus` remain for backward compat with Phase 2-3 provider YAMLs.
- `EffectiveSuccessCodes()` returns `SuccessCodes` → `ValidStatus` → `[200]`; `EffectiveFailureCodes()` returns `FailureCodes` → `InvalidStatus` → `[401, 403]`; `EffectiveRateLimitCodes()` returns `RateLimitCodes` → `[429]`.
- `ExtractMetadata([]byte) map[string]string` added as a minimal gjson-backed helper so the dep is promoted to direct in `go.mod`. Plan 05-03 will expand it.
- `engine.Finding` gains `Verified`, `VerifyStatus`, `VerifyHTTPCode`, `VerifyMetadata`, `VerifyError`.
- Tests: `TestVerifySpec_NewFieldsParse`, `TestVerifySpec_LegacyFieldsStillWork`, `TestVerifySpec_Defaults`, `TestVerifySpec_CanonicalFieldsPreferred` — all pass.

### Task 2 — Migrate storage schema and persist verify fields
**Commits:** `2654487` (RED), `aec559d` (GREEN)

- `pkg/storage/schema.sql`: findings table now carries `verified INTEGER NOT NULL DEFAULT 0`, `verify_status TEXT NOT NULL DEFAULT ''`, `verify_http_code INTEGER NOT NULL DEFAULT 0`, `verify_metadata_json TEXT`.
- `pkg/storage/db.go`: new `migrateFindingsVerifyColumns` helper called from `Open()` — walks `PRAGMA table_info(findings)`, issues `ALTER TABLE ADD COLUMN` for any missing verify column. Safe to run on fresh DBs (all columns exist, no-op) and on legacy DBs.
- `pkg/storage/findings.go`: `Finding` struct extended; `SaveFinding` serializes `VerifyMetadata` as JSON (NULL when nil) and writes `verified` as 0/1 INTEGER; `ListFindings` round-trips the same.
- Tests: `TestSaveFinding_VerifyFields_RoundTrip`, `TestSaveFinding_VerifyFields_Empty`, `TestOpen_MigratesExistingDB` — all pass.

## Verification

```
go build ./...                            # clean
go test ./pkg/providers/... -run VerifySpec -v  # 4/4 pass
go test ./pkg/storage/... -v              # 10/10 pass
go test ./...                              # all green
grep -q 'tidwall/gjson' go.mod            # direct dep
grep -rn 'SuccessCodes|VerifyStatus|verify_metadata_json' pkg/  # extensions landed
```

## Deviations from Plan

**None material.** Two small additions made inline:

1. **Added `ExtractMetadata` helper on `VerifySpec`** (Rule 3 — blocking issue). The plan required gjson to be in `go.mod` as a usable dependency, but `go get` without an actual import in code leaves it marked `// indirect`. Rather than run `go mod tidy` (explicitly forbidden in the plan), a tiny `ExtractMetadata` method was added that uses `gjson.GetBytes`. This promotes gjson to a direct require and gives Plan 05-03 a real starting point. The helper is intentionally minimal; Plan 05-03 owns the final extraction logic.

2. **Added `TestVerifySpec_CanonicalFieldsPreferred`** (belt-and-braces test). Not in the plan's three required tests, but verifies that when both canonical and legacy fields are set, canonical wins — a contract every downstream plan depends on.

## Downstream Contracts Ready

| Consumer   | Contract                                                                                      |
| ---------- | --------------------------------------------------------------------------------------------- |
| Plan 05-02 | `VerifySpec.Effective*Codes()`, `Finding.Verify*` fields for HTTPVerifier implementation      |
| Plan 05-03 | `VerifySpec.MetadataPaths`, `ExtractMetadata` skeleton, gjson available                       |
| Plan 05-04 | `storage.Finding.Verified/VerifyStatus/VerifyMetadata` round-trip, migration is automatic     |
| Plan 05-05 | YAML schema ready for Tier 1 provider verify-block enrichment                                 |

## Known Stubs

- `VerifySpec.ExtractMetadata` is a minimal gjson extraction — only handles flat string values via `GetBytes`. **Plan 05-03 will replace this** with full metadata extraction (nested paths, type coercion, Content-Type gating).

## Self-Check: PASSED

Files verified:
- `pkg/providers/schema.go` — FOUND (SuccessCodes, MetadataPaths, ExtractMetadata, Effective* present)
- `pkg/providers/schema_test.go` — FOUND (4 tests)
- `pkg/engine/finding.go` — FOUND (Verified/VerifyStatus/VerifyHTTPCode/VerifyMetadata/VerifyError present)
- `pkg/storage/schema.sql` — FOUND (verify_metadata_json column present)
- `pkg/storage/db.go` — FOUND (migrateFindingsVerifyColumns present)
- `pkg/storage/findings.go` — FOUND (verify_* columns in SELECT/INSERT)
- `pkg/storage/findings_test.go` — FOUND (3 tests)
- `go.mod` — FOUND (`github.com/tidwall/gjson v1.18.0` in direct require block)

Commits verified in `git log`:
- `499f5d5` — test(05-01) VerifySpec RED
- `30c0e98` — feat(05-01) VerifySpec GREEN
- `2654487` — test(05-01) storage RED
- `aec559d` — feat(05-01) storage GREEN

Build + tests: `go build ./...` clean, `go test ./...` all pass.