keyhunter/.planning/phases/02-tier-1-2-providers/02-05-SUMMARY.md

phase, plan, subsystem, tags, requires, provides, affects, tech-stack, key-files, key-decisions, patterns-established, requirements-completed, duration, completed

phase	plan	subsystem	tags	requires	provides	affects	tech-stack	key-files	key-decisions	patterns-established	requirements-completed	duration	completed
02-tier-1-2-providers	05	testing	providers guardrail integration-test re2 aho-corasick	phase provides 02-tier-1-2-providers (plans 01-04) 26 Tier 1+2 provider YAML definitions embedded in registry	Compile-time guardrail locking Tier 1 (12) and Tier 2 (14) provider counts Runtime verification that every provider regex compiles under Go RE2 Runtime verification that every provider has non-empty keywords (AC pre-filter)	03-tier-3-providers all future phases touching pkg/providers	added patterns Name-list guardrail: test enumerates expected provider names so silent deletions fail loudly	created modified pkg/providers/tier12_test.go	Guardrail uses Registry.List() and Registry.Get() public API (no reflection / internal poking) Test scoped to pkg/providers only — unrelated pkg/engine regression pre-exists this plan	Tier-count regression test: lock phase deliverables with explicit count assertions Name-enumeration test: silent provider removal breaks TestTier{1,2}ProviderNames	PROV-01 PROV-02	2min	2026-04-05

Phase 02 Plan 05: Tier 1+2 Provider Guardrail Test Summary

Six-function integration test that locks 12 Tier 1 + 14 Tier 2 provider counts, RE2 regex compilation, and keyword presence against future regressions.

Performance

• Duration: ~2 min
• Started: 2026-04-05T11:14:00Z
• Completed: 2026-04-05T11:16:00Z
• Tasks: 1
• Files modified: 1

Accomplishments

• Added pkg/providers/tier12_test.go with 6 guardrail tests
• All 6 tests pass against current registry state (12 Tier 1 + 14 Tier 2 providers)
• Full pkg/providers/... test suite remains green
• Phase 2 success criteria now machine-verified — any future drop of a provider from the registry fails CI loudly

Task Commits

1. Task 1: tier1/tier2 guardrail test - 58f302b (test)

Note: Plan was a single TDD-style task. Because the target tests assert state already built by plans 02-01 through 02-04, RED was skipped — tests went directly to GREEN and stayed there. No refactor needed.

Files Created/Modified

• pkg/providers/tier12_test.go — 103 lines, 6 test functions asserting Tier 1/2 counts, regex compilation, keyword non-emptiness, and expected provider names

Decisions Made

• Used Registry.List() rather than a hypothetical All()/Each() helper — matches the actual public API
• Used reg.Get(name) for name lookups — already exported
• Kept expected name lists as package-level var so future phases can reuse them if needed

Deviations from Plan

None in scope. Plan executed exactly as written.

Issues Encountered

Pre-existing regression (OUT OF SCOPE — documented per orchestrator instructions)

go test ./pkg/engine/... currently fails with TestScannerPipelineOpenAI, TestScannerPipelineAnthropic, and TestScannerPipelineMultipleKeys. Root cause: several Tier 2 provider regex patterns introduced in Wave 1 (plans 02-03 and 02-04 — ai21, baseten, cerebrium, cohere, deepinfra, fireworks, friendli, inflection, lepton, mistral, novita, octoai, sambanova) are too broad and match the OpenAI/Anthropic fixtures in testdata/samples/, producing 14–15 findings where exactly 1 is expected. TestScannerPipelineMultipleKeys additionally does not even find the anthropic provider by name in the over-matched result set.

Per orchestrator instructions, this regression was NOT fixed in plan 02-05. It pre-existed this plan's creation and requires either:

1. Tightening the over-broad Tier 2 regexes (prefixes, length bounds, character classes), or
2. Updating the engine test expectations to tolerate multi-provider matches on ambiguous keys.

This is a blocker for the phase-level success criterion go test ./... -count=1 passes and should be handled as a follow-up plan or verifier-driven fix in Phase 2 closeout.

The guardrail test added in this plan does not depend on and is not affected by the engine regression — it runs cleanly via go test ./pkg/providers/... -count=1.

User Setup Required

None — pure test code, no external services.

Next Phase Readiness

Ready for phase closeout verification with one blocker:

• Phase 2 guardrail in place (this plan) — DONE
• Pre-existing Tier 2 regex over-match breaking pkg/engine tests — BLOCKER for full go test ./...

Phase 3 (Tier 3 providers) should NOT start until the engine regression is resolved, because adding more providers on top of the current over-broad patterns will compound the problem.

Self-Check: PASSED

• File exists: pkg/providers/tier12_test.go — FOUND
• Commit exists: 58f302b — FOUND
• Test execution: go test ./pkg/providers/... -count=1 — PASSING
• Six test functions present and passing: TestTier1Count, TestTier2Count, TestAllPatternsCompile, TestAllProvidersHaveKeywords, TestTier1ProviderNames, TestTier2ProviderNames

---

Phase: 02-tier-1-2-providers Completed: 2026-04-05