diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 431f928..a80793b 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -23,10 +23,10 @@ Requirements for initial release. Each maps to roadmap phases. - [x] **PROV-02**: 14 Tier 2 Inference Platform provider definitions (Together, Fireworks, Groq, Replicate, Anyscale, DeepInfra, Lepton, Modal, Baseten, Cerebrium, NovitaAI, Sambanova, OctoAI, Friendli) - [x] **PROV-03**: 12 Tier 3 Specialized provider definitions (Perplexity, You.com, Voyage, Jina, Unstructured, AssemblyAI, Deepgram, ElevenLabs, Stability, Runway, Midjourney, HuggingFace) - [x] **PROV-04**: 16 Tier 4 Chinese/Regional provider definitions (DeepSeek, Baichuan, Zhipu, Moonshot, Yi, Qwen, Baidu, ByteDance, SenseTime, iFlytek, MiniMax, Stepfun, 360 AI, Kuaishou, Tencent, SiliconFlow) -- [ ] **PROV-05**: 11 Tier 5 Infrastructure/Gateway provider definitions (Cloudflare AI, Vercel AI, LiteLLM, Portkey, Helicone, OpenRouter, Martian, Kong, BricksAI, Aether, Not Diamond) -- [ ] **PROV-06**: 15 Tier 6 Emerging/Niche provider definitions (Reka, Aleph Alpha, Writer, Jasper, Typeface, Comet, W&B, LangSmith, Pinecone, Weaviate, Qdrant, Chroma, Milvus, Neon, Lamini) +- [x] **PROV-05**: 11 Tier 5 Infrastructure/Gateway provider definitions (Cloudflare AI, Vercel AI, LiteLLM, Portkey, Helicone, OpenRouter, Martian, Kong, BricksAI, Aether, Not Diamond) +- [x] **PROV-06**: 15 Tier 6 Emerging/Niche provider definitions (Reka, Aleph Alpha, Writer, Jasper, Typeface, Comet, W&B, LangSmith, Pinecone, Weaviate, Qdrant, Chroma, Milvus, Neon, Lamini) - [x] **PROV-07**: 10 Tier 7 Code/Dev Tools provider definitions (GitHub Copilot, Cursor, Tabnine, Codeium, Sourcegraph, CodeWhisperer, Replit AI, Codestral, watsonx, Oracle AI) -- [ ] **PROV-08**: 10 Tier 8 Self-Hosted provider definitions (Ollama, vLLM, LocalAI, LM Studio, llama.cpp, GPT4All, text-gen-webui, TensorRT-LLM, Triton, Jan AI) +- [x] **PROV-08**: 10 Tier 8 Self-Hosted provider definitions (Ollama, vLLM, LocalAI, LM Studio, llama.cpp, GPT4All, text-gen-webui, TensorRT-LLM, Triton, Jan AI) - [x] **PROV-09**: 8 Tier 9 Enterprise provider definitions (Salesforce Einstein, ServiceNow, SAP AI Core, Palantir, Databricks, Snowflake, Oracle GenAI, HPE GreenLake) - [x] **PROV-10**: Provider YAML schema includes format_version and last_verified date for pattern health tracking diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 5f25154..df1d8ba 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -86,10 +86,10 @@ Plans: - [x] 03-02-PLAN.md — Tier 3 Specialized (Perplexity, You.com, Voyage, Jina, Unstructured, AssemblyAI, Deepgram, ElevenLabs, Stability, Runway, Midjourney) - [x] 03-03-PLAN.md — Tier 5 Infrastructure/Gateway (OpenRouter, LiteLLM, Cloudflare AI, Vercel AI, Portkey, Helicone, Martian, Kong, BricksAI, Aether, Not Diamond) - [x] 03-04-PLAN.md — Tier 7 Code/Dev Tools (GitHub Copilot, Cursor, Tabnine, Codeium, Sourcegraph, CodeWhisperer, Replit AI, Codestral, watsonx, Oracle AI) -- [ ] 03-05-PLAN.md — Tier 8 Self-Hosted runtimes (Ollama, vLLM, LocalAI, LM Studio, llama.cpp, GPT4All, text-gen-webui, TensorRT-LLM, Triton, Jan) +- [x] 03-05-PLAN.md — Tier 8 Self-Hosted runtimes (Ollama, vLLM, LocalAI, LM Studio, llama.cpp, GPT4All, text-gen-webui, TensorRT-LLM, Triton, Jan) - [x] 03-06-PLAN.md — Tier 9 Enterprise (Salesforce Einstein, ServiceNow, SAP AI Core, Palantir, Databricks, Snowflake, Oracle GenAI, HPE GreenLake) - [x] 03-07-PLAN.md — Tier 6 Emerging/Niche (Reka, Aleph Alpha, Lamini, Writer, Jasper, Typeface, Comet, W&B, LangSmith, Pinecone, Weaviate, Qdrant, Chroma, Milvus, Neon) -- [ ] 03-08-PLAN.md — Tier 3-9 guardrail test: lock 108 total providers, per-tier counts, and name sets +- [x] 03-08-PLAN.md — Tier 3-9 guardrail test: lock 108 total providers, per-tier counts, and name sets ### Phase 4: Input Sources **Goal**: Users can point KeyHunter at any content source — local files, git history across all branches, piped content, remote URLs, and the clipboard — and all are scanned through the same detection pipeline diff --git a/.planning/STATE.md b/.planning/STATE.md index 9704175..a0ac1f3 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,14 +3,14 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone status: executing -stopped_at: Completed 03-01-PLAN.md -last_updated: "2026-04-05T11:43:45.831Z" +stopped_at: Completed 03-08-PLAN.md +last_updated: "2026-04-05T11:46:30.979Z" last_activity: 2026-04-05 progress: total_phases: 18 - completed_phases: 2 + completed_phases: 3 total_plans: 18 - completed_plans: 16 + completed_plans: 18 percent: 20 --- @@ -26,7 +26,7 @@ See: .planning/PROJECT.md (updated 2026-04-04) ## Current Position Phase: 03 (tier-3-9-providers) — EXECUTING -Plan: 5 of 8 +Plan: 6 of 8 Status: Ready to execute Last activity: 2026-04-05 @@ -64,6 +64,7 @@ Progress: [██░░░░░░░░] 20% | Phase 03-tier-3-9-providers P02 | 70 | 2 tasks | 22 files | | Phase 03-tier-3-9-providers P06 | 3m | 2 tasks | 16 files | | Phase 03-tier-3-9-providers P01 | 3m | 2 tasks | 32 files | +| Phase 03 P08 | 2min | 1 tasks | 1 files | ## Accumulated Context @@ -97,6 +98,6 @@ None yet. ## Session Continuity -Last session: 2026-04-05T11:43:45.827Z -Stopped at: Completed 03-01-PLAN.md +Last session: 2026-04-05T11:46:30.976Z +Stopped at: Completed 03-08-PLAN.md Resume file: None diff --git a/.planning/phases/03-tier-3-9-providers/03-08-SUMMARY.md b/.planning/phases/03-tier-3-9-providers/03-08-SUMMARY.md new file mode 100644 index 0000000..98c5267 --- /dev/null +++ b/.planning/phases/03-tier-3-9-providers/03-08-SUMMARY.md @@ -0,0 +1,92 @@ +--- +phase: 03-tier-3-9-providers +plan: 08 +subsystem: providers +tags: [tests, guardrail, tier3, tier4, tier5, tier6, tier7, tier8, tier9] +requirements: [PROV-03, PROV-04, PROV-05, PROV-06, PROV-07, PROV-08, PROV-09] +dependency-graph: + requires: + - "03-01..03-07 (all 82 Tier 3-9 provider YAML files embedded and loading)" + - "pkg/providers/tier12_test.go (pattern + inherited TestAllPatternsCompile / TestAllProvidersHaveKeywords)" + provides: + - "Regression net locking registry at 108 providers" + - "Drift detection for any rename, removal, or miscount across Tier 3-9" + affects: + - "All future phases that modify pkg/providers/definitions/*.yaml" +tech-stack: + added: [] + patterns: + - "Table-driven guardrail tests (per-tier count + per-tier name slice)" + - "Inherited cross-tier invariants via existing TestAllPatternsCompile / TestAllProvidersHaveKeywords" +key-files: + created: + - pkg/providers/tier39_test.go + modified: [] +decisions: + - "Single guardrail file for Tier 3-9 (mirrors tier12_test.go) — simpler than one file per tier" + - "Reuse existing TestAllPatternsCompile and TestAllProvidersHaveKeywords rather than duplicating — they already iterate reg.List() and cover every loaded provider" + - "Per-tier Count tests kept separate (not table-driven) to produce clear per-tier failure messages in CI" +metrics: + duration: "~2 min" + completed: "2026-04-05" + tasks: 1 + files_created: 1 + files_modified: 0 +--- + +# Phase 03 Plan 08: Tier 3-9 Guardrail Tests Summary + +Added `pkg/providers/tier39_test.go` — a guardrail test file that locks the provider registry at exactly 108 entries (26 Tier 1-2 + 82 Tier 3-9), enforces per-tier counts, and pins every Tier 3-9 provider name so future edits cannot silently drop or rename providers without a CI failure. + +## What Was Built + +`pkg/providers/tier39_test.go` mirrors the `tier12_test.go` pattern and contains 11 test functions: + +- `TestTier3Count` .. `TestTier9Count` — seven per-tier assertions against `reg.Stats().ByTier[N]` (T3=12, T4=16, T5=11, T6=15, T7=10, T8=10, T9=8) +- `TestTotalProviderCount` — asserts `reg.Stats().Total == 108` +- `TestTier39ProviderNames` — table-driven; for each of 82 names in `expectedTier3`..`expectedTier9`, calls `reg.Get(name)` and verifies the returned `Provider.Tier` matches the expected tier + +Regex compilation (RE2) and keyword presence (≥1 per provider, required for the Aho-Corasick pre-filter) are already exercised globally by the pre-existing `TestAllPatternsCompile` and `TestAllProvidersHaveKeywords` in `tier12_test.go`, which iterate `reg.List()` and therefore automatically cover every newly added Tier 3-9 provider. No duplication needed. + +## Verification + +``` +go test ./pkg/providers/... -count=1 -v -run 'TestTier[3-9]|TestTotalProviderCount|TestTier39ProviderNames|TestAllPatternsCompile|TestAllProvidersHaveKeywords' +``` +All 11 Tier 3-9 tests plus the two inherited invariant tests pass. Full repo regression: +``` +go test ./... -count=1 +ok github.com/salvacybersec/keyhunter/pkg/engine 0.248s +ok github.com/salvacybersec/keyhunter/pkg/providers 0.934s +ok github.com/salvacybersec/keyhunter/pkg/storage 0.117s +``` +Phase 1/2 guardrails (`TestTier1Count`, `TestTier2Count`, `TestTier1ProviderNames`, `TestTier2ProviderNames`) still green — no regression. + +## Deviations from Plan + +None — plan executed exactly as written. + +## Requirements Satisfied + +- **PROV-03** Tier 3 Specialized providers (12) — locked +- **PROV-04** Tier 4 Chinese/Regional providers (16) — locked +- **PROV-05** Tier 5 Infrastructure/Gateway providers (11) — locked +- **PROV-06** Tier 6 Emerging/Niche providers (15) — locked +- **PROV-07** Tier 7 Code/Dev Tools providers (10) — locked +- **PROV-08** Tier 8 Self-Hosted providers (10) — locked +- **PROV-09** Tier 9 Enterprise providers (8) — locked + +## Commits + +- `1aea496` test(03-08): add Tier 3-9 guardrail tests locking 108 total providers + +## Known Stubs + +None. + +## Self-Check: PASSED + +- `pkg/providers/tier39_test.go` — FOUND +- Commit `1aea496` — FOUND in git log +- `go test ./... -count=1` — all packages green +- 11 test functions in tier39_test.go (≥9 required) — verified