docs(03-tier-3-9-providers): create phase plan
This commit is contained in:
salvacybersec
284
.planning/phases/03-tier-3-9-providers/03-08-PLAN.md
Normal file
284
.planning/phases/03-tier-3-9-providers/03-08-PLAN.md
Normal file
@@ -0,0 +1,284 @@
|
||||
---
|
||||
phase: 03-tier-3-9-providers
|
||||
plan: 08
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on: ["03-01", "03-02", "03-03", "03-04", "03-05", "03-06", "03-07"]
|
||||
files_modified:
|
||||
- pkg/providers/tier39_test.go
|
||||
autonomous: true
|
||||
requirements: [PROV-03, PROV-04, PROV-05, PROV-06, PROV-07, PROV-08, PROV-09]
|
||||
must_haves:
|
||||
truths:
|
||||
- "Registry loads 108 total providers (26 Tier 1-2 + 82 Tier 3-9)"
|
||||
- "Per-tier counts enforced: T3=12, T4=16, T5=11, T6=15, T7=10, T8=10, T9=8"
|
||||
- "All 82 Tier 3-9 provider names are known (lock set against drift)"
|
||||
- "All regex patterns compile under RE2"
|
||||
- "Every provider has ≥1 keyword (Aho-Corasick pre-filter requirement)"
|
||||
- "Phase 2 Tier 1-2 guardrails still pass (no regression)"
|
||||
artifacts:
|
||||
- path: "pkg/providers/tier39_test.go"
|
||||
provides: "Tier 3-9 guardrail tests"
|
||||
contains: "TestTier39"
|
||||
key_links:
|
||||
- from: "pkg/providers/tier39_test.go"
|
||||
to: "registry.NewRegistry() Stats() Get() All()"
|
||||
via: "go test ./pkg/providers/..."
|
||||
pattern: "NewRegistry"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Create the Tier 3-9 guardrail test file `pkg/providers/tier39_test.go` that locks in exact provider counts, names, regex compilation, and keyword presence across all 82 new Tier 3-9 providers. Replicates the `tier12_test.go` pattern and serves as the regression net for all future phases.
|
||||
|
||||
Purpose: Satisfy all 7 Phase 3 requirements (PROV-03..PROV-09) with a single enforceable test. Any future change that drops a provider, renames one, or introduces a non-compiling regex will break this test in CI.
|
||||
|
||||
Output: 1 test file with 10+ test functions. Total registry provider count locked at 108.
|
||||
|
||||
Addresses PROV-03, PROV-04, PROV-05, PROV-06, PROV-07, PROV-08, PROV-09.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/phases/03-tier-3-9-providers/03-CONTEXT.md
|
||||
@pkg/providers/tier12_test.go
|
||||
@pkg/providers/schema.go
|
||||
|
||||
<interfaces>
|
||||
Replicate the `tier12_test.go` pattern exactly. Registry API: `NewRegistry()`, `Stats()`, `Get(name)`, `List()`.
|
||||
|
||||
Expected provider name sets (must match plans 03-01..03-07 exactly):
|
||||
- Tier 3 (12): perplexity, you, voyage, jina, unstructured, assemblyai, deepgram, elevenlabs, stability, runway, midjourney, huggingface
|
||||
- Tier 4 (16): deepseek, zhipu, moonshot, qwen, baidu, bytedance, 01ai, minimax, baichuan, stepfun, sensetime, iflytek, tencent, siliconflow, 360ai, kuaishou
|
||||
- Tier 5 (11): openrouter, litellm, cloudflare-ai, vercel-ai, portkey, helicone, martian, kong, bricksai, aether, notdiamond
|
||||
- Tier 6 (15): reka, aleph-alpha, lamini, writer, jasper, typeface, comet, wandb, langsmith, pinecone, weaviate, qdrant, chroma, milvus, neon
|
||||
- Tier 7 (10): github-copilot, cursor, tabnine, codeium, sourcegraph, codewhisperer, replit-ai, codestral, watsonx, oracle-ai
|
||||
- Tier 8 (10): ollama, vllm, localai, lmstudio, llamacpp, gpt4all, text-gen-webui, tensorrt-llm, triton, jan
|
||||
- Tier 9 (8): salesforce-einstein, servicenow, sap-ai-core, palantir, databricks, snowflake, oracle-genai, hpe-greenlake
|
||||
|
||||
Total Tier 3-9: 82. Combined with Tier 1-2 (26): 108 total.
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Create tier39_test.go guardrail test file</name>
|
||||
<files>pkg/providers/tier39_test.go</files>
|
||||
<read_first>
|
||||
- pkg/providers/tier12_test.go (pattern to replicate)
|
||||
- pkg/providers/schema.go (Registry API)
|
||||
</read_first>
|
||||
<action>
|
||||
Create `pkg/providers/tier39_test.go` with the following contents. The expected name slices MUST match the names in each plan's files exactly.
|
||||
|
||||
```go
|
||||
package providers
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// expectedTier3 lists all 12 Tier 3 Specialized provider names (PROV-03).
|
||||
var expectedTier3 = []string{
|
||||
"perplexity", "you", "voyage", "jina", "unstructured", "assemblyai",
|
||||
"deepgram", "elevenlabs", "stability", "runway", "midjourney", "huggingface",
|
||||
}
|
||||
|
||||
// expectedTier4 lists all 16 Tier 4 Chinese/Regional provider names (PROV-04).
|
||||
var expectedTier4 = []string{
|
||||
"deepseek", "zhipu", "moonshot", "qwen", "baidu", "bytedance",
|
||||
"01ai", "minimax", "baichuan", "stepfun", "sensetime", "iflytek",
|
||||
"tencent", "siliconflow", "360ai", "kuaishou",
|
||||
}
|
||||
|
||||
// expectedTier5 lists all 11 Tier 5 Infrastructure/Gateway provider names (PROV-05).
|
||||
var expectedTier5 = []string{
|
||||
"openrouter", "litellm", "cloudflare-ai", "vercel-ai", "portkey",
|
||||
"helicone", "martian", "kong", "bricksai", "aether", "notdiamond",
|
||||
}
|
||||
|
||||
// expectedTier6 lists all 15 Tier 6 Emerging/Niche provider names (PROV-06).
|
||||
var expectedTier6 = []string{
|
||||
"reka", "aleph-alpha", "lamini", "writer", "jasper", "typeface",
|
||||
"comet", "wandb", "langsmith", "pinecone", "weaviate", "qdrant",
|
||||
"chroma", "milvus", "neon",
|
||||
}
|
||||
|
||||
// expectedTier7 lists all 10 Tier 7 Code/Dev Tools provider names (PROV-07).
|
||||
var expectedTier7 = []string{
|
||||
"github-copilot", "cursor", "tabnine", "codeium", "sourcegraph",
|
||||
"codewhisperer", "replit-ai", "codestral", "watsonx", "oracle-ai",
|
||||
}
|
||||
|
||||
// expectedTier8 lists all 10 Tier 8 Self-Hosted provider names (PROV-08).
|
||||
var expectedTier8 = []string{
|
||||
"ollama", "vllm", "localai", "lmstudio", "llamacpp",
|
||||
"gpt4all", "text-gen-webui", "tensorrt-llm", "triton", "jan",
|
||||
}
|
||||
|
||||
// expectedTier9 lists all 8 Tier 9 Enterprise provider names (PROV-09).
|
||||
var expectedTier9 = []string{
|
||||
"salesforce-einstein", "servicenow", "sap-ai-core", "palantir",
|
||||
"databricks", "snowflake", "oracle-genai", "hpe-greenlake",
|
||||
}
|
||||
|
||||
func TestTier3Count(t *testing.T) {
|
||||
reg, err := NewRegistry()
|
||||
if err != nil {
|
||||
t.Fatalf("NewRegistry failed: %v", err)
|
||||
}
|
||||
if got := reg.Stats().ByTier[3]; got != 12 {
|
||||
t.Errorf("Tier 3 count = %d, want 12", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTier4Count(t *testing.T) {
|
||||
reg, err := NewRegistry()
|
||||
if err != nil {
|
||||
t.Fatalf("NewRegistry failed: %v", err)
|
||||
}
|
||||
if got := reg.Stats().ByTier[4]; got != 16 {
|
||||
t.Errorf("Tier 4 count = %d, want 16", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTier5Count(t *testing.T) {
|
||||
reg, err := NewRegistry()
|
||||
if err != nil {
|
||||
t.Fatalf("NewRegistry failed: %v", err)
|
||||
}
|
||||
if got := reg.Stats().ByTier[5]; got != 11 {
|
||||
t.Errorf("Tier 5 count = %d, want 11", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTier6Count(t *testing.T) {
|
||||
reg, err := NewRegistry()
|
||||
if err != nil {
|
||||
t.Fatalf("NewRegistry failed: %v", err)
|
||||
}
|
||||
if got := reg.Stats().ByTier[6]; got != 15 {
|
||||
t.Errorf("Tier 6 count = %d, want 15", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTier7Count(t *testing.T) {
|
||||
reg, err := NewRegistry()
|
||||
if err != nil {
|
||||
t.Fatalf("NewRegistry failed: %v", err)
|
||||
}
|
||||
if got := reg.Stats().ByTier[7]; got != 10 {
|
||||
t.Errorf("Tier 7 count = %d, want 10", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTier8Count(t *testing.T) {
|
||||
reg, err := NewRegistry()
|
||||
if err != nil {
|
||||
t.Fatalf("NewRegistry failed: %v", err)
|
||||
}
|
||||
if got := reg.Stats().ByTier[8]; got != 10 {
|
||||
t.Errorf("Tier 8 count = %d, want 10", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTier9Count(t *testing.T) {
|
||||
reg, err := NewRegistry()
|
||||
if err != nil {
|
||||
t.Fatalf("NewRegistry failed: %v", err)
|
||||
}
|
||||
if got := reg.Stats().ByTier[9]; got != 8 {
|
||||
t.Errorf("Tier 9 count = %d, want 8", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTotalProviderCount(t *testing.T) {
|
||||
reg, err := NewRegistry()
|
||||
if err != nil {
|
||||
t.Fatalf("NewRegistry failed: %v", err)
|
||||
}
|
||||
if got := reg.Stats().Total; got != 108 {
|
||||
t.Errorf("Total provider count = %d, want 108", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTier39ProviderNames(t *testing.T) {
|
||||
reg, err := NewRegistry()
|
||||
if err != nil {
|
||||
t.Fatalf("NewRegistry failed: %v", err)
|
||||
}
|
||||
checks := []struct {
|
||||
tier int
|
||||
names []string
|
||||
}{
|
||||
{3, expectedTier3},
|
||||
{4, expectedTier4},
|
||||
{5, expectedTier5},
|
||||
{6, expectedTier6},
|
||||
{7, expectedTier7},
|
||||
{8, expectedTier8},
|
||||
{9, expectedTier9},
|
||||
}
|
||||
for _, c := range checks {
|
||||
for _, name := range c.names {
|
||||
p, ok := reg.Get(name)
|
||||
if !ok {
|
||||
t.Errorf("Tier %d provider %q not found in registry", c.tier, name)
|
||||
continue
|
||||
}
|
||||
if p.Tier != c.tier {
|
||||
t.Errorf("provider %q tier = %d, want %d", name, p.Tier, c.tier)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note: Do NOT add package-level imports for `regexp` — `TestAllPatternsCompile` and `TestAllProvidersHaveKeywords` already exist in `tier12_test.go` and will exercise every provider (including Tier 3-9) once the registry loads them.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/salva/Documents/apikey && go test ./pkg/providers/... -count=1 -v -run 'TestTier[3-9]|TestTotalProviderCount|TestTier39ProviderNames|TestAllPatternsCompile|TestAllProvidersHaveKeywords' && go test ./... -count=1</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- `pkg/providers/tier39_test.go` exists
|
||||
- `grep -c "^func Test" pkg/providers/tier39_test.go` returns ≥ 9
|
||||
- `go test ./pkg/providers/... -run TestTotalProviderCount -v` passes with Total == 108
|
||||
- `go test ./pkg/providers/... -run TestTier3Count -v` passes
|
||||
- `go test ./pkg/providers/... -run TestTier4Count -v` passes
|
||||
- `go test ./pkg/providers/... -run TestTier5Count -v` passes
|
||||
- `go test ./pkg/providers/... -run TestTier6Count -v` passes
|
||||
- `go test ./pkg/providers/... -run TestTier7Count -v` passes
|
||||
- `go test ./pkg/providers/... -run TestTier8Count -v` passes
|
||||
- `go test ./pkg/providers/... -run TestTier9Count -v` passes
|
||||
- `go test ./pkg/providers/... -run TestTier39ProviderNames -v` passes (all 82 names resolve)
|
||||
- `go test ./pkg/providers/... -run TestAllPatternsCompile -v` passes (inherited from tier12_test.go — all Tier 3-9 regexes compile under RE2)
|
||||
- `go test ./pkg/providers/... -run TestAllProvidersHaveKeywords -v` passes (every provider ≥1 keyword)
|
||||
- Phase 2 guardrails still pass: `go test ./pkg/providers/... -run 'TestTier1|TestTier2'` green
|
||||
- Full repo regression: `go test ./... -count=1` green (engine, storage, providers)
|
||||
</acceptance_criteria>
|
||||
<done>Tier 3-9 guardrail test file exists, 9+ test functions pass, total provider count locked at 108, all 82 Tier 3-9 provider names verified, no regression in Phase 1/2 tests.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
`go test ./... -count=1` passes. `go run . providers stats` shows Total 108, with Tier 1: 12, Tier 2: 14, Tier 3: 12, Tier 4: 16, Tier 5: 11, Tier 6: 15, Tier 7: 10, Tier 8: 10, Tier 9: 8.
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- tier39_test.go created with ≥9 test functions
|
||||
- Total provider count locked at 108
|
||||
- All 82 Tier 3-9 provider names locked against drift
|
||||
- All regex patterns compile under RE2 (via existing TestAllPatternsCompile)
|
||||
- Every provider has ≥1 keyword (via existing TestAllProvidersHaveKeywords)
|
||||
- Full repo `go test ./...` green
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/03-tier-3-9-providers/03-08-SUMMARY.md`
|
||||
</output>
|
||||
Reference in New Issue
Block a user