docs(02): create phase 2 tier 1-2 provider plans

2026-04-05 14:08:04 +03:00
parent d0f9535852
commit 5079c65161
6 changed files with 1538 additions and 1 deletions
--- a/.planning/phases/02-tier-1-2-providers/02-01-PLAN.md
+++ b/.planning/phases/02-tier-1-2-providers/02-01-PLAN.md
@@ -0,0 +1,341 @@
+---
+phase: 02-tier-1-2-providers
+plan: 01
+type: execute
+wave: 1
+depends_on: []
+files_modified:
+  - providers/openai.yaml
+  - providers/anthropic.yaml
+  - providers/google-ai.yaml
+  - providers/vertex-ai.yaml
+  - providers/aws-bedrock.yaml
+  - providers/xai.yaml
+  - pkg/providers/definitions/openai.yaml
+  - pkg/providers/definitions/anthropic.yaml
+  - pkg/providers/definitions/google-ai.yaml
+  - pkg/providers/definitions/vertex-ai.yaml
+  - pkg/providers/definitions/aws-bedrock.yaml
+  - pkg/providers/definitions/xai.yaml
+autonomous: true
+requirements: [PROV-01]
+must_haves:
+  truths:
+    - "Registry loads 6 high-confidence Tier 1 providers with valid schemas"
+    - "All regex patterns compile under Go RE2"
+    - "Each provider YAML exists in BOTH providers/ and pkg/providers/definitions/"
+  artifacts:
+    - path: "providers/google-ai.yaml"
+      provides: "Google AI (Gemini) AIzaSy pattern"
+      contains: "AIzaSy"
+    - path: "providers/xai.yaml"
+      provides: "xAI Grok xai- pattern"
+      contains: "xai-"
+    - path: "providers/aws-bedrock.yaml"
+      provides: "AWS Bedrock ABSK pattern"
+      contains: "ABSK"
+    - path: "pkg/providers/definitions/google-ai.yaml"
+      provides: "Embedded Google AI definition"
+      contains: "AIzaSy"
+  key_links:
+    - from: "pkg/providers/definitions/*.yaml"
+      to: "pkg/providers/loader.go"
+      via: "go:embed definitions/*.yaml"
+      pattern: "go:embed"
+---
+
+<objective>
+Create or upgrade the 6 high-confidence Tier 1 LLM provider YAML definitions with distinctive key prefixes: OpenAI (upgrade), Anthropic (upgrade), Google AI (Gemini), Google Vertex AI, AWS Bedrock, xAI (Grok).
+
+Purpose: These providers have well-documented, distinctive key prefixes (sk-proj-, sk-ant-api03-, AIzaSy, ABSK, xai-) — they anchor the detection engine with HIGH confidence patterns validated by TruffleHog/gitleaks.
+
+Output: 12 YAML files (6 provider definitions x 2 locations).
+
+Addresses PROV-01 requirement.
+</objective>
+
+<execution_context>
+@$HOME/.claude/get-shit-done/workflows/execute-plan.md
+@$HOME/.claude/get-shit-done/templates/summary.md
+</execution_context>
+
+<context>
+@.planning/PROJECT.md
+@.planning/ROADMAP.md
+@.planning/STATE.md
+@.planning/phases/02-tier-1-2-providers/02-RESEARCH.md
+@pkg/providers/schema.go
+@providers/openai.yaml
+@providers/anthropic.yaml
+
+<interfaces>
+The Provider YAML schema (from pkg/providers/schema.go):
+
+```go
+type Provider struct {
+  FormatVersion int        `yaml:"format_version"`  // must be >= 1
+  Name          string     `yaml:"name"`
+  DisplayName   string     `yaml:"display_name"`
+  Tier          int        `yaml:"tier"`
+  LastVerified  string     `yaml:"last_verified"`   // required, non-empty
+  Keywords      []string   `yaml:"keywords"`
+  Patterns      []Pattern  `yaml:"patterns"`
+  Verify        VerifySpec `yaml:"verify"`
+}
+type Pattern struct {
+  Regex      string  `yaml:"regex"`
+  EntropyMin float64 `yaml:"entropy_min"`
+  Confidence string  `yaml:"confidence"` // "high" | "medium" | "low"
+}
+type VerifySpec struct {
+  Method        string
+  URL           string
+  Headers       map[string]string
+  ValidStatus   []int
+  InvalidStatus []int
+}
+```
+
+Note: schema has NO `category` field. Do not include category in YAML.
+Loader uses `go:embed definitions/*.yaml` — files must be in `pkg/providers/definitions/`.
+</interfaces>
+</context>
+
+<tasks>
+
+<task type="auto">
+  <name>Task 1: Upgrade OpenAI and Anthropic YAMLs</name>
+  <files>providers/openai.yaml, providers/anthropic.yaml, pkg/providers/definitions/openai.yaml, pkg/providers/definitions/anthropic.yaml</files>
+  <read_first>
+    - providers/openai.yaml (current baseline)
+    - providers/anthropic.yaml (current baseline)
+    - pkg/providers/schema.go (validation rules)
+    - .planning/phases/02-tier-1-2-providers/02-RESEARCH.md sections "1. OpenAI", "2. Anthropic"
+  </read_first>
+  <action>
+Overwrite providers/openai.yaml with:
+
+```yaml
+format_version: 1
+name: openai
+display_name: OpenAI
+tier: 1
+last_verified: "2026-04-05"
+keywords:
+  - "sk-proj-"
+  - "sk-svcacct-"
+  - "sk-none-"
+  - "openai"
+  - "t3blbkfj"
+patterns:
+  - regex: 'sk-proj-[A-Za-z0-9_\-]{48,}'
+    entropy_min: 3.5
+    confidence: high
+  - regex: 'sk-svcacct-[A-Za-z0-9_\-]{48,}'
+    entropy_min: 3.5
+    confidence: high
+  - regex: 'sk-[A-Za-z0-9]{20,}T3BlbkFJ[A-Za-z0-9_\-]{20,}'
+    entropy_min: 3.5
+    confidence: high
+verify:
+  method: GET
+  url: https://api.openai.com/v1/models
+  headers:
+    Authorization: "Bearer {KEY}"
+  valid_status: [200]
+  invalid_status: [401, 403]
+```
+
+Overwrite providers/anthropic.yaml with:
+
+```yaml
+format_version: 1
+name: anthropic
+display_name: Anthropic
+tier: 1
+last_verified: "2026-04-05"
+keywords:
+  - "sk-ant-api03-"
+  - "sk-ant-admin01-"
+  - "anthropic"
+patterns:
+  - regex: 'sk-ant-api03-[A-Za-z0-9_\-]{93}AA'
+    entropy_min: 3.5
+    confidence: high
+  - regex: 'sk-ant-admin01-[A-Za-z0-9_\-]{93}AA'
+    entropy_min: 3.5
+    confidence: high
+verify:
+  method: GET
+  url: https://api.anthropic.com/v1/models
+  headers:
+    x-api-key: "{KEY}"
+    anthropic-version: "2023-06-01"
+  valid_status: [200]
+  invalid_status: [401, 403]
+```
+
+Copy both files VERBATIM to pkg/providers/definitions/openai.yaml and pkg/providers/definitions/anthropic.yaml (dual-location sync required — Go embed only reads pkg/providers/definitions/).
+  </action>
+  <verify>
+    <automated>cd /home/salva/Documents/apikey && diff providers/openai.yaml pkg/providers/definitions/openai.yaml && diff providers/anthropic.yaml pkg/providers/definitions/anthropic.yaml && go test ./pkg/providers/... -run TestRegistry -count=1</automated>
+  </verify>
+  <acceptance_criteria>
+    - `grep -q 'sk-svcacct-' providers/openai.yaml` exits 0
+    - `grep -q 'T3BlbkFJ' providers/openai.yaml` exits 0
+    - `grep -q 'AA$\|AA'"'" providers/anthropic.yaml | grep -q api03` (AA suffix present)
+    - `grep -q 'sk-ant-admin01-' providers/anthropic.yaml` exits 0
+    - `diff providers/openai.yaml pkg/providers/definitions/openai.yaml` returns no diff
+    - `diff providers/anthropic.yaml pkg/providers/definitions/anthropic.yaml` returns no diff
+    - `go test ./pkg/providers/... -count=1` passes
+  </acceptance_criteria>
+  <done>OpenAI YAML has 3 patterns (sk-proj-, sk-svcacct-, legacy T3BlbkFJ). Anthropic YAML has 2 patterns (api03, admin01) with AA suffix. Both files dual-located and registry loads cleanly.</done>
+</task>
+
+<task type="auto">
+  <name>Task 2: Create Google AI, Vertex AI, AWS Bedrock, xAI YAMLs</name>
+  <files>providers/google-ai.yaml, providers/vertex-ai.yaml, providers/aws-bedrock.yaml, providers/xai.yaml, pkg/providers/definitions/google-ai.yaml, pkg/providers/definitions/vertex-ai.yaml, pkg/providers/definitions/aws-bedrock.yaml, pkg/providers/definitions/xai.yaml</files>
+  <read_first>
+    - pkg/providers/schema.go (validation rules)
+    - providers/openai.yaml (template style after Task 1)
+    - .planning/phases/02-tier-1-2-providers/02-RESEARCH.md sections "3. Google AI", "4. Google Vertex AI", "5. AWS Bedrock", "8. xAI"
+  </read_first>
+  <action>
+Create providers/google-ai.yaml:
+
+```yaml
+format_version: 1
+name: google-ai
+display_name: Google AI (Gemini)
+tier: 1
+last_verified: "2026-04-05"
+keywords:
+  - "AIzaSy"
+  - "gemini"
+  - "google_api"
+  - "generativelanguage"
+patterns:
+  - regex: 'AIzaSy[A-Za-z0-9_\-]{33}'
+    entropy_min: 3.5
+    confidence: high
+verify:
+  method: GET
+  url: https://generativelanguage.googleapis.com/v1/models?key={KEY}
+  headers: {}
+  valid_status: [200]
+  invalid_status: [400, 401, 403]
+```
+
+Create providers/vertex-ai.yaml:
+
+```yaml
+format_version: 1
+name: vertex-ai
+display_name: Google Vertex AI
+tier: 1
+last_verified: "2026-04-05"
+keywords:
+  - "vertex"
+  - "vertex_ai"
+  - "google_cloud"
+  - "aiplatform.googleapis.com"
+  - "AIzaSy"
+patterns:
+  - regex: 'AIzaSy[A-Za-z0-9_\-]{33}'
+    entropy_min: 3.5
+    confidence: medium
+verify:
+  method: GET
+  url: https://aiplatform.googleapis.com/v1/projects
+  headers:
+    Authorization: "Bearer {KEY}"
+  valid_status: [200]
+  invalid_status: [401, 403]
+```
+
+Create providers/aws-bedrock.yaml:
+
+```yaml
+format_version: 1
+name: aws-bedrock
+display_name: AWS Bedrock
+tier: 1
+last_verified: "2026-04-05"
+keywords:
+  - "ABSK"
+  - "bedrock"
+  - "aws_bedrock"
+  - "bedrock-runtime"
+  - "AKIA"
+patterns:
+  - regex: 'ABSK[A-Za-z0-9+/]{109,269}={0,2}'
+    entropy_min: 4.0
+    confidence: high
+  - regex: 'AKIA[0-9A-Z]{16}'
+    entropy_min: 3.0
+    confidence: medium
+verify:
+  method: GET
+  url: ""
+  headers: {}
+  valid_status: []
+  invalid_status: []
+```
+
+Create providers/xai.yaml:
+
+```yaml
+format_version: 1
+name: xai
+display_name: xAI (Grok)
+tier: 1
+last_verified: "2026-04-05"
+keywords:
+  - "xai-"
+  - "xai"
+  - "grok"
+patterns:
+  - regex: 'xai-[0-9a-zA-Z_]{80}'
+    entropy_min: 3.5
+    confidence: high
+verify:
+  method: GET
+  url: https://api.x.ai/v1/api-key
+  headers:
+    Authorization: "Bearer {KEY}"
+  valid_status: [200]
+  invalid_status: [401, 403]
+```
+
+Copy ALL FOUR files VERBATIM to pkg/providers/definitions/ with same names. The loader uses `go:embed definitions/*.yaml` — files in providers/ are user-visible only.
+  </action>
+  <verify>
+    <automated>cd /home/salva/Documents/apikey && for f in google-ai vertex-ai aws-bedrock xai; do diff providers/$f.yaml pkg/providers/definitions/$f.yaml || exit 1; done && go test ./pkg/providers/... -count=1</automated>
+  </verify>
+  <acceptance_criteria>
+    - All 8 files exist: `test -f providers/google-ai.yaml -a -f providers/vertex-ai.yaml -a -f providers/aws-bedrock.yaml -a -f providers/xai.yaml -a -f pkg/providers/definitions/google-ai.yaml -a -f pkg/providers/definitions/vertex-ai.yaml -a -f pkg/providers/definitions/aws-bedrock.yaml -a -f pkg/providers/definitions/xai.yaml`
+    - `grep -q 'AIzaSy\[A-Za-z0-9_' providers/google-ai.yaml`
+    - `grep -q 'ABSK\[A-Za-z0-9' providers/aws-bedrock.yaml`
+    - `grep -q 'xai-\[0-9a-zA-Z_' providers/xai.yaml`
+    - `diff providers/xai.yaml pkg/providers/definitions/xai.yaml` returns no diff
+    - `go test ./pkg/providers/... -count=1` passes (registry loads all new YAMLs without schema errors)
+  </acceptance_criteria>
+  <done>4 new providers created and dual-located. Registry loads 9 providers total (3 pre-existing + 6 upgraded/new). All regex patterns compile under RE2.</done>
+</task>
+
+</tasks>
+
+<verification>
+Run `go test ./pkg/providers/... -count=1` and confirm registry loads at least 9 providers (existing 3 + 6 from this plan, noting openai/anthropic were already counted). Count provider files in both directories — they must match.
+</verification>
+
+<success_criteria>
+- 6 Tier 1 high-confidence providers defined (2 upgraded, 4 new)
+- Every YAML dual-located
+- All regex patterns are RE2-compatible and compile
+- go test ./pkg/providers/... passes
+</success_criteria>
+
+<output>
+After completion, create `.planning/phases/02-tier-1-2-providers/02-01-SUMMARY.md`
+</output>