Files
keyhunter/.planning/phases/03-tier-3-9-providers/03-02-PLAN.md
2026-04-05 14:39:54 +03:00

419 lines
11 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
phase: 03-tier-3-9-providers
plan: 02
type: execute
wave: 1
depends_on: []
files_modified:
- providers/perplexity.yaml
- providers/you.yaml
- providers/voyage.yaml
- providers/jina.yaml
- providers/unstructured.yaml
- providers/assemblyai.yaml
- providers/deepgram.yaml
- providers/elevenlabs.yaml
- providers/stability.yaml
- providers/runway.yaml
- providers/midjourney.yaml
- pkg/providers/definitions/perplexity.yaml
- pkg/providers/definitions/you.yaml
- pkg/providers/definitions/voyage.yaml
- pkg/providers/definitions/jina.yaml
- pkg/providers/definitions/unstructured.yaml
- pkg/providers/definitions/assemblyai.yaml
- pkg/providers/definitions/deepgram.yaml
- pkg/providers/definitions/elevenlabs.yaml
- pkg/providers/definitions/stability.yaml
- pkg/providers/definitions/runway.yaml
- pkg/providers/definitions/midjourney.yaml
autonomous: true
requirements: [PROV-03]
must_haves:
truths:
- "11 new Tier 3 Specialized provider YAMLs load (huggingface already exists → 12 total)"
- "Providers with documented prefixes (pplx-, xai-voyage, sk-, jina_) use tight regex"
- "Voice/image providers without documented prefixes use keyword-only"
artifacts:
- path: "providers/perplexity.yaml"
provides: "Perplexity pplx- prefix pattern"
contains: "pplx-"
- path: "providers/elevenlabs.yaml"
provides: "ElevenLabs API key keyword detection"
contains: "elevenlabs"
- path: "providers/assemblyai.yaml"
provides: "AssemblyAI keyword detection"
contains: "assemblyai"
key_links:
- from: "provider keywords[]"
to: "Registry Aho-Corasick automaton"
via: "NewRegistry()"
pattern: "keywords"
---
<objective>
Create 11 Tier 3 Specialized LLM/AI provider YAMLs — search (Perplexity, You.com), embeddings (Voyage, Jina, Unstructured), voice (AssemblyAI, Deepgram, ElevenLabs), and image/video (Stability, Runway, Midjourney). Huggingface is pre-existing from Phase 2 and must NOT be recreated. Total Tier 3 = 12 after this plan.
Purpose: Satisfy PROV-03 (12 Tier 3 Specialized providers).
Output: 22 YAML files (11 providers × 2 locations).
Addresses PROV-03.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/ROADMAP.md
@.planning/phases/03-tier-3-9-providers/03-CONTEXT.md
@pkg/providers/schema.go
@providers/huggingface.yaml
@providers/cohere.yaml
<interfaces>
Schema: pkg/providers/schema.go. No `category` field. Confidence: high|medium|low.
Patterns field may be omitted for keyword-only providers. Keywords ≥3 required.
Dual-location required (providers/ + pkg/providers/definitions/).
RE2 regex only.
IMPORTANT: providers/huggingface.yaml ALREADY EXISTS (tier: 3). Do NOT modify or recreate it.
</interfaces>
</context>
<tasks>
<task type="auto">
<name>Task 1: Search + embeddings providers (Perplexity, You.com, Voyage, Jina, Unstructured, AssemblyAI)</name>
<files>providers/perplexity.yaml, providers/you.yaml, providers/voyage.yaml, providers/jina.yaml, providers/unstructured.yaml, providers/assemblyai.yaml, pkg/providers/definitions/perplexity.yaml, pkg/providers/definitions/you.yaml, pkg/providers/definitions/voyage.yaml, pkg/providers/definitions/jina.yaml, pkg/providers/definitions/unstructured.yaml, pkg/providers/definitions/assemblyai.yaml</files>
<read_first>
- pkg/providers/schema.go
- providers/huggingface.yaml (existing tier 3 reference — DO NOT MODIFY)
</read_first>
<action>
Create each file in providers/ AND copy verbatim to pkg/providers/definitions/.
providers/perplexity.yaml:
```yaml
format_version: 1
name: perplexity
display_name: Perplexity AI
tier: 3
last_verified: "2026-04-05"
keywords:
- "perplexity"
- "PERPLEXITY_API_KEY"
- "pplx-"
- "api.perplexity.ai"
patterns:
- regex: 'pplx-[A-Za-z0-9]{48,}'
entropy_min: 4.0
confidence: high
verify:
method: POST
url: https://api.perplexity.ai/chat/completions
headers:
Authorization: "Bearer {KEY}"
valid_status: [200, 400]
invalid_status: [401, 403]
```
providers/you.yaml:
```yaml
format_version: 1
name: you
display_name: You.com
tier: 3
last_verified: "2026-04-05"
keywords:
- "you.com"
- "YDC_API_KEY"
- "YOU_API_KEY"
- "api.ydc-index.io"
verify:
method: GET
url: https://api.ydc-index.io/search
headers:
X-API-Key: "{KEY}"
valid_status: [200, 400]
invalid_status: [401, 403]
```
providers/voyage.yaml:
```yaml
format_version: 1
name: voyage
display_name: Voyage AI
tier: 3
last_verified: "2026-04-05"
keywords:
- "voyage"
- "VOYAGE_API_KEY"
- "voyageai"
- "api.voyageai.com"
patterns:
- regex: 'pa-[A-Za-z0-9_\-]{40,}'
entropy_min: 4.0
confidence: medium
verify:
method: POST
url: https://api.voyageai.com/v1/embeddings
headers:
Authorization: "Bearer {KEY}"
valid_status: [200, 400]
invalid_status: [401, 403]
```
providers/jina.yaml:
```yaml
format_version: 1
name: jina
display_name: Jina AI
tier: 3
last_verified: "2026-04-05"
keywords:
- "jina"
- "JINA_API_KEY"
- "api.jina.ai"
- "jinaai"
patterns:
- regex: 'jina_[A-Za-z0-9]{40,}'
entropy_min: 4.0
confidence: high
verify:
method: GET
url: https://api.jina.ai/v1/models
headers:
Authorization: "Bearer {KEY}"
valid_status: [200]
invalid_status: [401, 403]
```
providers/unstructured.yaml:
```yaml
format_version: 1
name: unstructured
display_name: Unstructured.io
tier: 3
last_verified: "2026-04-05"
keywords:
- "unstructured"
- "UNSTRUCTURED_API_KEY"
- "api.unstructured.io"
- "unstructuredio"
verify:
method: GET
url: https://api.unstructured.io/general/v0/general
headers:
unstructured-api-key: "{KEY}"
valid_status: [200, 400]
invalid_status: [401, 403]
```
providers/assemblyai.yaml:
```yaml
format_version: 1
name: assemblyai
display_name: AssemblyAI
tier: 3
last_verified: "2026-04-05"
keywords:
- "assemblyai"
- "ASSEMBLYAI_API_KEY"
- "api.assemblyai.com"
- "assembly.ai"
patterns:
- regex: '[a-f0-9]{32}'
entropy_min: 4.0
confidence: low
verify:
method: GET
url: https://api.assemblyai.com/v2/transcript
headers:
authorization: "{KEY}"
valid_status: [200]
invalid_status: [401, 403]
```
Copy all 6 files verbatim to pkg/providers/definitions/.
</action>
<verify>
<automated>cd /home/salva/Documents/apikey && for f in perplexity you voyage jina unstructured assemblyai; do diff providers/$f.yaml pkg/providers/definitions/$f.yaml || exit 1; done && go test ./pkg/providers/... -count=1 && go test ./pkg/engine/... -count=1</automated>
</verify>
<acceptance_criteria>
- All 12 files exist (6 providers × 2 locations)
- `grep -q 'pplx-' providers/perplexity.yaml`
- `grep -q 'jina_' providers/jina.yaml`
- `grep -q 'api.voyageai.com' providers/voyage.yaml`
- `diff providers/perplexity.yaml pkg/providers/definitions/perplexity.yaml` returns no diff
- `go test ./pkg/providers/... -count=1` passes
- `go test ./pkg/engine/... -count=1` passes
</acceptance_criteria>
<done>6 search/embeddings providers dual-located.</done>
</task>
<task type="auto">
<name>Task 2: Voice + image/video providers (Deepgram, ElevenLabs, Stability, Runway, Midjourney)</name>
<files>providers/deepgram.yaml, providers/elevenlabs.yaml, providers/stability.yaml, providers/runway.yaml, providers/midjourney.yaml, pkg/providers/definitions/deepgram.yaml, pkg/providers/definitions/elevenlabs.yaml, pkg/providers/definitions/stability.yaml, pkg/providers/definitions/runway.yaml, pkg/providers/definitions/midjourney.yaml</files>
<read_first>
- pkg/providers/schema.go
</read_first>
<action>
providers/deepgram.yaml:
```yaml
format_version: 1
name: deepgram
display_name: Deepgram
tier: 3
last_verified: "2026-04-05"
keywords:
- "deepgram"
- "DEEPGRAM_API_KEY"
- "api.deepgram.com"
- "dg-api"
patterns:
- regex: '[a-f0-9]{40}'
entropy_min: 4.0
confidence: low
verify:
method: GET
url: https://api.deepgram.com/v1/projects
headers:
Authorization: "Token {KEY}"
valid_status: [200]
invalid_status: [401, 403]
```
providers/elevenlabs.yaml:
```yaml
format_version: 1
name: elevenlabs
display_name: ElevenLabs
tier: 3
last_verified: "2026-04-05"
keywords:
- "elevenlabs"
- "ELEVENLABS_API_KEY"
- "ELEVEN_API_KEY"
- "XI_API_KEY"
- "api.elevenlabs.io"
patterns:
- regex: '[a-f0-9]{32}'
entropy_min: 4.0
confidence: low
verify:
method: GET
url: https://api.elevenlabs.io/v1/user
headers:
xi-api-key: "{KEY}"
valid_status: [200]
invalid_status: [401, 403]
```
providers/stability.yaml:
```yaml
format_version: 1
name: stability
display_name: Stability AI
tier: 3
last_verified: "2026-04-05"
keywords:
- "stability"
- "STABILITY_API_KEY"
- "STABILITY_KEY"
- "api.stability.ai"
- "stable-diffusion"
patterns:
- regex: 'sk-[A-Za-z0-9]{48}'
entropy_min: 4.0
confidence: medium
verify:
method: GET
url: https://api.stability.ai/v1/user/account
headers:
Authorization: "Bearer {KEY}"
valid_status: [200]
invalid_status: [401, 403]
```
providers/runway.yaml:
```yaml
format_version: 1
name: runway
display_name: Runway
tier: 3
last_verified: "2026-04-05"
keywords:
- "runway"
- "runwayml"
- "RUNWAY_API_KEY"
- "RUNWAYML_API_SECRET"
- "api.runwayml.com"
- "dev.runwayml.com"
verify:
method: GET
url: https://api.dev.runwayml.com/v1/tasks
headers:
Authorization: "Bearer {KEY}"
valid_status: [200, 400]
invalid_status: [401, 403]
```
providers/midjourney.yaml:
```yaml
format_version: 1
name: midjourney
display_name: Midjourney
tier: 3
last_verified: "2026-04-05"
keywords:
- "midjourney"
- "MIDJOURNEY_API_KEY"
- "MJ_API_KEY"
- "midjourney.com"
- "useapi.net"
verify:
method: GET
url: ""
headers: {}
valid_status: []
invalid_status: []
```
Copy all 5 files verbatim to pkg/providers/definitions/.
</action>
<verify>
<automated>cd /home/salva/Documents/apikey && for f in deepgram elevenlabs stability runway midjourney; do diff providers/$f.yaml pkg/providers/definitions/$f.yaml || exit 1; done && go test ./pkg/providers/... -count=1 && go test ./pkg/engine/... -count=1</automated>
</verify>
<acceptance_criteria>
- All 10 files exist
- `grep -q 'XI_API_KEY' providers/elevenlabs.yaml`
- `grep -q 'stable-diffusion' providers/stability.yaml`
- `grep -q 'runwayml' providers/runway.yaml`
- `grep -L 'patterns:' providers/runway.yaml providers/midjourney.yaml` returns both (keyword-only)
- Total Tier 3 count: `grep -l 'tier: 3' providers/*.yaml | wc -l` → 12 (11 new + pre-existing huggingface)
- `go test ./pkg/providers/... -count=1` passes
- `go test ./pkg/engine/... -count=1` passes
</acceptance_criteria>
<done>All 11 new Tier 3 Specialized providers dual-located. PROV-03 satisfied (12 total with pre-existing huggingface).</done>
</task>
</tasks>
<verification>
`grep -l 'tier: 3' providers/*.yaml | wc -l` returns 12.
</verification>
<success_criteria>
- 11 new Tier 3 providers added (huggingface pre-existing, total 12)
- All dual-located
- Documented prefixes use tight regex; undocumented use keyword-only
- No engine regression
</success_criteria>
<output>
After completion, create `.planning/phases/03-tier-3-9-providers/03-02-SUMMARY.md`
</output>