419 lines
11 KiB
Markdown
419 lines
11 KiB
Markdown
---
|
||
phase: 03-tier-3-9-providers
|
||
plan: 02
|
||
type: execute
|
||
wave: 1
|
||
depends_on: []
|
||
files_modified:
|
||
- providers/perplexity.yaml
|
||
- providers/you.yaml
|
||
- providers/voyage.yaml
|
||
- providers/jina.yaml
|
||
- providers/unstructured.yaml
|
||
- providers/assemblyai.yaml
|
||
- providers/deepgram.yaml
|
||
- providers/elevenlabs.yaml
|
||
- providers/stability.yaml
|
||
- providers/runway.yaml
|
||
- providers/midjourney.yaml
|
||
- pkg/providers/definitions/perplexity.yaml
|
||
- pkg/providers/definitions/you.yaml
|
||
- pkg/providers/definitions/voyage.yaml
|
||
- pkg/providers/definitions/jina.yaml
|
||
- pkg/providers/definitions/unstructured.yaml
|
||
- pkg/providers/definitions/assemblyai.yaml
|
||
- pkg/providers/definitions/deepgram.yaml
|
||
- pkg/providers/definitions/elevenlabs.yaml
|
||
- pkg/providers/definitions/stability.yaml
|
||
- pkg/providers/definitions/runway.yaml
|
||
- pkg/providers/definitions/midjourney.yaml
|
||
autonomous: true
|
||
requirements: [PROV-03]
|
||
must_haves:
|
||
truths:
|
||
- "11 new Tier 3 Specialized provider YAMLs load (huggingface already exists → 12 total)"
|
||
- "Providers with documented prefixes (pplx-, xai-voyage, sk-, jina_) use tight regex"
|
||
- "Voice/image providers without documented prefixes use keyword-only"
|
||
artifacts:
|
||
- path: "providers/perplexity.yaml"
|
||
provides: "Perplexity pplx- prefix pattern"
|
||
contains: "pplx-"
|
||
- path: "providers/elevenlabs.yaml"
|
||
provides: "ElevenLabs API key keyword detection"
|
||
contains: "elevenlabs"
|
||
- path: "providers/assemblyai.yaml"
|
||
provides: "AssemblyAI keyword detection"
|
||
contains: "assemblyai"
|
||
key_links:
|
||
- from: "provider keywords[]"
|
||
to: "Registry Aho-Corasick automaton"
|
||
via: "NewRegistry()"
|
||
pattern: "keywords"
|
||
---
|
||
|
||
<objective>
|
||
Create 11 Tier 3 Specialized LLM/AI provider YAMLs — search (Perplexity, You.com), embeddings (Voyage, Jina, Unstructured), voice (AssemblyAI, Deepgram, ElevenLabs), and image/video (Stability, Runway, Midjourney). Huggingface is pre-existing from Phase 2 and must NOT be recreated. Total Tier 3 = 12 after this plan.
|
||
|
||
Purpose: Satisfy PROV-03 (12 Tier 3 Specialized providers).
|
||
|
||
Output: 22 YAML files (11 providers × 2 locations).
|
||
|
||
Addresses PROV-03.
|
||
</objective>
|
||
|
||
<execution_context>
|
||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||
</execution_context>
|
||
|
||
<context>
|
||
@.planning/ROADMAP.md
|
||
@.planning/phases/03-tier-3-9-providers/03-CONTEXT.md
|
||
@pkg/providers/schema.go
|
||
@providers/huggingface.yaml
|
||
@providers/cohere.yaml
|
||
|
||
<interfaces>
|
||
Schema: pkg/providers/schema.go. No `category` field. Confidence: high|medium|low.
|
||
Patterns field may be omitted for keyword-only providers. Keywords ≥3 required.
|
||
Dual-location required (providers/ + pkg/providers/definitions/).
|
||
RE2 regex only.
|
||
|
||
IMPORTANT: providers/huggingface.yaml ALREADY EXISTS (tier: 3). Do NOT modify or recreate it.
|
||
</interfaces>
|
||
</context>
|
||
|
||
<tasks>
|
||
|
||
<task type="auto">
|
||
<name>Task 1: Search + embeddings providers (Perplexity, You.com, Voyage, Jina, Unstructured, AssemblyAI)</name>
|
||
<files>providers/perplexity.yaml, providers/you.yaml, providers/voyage.yaml, providers/jina.yaml, providers/unstructured.yaml, providers/assemblyai.yaml, pkg/providers/definitions/perplexity.yaml, pkg/providers/definitions/you.yaml, pkg/providers/definitions/voyage.yaml, pkg/providers/definitions/jina.yaml, pkg/providers/definitions/unstructured.yaml, pkg/providers/definitions/assemblyai.yaml</files>
|
||
<read_first>
|
||
- pkg/providers/schema.go
|
||
- providers/huggingface.yaml (existing tier 3 reference — DO NOT MODIFY)
|
||
</read_first>
|
||
<action>
|
||
Create each file in providers/ AND copy verbatim to pkg/providers/definitions/.
|
||
|
||
providers/perplexity.yaml:
|
||
```yaml
|
||
format_version: 1
|
||
name: perplexity
|
||
display_name: Perplexity AI
|
||
tier: 3
|
||
last_verified: "2026-04-05"
|
||
keywords:
|
||
- "perplexity"
|
||
- "PERPLEXITY_API_KEY"
|
||
- "pplx-"
|
||
- "api.perplexity.ai"
|
||
patterns:
|
||
- regex: 'pplx-[A-Za-z0-9]{48,}'
|
||
entropy_min: 4.0
|
||
confidence: high
|
||
verify:
|
||
method: POST
|
||
url: https://api.perplexity.ai/chat/completions
|
||
headers:
|
||
Authorization: "Bearer {KEY}"
|
||
valid_status: [200, 400]
|
||
invalid_status: [401, 403]
|
||
```
|
||
|
||
providers/you.yaml:
|
||
```yaml
|
||
format_version: 1
|
||
name: you
|
||
display_name: You.com
|
||
tier: 3
|
||
last_verified: "2026-04-05"
|
||
keywords:
|
||
- "you.com"
|
||
- "YDC_API_KEY"
|
||
- "YOU_API_KEY"
|
||
- "api.ydc-index.io"
|
||
verify:
|
||
method: GET
|
||
url: https://api.ydc-index.io/search
|
||
headers:
|
||
X-API-Key: "{KEY}"
|
||
valid_status: [200, 400]
|
||
invalid_status: [401, 403]
|
||
```
|
||
|
||
providers/voyage.yaml:
|
||
```yaml
|
||
format_version: 1
|
||
name: voyage
|
||
display_name: Voyage AI
|
||
tier: 3
|
||
last_verified: "2026-04-05"
|
||
keywords:
|
||
- "voyage"
|
||
- "VOYAGE_API_KEY"
|
||
- "voyageai"
|
||
- "api.voyageai.com"
|
||
patterns:
|
||
- regex: 'pa-[A-Za-z0-9_\-]{40,}'
|
||
entropy_min: 4.0
|
||
confidence: medium
|
||
verify:
|
||
method: POST
|
||
url: https://api.voyageai.com/v1/embeddings
|
||
headers:
|
||
Authorization: "Bearer {KEY}"
|
||
valid_status: [200, 400]
|
||
invalid_status: [401, 403]
|
||
```
|
||
|
||
providers/jina.yaml:
|
||
```yaml
|
||
format_version: 1
|
||
name: jina
|
||
display_name: Jina AI
|
||
tier: 3
|
||
last_verified: "2026-04-05"
|
||
keywords:
|
||
- "jina"
|
||
- "JINA_API_KEY"
|
||
- "api.jina.ai"
|
||
- "jinaai"
|
||
patterns:
|
||
- regex: 'jina_[A-Za-z0-9]{40,}'
|
||
entropy_min: 4.0
|
||
confidence: high
|
||
verify:
|
||
method: GET
|
||
url: https://api.jina.ai/v1/models
|
||
headers:
|
||
Authorization: "Bearer {KEY}"
|
||
valid_status: [200]
|
||
invalid_status: [401, 403]
|
||
```
|
||
|
||
providers/unstructured.yaml:
|
||
```yaml
|
||
format_version: 1
|
||
name: unstructured
|
||
display_name: Unstructured.io
|
||
tier: 3
|
||
last_verified: "2026-04-05"
|
||
keywords:
|
||
- "unstructured"
|
||
- "UNSTRUCTURED_API_KEY"
|
||
- "api.unstructured.io"
|
||
- "unstructuredio"
|
||
verify:
|
||
method: GET
|
||
url: https://api.unstructured.io/general/v0/general
|
||
headers:
|
||
unstructured-api-key: "{KEY}"
|
||
valid_status: [200, 400]
|
||
invalid_status: [401, 403]
|
||
```
|
||
|
||
providers/assemblyai.yaml:
|
||
```yaml
|
||
format_version: 1
|
||
name: assemblyai
|
||
display_name: AssemblyAI
|
||
tier: 3
|
||
last_verified: "2026-04-05"
|
||
keywords:
|
||
- "assemblyai"
|
||
- "ASSEMBLYAI_API_KEY"
|
||
- "api.assemblyai.com"
|
||
- "assembly.ai"
|
||
patterns:
|
||
- regex: '[a-f0-9]{32}'
|
||
entropy_min: 4.0
|
||
confidence: low
|
||
verify:
|
||
method: GET
|
||
url: https://api.assemblyai.com/v2/transcript
|
||
headers:
|
||
authorization: "{KEY}"
|
||
valid_status: [200]
|
||
invalid_status: [401, 403]
|
||
```
|
||
|
||
Copy all 6 files verbatim to pkg/providers/definitions/.
|
||
</action>
|
||
<verify>
|
||
<automated>cd /home/salva/Documents/apikey && for f in perplexity you voyage jina unstructured assemblyai; do diff providers/$f.yaml pkg/providers/definitions/$f.yaml || exit 1; done && go test ./pkg/providers/... -count=1 && go test ./pkg/engine/... -count=1</automated>
|
||
</verify>
|
||
<acceptance_criteria>
|
||
- All 12 files exist (6 providers × 2 locations)
|
||
- `grep -q 'pplx-' providers/perplexity.yaml`
|
||
- `grep -q 'jina_' providers/jina.yaml`
|
||
- `grep -q 'api.voyageai.com' providers/voyage.yaml`
|
||
- `diff providers/perplexity.yaml pkg/providers/definitions/perplexity.yaml` returns no diff
|
||
- `go test ./pkg/providers/... -count=1` passes
|
||
- `go test ./pkg/engine/... -count=1` passes
|
||
</acceptance_criteria>
|
||
<done>6 search/embeddings providers dual-located.</done>
|
||
</task>
|
||
|
||
<task type="auto">
|
||
<name>Task 2: Voice + image/video providers (Deepgram, ElevenLabs, Stability, Runway, Midjourney)</name>
|
||
<files>providers/deepgram.yaml, providers/elevenlabs.yaml, providers/stability.yaml, providers/runway.yaml, providers/midjourney.yaml, pkg/providers/definitions/deepgram.yaml, pkg/providers/definitions/elevenlabs.yaml, pkg/providers/definitions/stability.yaml, pkg/providers/definitions/runway.yaml, pkg/providers/definitions/midjourney.yaml</files>
|
||
<read_first>
|
||
- pkg/providers/schema.go
|
||
</read_first>
|
||
<action>
|
||
providers/deepgram.yaml:
|
||
```yaml
|
||
format_version: 1
|
||
name: deepgram
|
||
display_name: Deepgram
|
||
tier: 3
|
||
last_verified: "2026-04-05"
|
||
keywords:
|
||
- "deepgram"
|
||
- "DEEPGRAM_API_KEY"
|
||
- "api.deepgram.com"
|
||
- "dg-api"
|
||
patterns:
|
||
- regex: '[a-f0-9]{40}'
|
||
entropy_min: 4.0
|
||
confidence: low
|
||
verify:
|
||
method: GET
|
||
url: https://api.deepgram.com/v1/projects
|
||
headers:
|
||
Authorization: "Token {KEY}"
|
||
valid_status: [200]
|
||
invalid_status: [401, 403]
|
||
```
|
||
|
||
providers/elevenlabs.yaml:
|
||
```yaml
|
||
format_version: 1
|
||
name: elevenlabs
|
||
display_name: ElevenLabs
|
||
tier: 3
|
||
last_verified: "2026-04-05"
|
||
keywords:
|
||
- "elevenlabs"
|
||
- "ELEVENLABS_API_KEY"
|
||
- "ELEVEN_API_KEY"
|
||
- "XI_API_KEY"
|
||
- "api.elevenlabs.io"
|
||
patterns:
|
||
- regex: '[a-f0-9]{32}'
|
||
entropy_min: 4.0
|
||
confidence: low
|
||
verify:
|
||
method: GET
|
||
url: https://api.elevenlabs.io/v1/user
|
||
headers:
|
||
xi-api-key: "{KEY}"
|
||
valid_status: [200]
|
||
invalid_status: [401, 403]
|
||
```
|
||
|
||
providers/stability.yaml:
|
||
```yaml
|
||
format_version: 1
|
||
name: stability
|
||
display_name: Stability AI
|
||
tier: 3
|
||
last_verified: "2026-04-05"
|
||
keywords:
|
||
- "stability"
|
||
- "STABILITY_API_KEY"
|
||
- "STABILITY_KEY"
|
||
- "api.stability.ai"
|
||
- "stable-diffusion"
|
||
patterns:
|
||
- regex: 'sk-[A-Za-z0-9]{48}'
|
||
entropy_min: 4.0
|
||
confidence: medium
|
||
verify:
|
||
method: GET
|
||
url: https://api.stability.ai/v1/user/account
|
||
headers:
|
||
Authorization: "Bearer {KEY}"
|
||
valid_status: [200]
|
||
invalid_status: [401, 403]
|
||
```
|
||
|
||
providers/runway.yaml:
|
||
```yaml
|
||
format_version: 1
|
||
name: runway
|
||
display_name: Runway
|
||
tier: 3
|
||
last_verified: "2026-04-05"
|
||
keywords:
|
||
- "runway"
|
||
- "runwayml"
|
||
- "RUNWAY_API_KEY"
|
||
- "RUNWAYML_API_SECRET"
|
||
- "api.runwayml.com"
|
||
- "dev.runwayml.com"
|
||
verify:
|
||
method: GET
|
||
url: https://api.dev.runwayml.com/v1/tasks
|
||
headers:
|
||
Authorization: "Bearer {KEY}"
|
||
valid_status: [200, 400]
|
||
invalid_status: [401, 403]
|
||
```
|
||
|
||
providers/midjourney.yaml:
|
||
```yaml
|
||
format_version: 1
|
||
name: midjourney
|
||
display_name: Midjourney
|
||
tier: 3
|
||
last_verified: "2026-04-05"
|
||
keywords:
|
||
- "midjourney"
|
||
- "MIDJOURNEY_API_KEY"
|
||
- "MJ_API_KEY"
|
||
- "midjourney.com"
|
||
- "useapi.net"
|
||
verify:
|
||
method: GET
|
||
url: ""
|
||
headers: {}
|
||
valid_status: []
|
||
invalid_status: []
|
||
```
|
||
|
||
Copy all 5 files verbatim to pkg/providers/definitions/.
|
||
</action>
|
||
<verify>
|
||
<automated>cd /home/salva/Documents/apikey && for f in deepgram elevenlabs stability runway midjourney; do diff providers/$f.yaml pkg/providers/definitions/$f.yaml || exit 1; done && go test ./pkg/providers/... -count=1 && go test ./pkg/engine/... -count=1</automated>
|
||
</verify>
|
||
<acceptance_criteria>
|
||
- All 10 files exist
|
||
- `grep -q 'XI_API_KEY' providers/elevenlabs.yaml`
|
||
- `grep -q 'stable-diffusion' providers/stability.yaml`
|
||
- `grep -q 'runwayml' providers/runway.yaml`
|
||
- `grep -L 'patterns:' providers/runway.yaml providers/midjourney.yaml` returns both (keyword-only)
|
||
- Total Tier 3 count: `grep -l 'tier: 3' providers/*.yaml | wc -l` → 12 (11 new + pre-existing huggingface)
|
||
- `go test ./pkg/providers/... -count=1` passes
|
||
- `go test ./pkg/engine/... -count=1` passes
|
||
</acceptance_criteria>
|
||
<done>All 11 new Tier 3 Specialized providers dual-located. PROV-03 satisfied (12 total with pre-existing huggingface).</done>
|
||
</task>
|
||
|
||
</tasks>
|
||
|
||
<verification>
|
||
`grep -l 'tier: 3' providers/*.yaml | wc -l` returns 12.
|
||
</verification>
|
||
|
||
<success_criteria>
|
||
- 11 new Tier 3 providers added (huggingface pre-existing, total 12)
|
||
- All dual-located
|
||
- Documented prefixes use tight regex; undocumented use keyword-only
|
||
- No engine regression
|
||
</success_criteria>
|
||
|
||
<output>
|
||
After completion, create `.planning/phases/03-tier-3-9-providers/03-02-SUMMARY.md`
|
||
</output>
|