--- phase: 09-osint-infrastructure plan: 01 type: execute wave: 1 depends_on: [] files_modified: - pkg/recon/source.go - pkg/recon/engine.go - pkg/recon/example.go - pkg/recon/engine_test.go autonomous: true requirements: [RECON-INFRA-08] must_haves: truths: - "pkg/recon package compiles with a ReconSource interface" - "Engine.Register adds a source; Engine.List returns registered names" - "Engine.SweepAll fans out to all enabled sources via ants pool and returns aggregated Findings" - "ExampleSource implements ReconSource end-to-end and emits a deterministic fake Finding" artifacts: - path: "pkg/recon/source.go" provides: "ReconSource interface + Finding type alias + Config struct" contains: "type ReconSource interface" - path: "pkg/recon/engine.go" provides: "Engine with Register, List, SweepAll (parallel fanout via ants)" contains: "func (e *Engine) SweepAll" - path: "pkg/recon/example.go" provides: "ExampleSource stub that emits hardcoded findings" contains: "type ExampleSource" - path: "pkg/recon/engine_test.go" provides: "Tests for Register/List/SweepAll with ExampleSource" contains: "func TestSweepAll" key_links: - from: "pkg/recon/engine.go" to: "github.com/panjf2000/ants/v2" via: "parallel source fanout" pattern: "ants\\.NewPool" - from: "pkg/recon/engine.go" to: "pkg/engine.Finding" via: "aliased as recon.Finding for SourceType=\"recon:*\"" pattern: "engine\\.Finding" --- Create the pkg/recon/ package foundation: ReconSource interface, Engine orchestrator with parallel fanout via ants pool, and an ExampleSource stub that proves the pipeline end-to-end. This is the contract that all later sources (Phases 10-16) will implement. Purpose: Establish the interface + engine skeleton so subsequent Wave 1 plans (limiter, stealth, robots) can land in parallel without conflict, and Wave 2 can wire the CLI. Output: pkg/recon/source.go, pkg/recon/engine.go, pkg/recon/example.go, pkg/recon/engine_test.go @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md @.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/09-osint-infrastructure/09-CONTEXT.md @pkg/engine/engine.go @pkg/engine/finding.go From pkg/engine/finding.go: ```go type Finding struct { ProviderName string KeyValue string KeyMasked string Confidence string Source string SourceType string // existing: "file","git","stdin","url","clipboard". New: "recon:" LineNumber int Offset int64 DetectedAt time.Time Verified bool VerifyStatus string VerifyHTTPCode int VerifyMetadata map[string]string VerifyError string } ``` ants pool pattern (pkg/engine/engine.go): `ants.NewPool(workers)`, `pool.Submit(func(){...})`, `pool.Release()`, coordinated via `sync.WaitGroup`. Task 1: Define ReconSource interface and Config pkg/recon/source.go - ReconSource interface has methods: Name() string, RateLimit() rate.Limit, Burst() int, RespectsRobots() bool, Enabled(cfg Config) bool, Sweep(ctx, query, out chan<- Finding) error - Finding is a type alias for pkg/engine.Finding so downstream code reuses the existing storage path - Config struct carries Stealth bool, RespectRobots bool, EnabledSources []string, Query string Create pkg/recon/source.go with package recon. Import golang.org/x/time/rate, context, and pkg/engine. ```go package recon import ( "context" "golang.org/x/time/rate" "github.com/salvacybersec/keyhunter/pkg/engine" ) // Finding is the recon package's alias for the canonical engine.Finding. // Recon sources set SourceType = "recon:". type Finding = engine.Finding // Config controls a recon sweep. type Config struct { Stealth bool RespectRobots bool EnabledSources []string // empty = all Query string } // ReconSource is implemented by every OSINT source module (Phases 10-16). // Each source owns its own rate.Limiter constructed from RateLimit()/Burst(). type ReconSource interface { Name() string RateLimit() rate.Limit Burst() int RespectsRobots() bool Enabled(cfg Config) bool Sweep(ctx context.Context, query string, out chan<- Finding) error } ``` Per Config decisions in 09-CONTEXT.md. No external deps beyond golang.org/x/time/rate (already in go.mod) and pkg/engine. cd /home/salva/Documents/apikey && go build ./pkg/recon/... pkg/recon/source.go compiles; ReconSource interface exported; Finding aliased to engine.Finding. Task 2: Engine with Register/List/SweepAll + ExampleSource + tests pkg/recon/engine.go, pkg/recon/example.go, pkg/recon/engine_test.go - Engine.Register(src ReconSource) adds to internal map keyed by Name() - Engine.List() returns sorted source names - Engine.SweepAll(ctx, cfg) runs every enabled source in parallel via ants pool, collects Findings from a shared channel, and returns []Finding. Dedup is NOT done here (Plan 09-03 owns dedup.go); SweepAll just aggregates. - Each source call is wrapped in its own goroutine submitted to ants.Pool; uses sync.WaitGroup to close the out channel after all sources finish - ExampleSource.Name()="example", RateLimit()=rate.Limit(10), Burst()=1, RespectsRobots()=false, Enabled always true, Sweep emits two deterministic Findings with SourceType="recon:example" - TestSweepAll registers ExampleSource, runs SweepAll, asserts exactly 2 findings with SourceType="recon:example" - TestRegisterList asserts List() returns ["example"] after registering Create pkg/recon/engine.go: ```go package recon import ( "context" "sort" "sync" "github.com/panjf2000/ants/v2" ) type Engine struct { mu sync.RWMutex sources map[string]ReconSource } func NewEngine() *Engine { return &Engine{sources: make(map[string]ReconSource)} } func (e *Engine) Register(s ReconSource) { e.mu.Lock() defer e.mu.Unlock() e.sources[s.Name()] = s } func (e *Engine) List() []string { e.mu.RLock() defer e.mu.RUnlock() names := make([]string, 0, len(e.sources)) for n := range e.sources { names = append(names, n) } sort.Strings(names) return names } // SweepAll fans out to every enabled source in parallel via ants pool and // returns aggregated findings. Deduplication is performed by callers using // pkg/recon.Dedup (plan 09-03). func (e *Engine) SweepAll(ctx context.Context, cfg Config) ([]Finding, error) { e.mu.RLock() active := make([]ReconSource, 0, len(e.sources)) for _, s := range e.sources { if s.Enabled(cfg) { active = append(active, s) } } e.mu.RUnlock() if len(active) == 0 { return nil, nil } pool, err := ants.NewPool(len(active)) if err != nil { return nil, err } defer pool.Release() out := make(chan Finding, 256) var wg sync.WaitGroup for _, s := range active { s := s wg.Add(1) _ = pool.Submit(func() { defer wg.Done() _ = s.Sweep(ctx, cfg.Query, out) }) } go func() { wg.Wait(); close(out) }() var all []Finding for f := range out { all = append(all, f) } return all, nil } ``` Create pkg/recon/example.go with an ExampleSource emitting two deterministic Findings (SourceType="recon:example", fake masked keys, distinct Source URLs) to prove the pipeline. ```go package recon import ( "context" "time" "golang.org/x/time/rate" ) type ExampleSource struct{} func (ExampleSource) Name() string { return "example" } func (ExampleSource) RateLimit() rate.Limit { return rate.Limit(10) } func (ExampleSource) Burst() int { return 1 } func (ExampleSource) RespectsRobots() bool { return false } func (ExampleSource) Enabled(_ Config) bool { return true } func (ExampleSource) Sweep(ctx context.Context, query string, out chan<- Finding) error { fakes := []Finding{ {ProviderName: "openai", KeyMasked: "sk-examp...AAAA", Source: "https://example.invalid/a", SourceType: "recon:example", DetectedAt: time.Now()}, {ProviderName: "anthropic", KeyMasked: "sk-ant-e...BBBB", Source: "https://example.invalid/b", SourceType: "recon:example", DetectedAt: time.Now()}, } for _, f := range fakes { select { case out <- f: case <-ctx.Done(): return ctx.Err() } } return nil } ``` Create pkg/recon/engine_test.go with TestRegisterList and TestSweepAll using ExampleSource. Use testify require. TDD: write tests first, they fail, then implement. cd /home/salva/Documents/apikey && go test ./pkg/recon/ -run 'TestRegisterList|TestSweepAll' -count=1 Tests pass. Engine registers ExampleSource, SweepAll returns 2 findings with SourceType="recon:example". - `go build ./pkg/recon/...` succeeds - `go test ./pkg/recon/ -count=1` passes - `go vet ./pkg/recon/...` clean - ReconSource interface exported - Engine.Register/List/SweepAll implemented and tested - ExampleSource proves end-to-end fanout - No cycles with pkg/engine (recon imports engine, not vice versa) After completion, create `.planning/phases/09-osint-infrastructure/09-01-SUMMARY.md`