--- phase: 10-osint-code-hosting plan: 09 subsystem: recon tags: [register, integration, cmd, viper, httptest] requires: - phase: 10-osint-code-hosting provides: "Ten code-hosting ReconSource implementations (Plans 10-01..10-08)" provides: - "sources.RegisterAll wires all ten Phase 10 sources onto a recon.Engine" - "cmd/recon.go constructs real SourcesConfig from env + viper and invokes RegisterAll" - "End-to-end SweepAll integration test exercising every source against one multiplexed httptest server" affects: [11-osint-pastebins, 12-osint-search-engines, cli-recon] tech-stack: added: [] patterns: - "Env-var → viper fallback (firstNonEmpty) for recon credential lookup" - "Unconditional source registration: credless sources register but Enabled()==false, uniform CLI surface" - "Single httptest.ServeMux routing per-path fixtures for multi-source integration tests" key-files: created: - pkg/recon/sources/register_test.go - pkg/recon/sources/integration_test.go - .planning/phases/10-osint-code-hosting/deferred-items.md modified: - pkg/recon/sources/register.go - cmd/recon.go key-decisions: - "Register all ten sources unconditionally so `keyhunter recon list` shows the full catalog regardless of configured credentials; missing creds just flip Enabled()==false" - "Integration test constructs sources directly with BaseURL overrides (not via RegisterAll) because RegisterAll wires production URLs" - "Credential precedence: env var → viper config key → empty (source disabled)" - "Single multiplexed httptest server used instead of ten separate servers — simpler and matches how recon.Engine fans out in parallel" - "firstNonEmpty helper kept local to cmd/recon.go rather than pkg-level to avoid exporting a trivial utility" patterns-established: - "sources.RegisterAll(engine, cfg) is the single call cmd-layer code must make to wire Phase 10" - "Integration tests that need to drive many sources from one server encode the sub-source into the URL path (/search/code, /api/v4/search, etc.)" - "Struct literals for sources that lazy-init `client` in Sweep; NewXxxSource constructor for sources that don't (GitHubSource, KaggleSource, HuggingFaceSource)" requirements-completed: [RECON-CODE-10] duration: 12min completed: 2026-04-05 --- # Phase 10 Plan 09: RegisterAll + cmd/recon + Integration Test Summary **Ten Phase 10 code-hosting sources now wire onto recon.Engine via sources.RegisterAll, the CLI reads credentials from env+viper, and an end-to-end integration test drives every source through SweepAll against one multiplexed httptest server.** ## Performance - **Duration:** ~12 min - **Tasks:** 2 (both TDD) - **Files created:** 3 - **Files modified:** 2 ## Accomplishments - `sources.RegisterAll` wires all ten sources (github, gitlab, bitbucket, gist, codeberg, huggingface, replit, codesandbox, sandboxes, kaggle) onto a `*recon.Engine` in one call - Extended `SourcesConfig` with `BitbucketWorkspace` and `CodebergToken` fields to match Wave 2 constructor signatures - `cmd/recon.go` now loads providers.Registry, constructs a full `SourcesConfig` from env vars (`GITHUB_TOKEN`, `GITLAB_TOKEN`, `BITBUCKET_TOKEN`, `BITBUCKET_WORKSPACE`, `CODEBERG_TOKEN`, `HUGGINGFACE_TOKEN`, `KAGGLE_USERNAME`, `KAGGLE_KEY`) with viper fallback keys under `recon..*`, and calls `sources.RegisterAll` - `keyhunter recon list` now prints all eleven source names (`example` + ten Phase 10 sources) - Integration test (`integration_test.go::TestIntegration_AllSources_SweepAll`) spins up a single `httptest` server with per-path handlers for every source's API/HTML fixture, registers all ten sources (with BaseURL overrides) on a fresh `recon.Engine`, runs `SweepAll`, and asserts at least one `Finding` was emitted for each of the ten `recon:*` `SourceType` values - `register_test.go` covers RegisterAll contracts: exactly ten sources registered in deterministic sorted order, nil engine is a no-op, and empty credentials still produce a full registration list ## Verification - `go test ./pkg/recon/sources/ -run TestRegisterAll -v` → 4 passing (nil, empty cfg, all-ten, missing-creds) - `go test ./pkg/recon/sources/ -run TestIntegration_AllSources_SweepAll -v` → passing; asserts 10/10 SourceType buckets populated - `go test ./pkg/recon/...` → all green (35s, includes pre-existing per-source suites) - `go vet ./...` → clean - `go build ./...` → clean - `go run . recon list` → prints `bitbucket codeberg codesandbox example gist github gitlab huggingface kaggle replit sandboxes` ## Deviations from Plan None — plan executed as written. One out-of-scope finding was identified and logged to `deferred-items.md` (GitHubSource.Sweep dereferences `s.client` without a nil check; safe in current code paths because `RegisterAll` uses `NewGitHubSource` which initializes it, but a latent footgun for future struct-literal callers). ## Known Stubs None. All ten sources are production-wired through `RegisterAll` and exercised by the integration test against realistic fixtures. ## Commits - `4628ccf` test(10-09): add failing RegisterAll wiring tests - `fb3e573` feat(10-09): wire all ten Phase 10 sources in RegisterAll - `8528108` test(10-09): add end-to-end SweepAll integration test across all ten sources - `e00fb17` feat(10-09): wire sources.RegisterAll into cmd/recon with viper+env credential lookup ## Self-Check: PASSED - pkg/recon/sources/register.go — FOUND - pkg/recon/sources/register_test.go — FOUND - pkg/recon/sources/integration_test.go — FOUND - cmd/recon.go — FOUND - commits 4628ccf, fb3e573, 8528108, e00fb17 — FOUND