docs(09-05): complete recon CLI command tree plan

This commit is contained in:
salvacybersec
2026-04-06 00:48:42 +03:00
parent 86a6bb864b
commit 0ff9edc6c1
4 changed files with 105 additions and 12 deletions

View File

@@ -207,7 +207,7 @@ Requirements for initial release. Each maps to roadmap phases.
- [x] **RECON-INFRA-05**: Per-source rate limiter with configurable limits
- [ ] **RECON-INFRA-06**: Stealth mode (--stealth) with UA rotation and increased delays
- [x] **RECON-INFRA-07**: robots.txt respect (--respect-robots, default on)
- [ ] **RECON-INFRA-08**: Recon full command — parallel sweep across all sources with deduplication
- [x] **RECON-INFRA-08**: Recon full command — parallel sweep across all sources with deduplication
### Dork Engine

View File

@@ -198,11 +198,11 @@ Plans:
3. `keyhunter recon full --respect-robots` (default on) respects robots.txt for web-scraping sources before making any requests
4. `keyhunter recon full` fans out to all enabled sources in parallel and deduplicates findings before persisting to the database
**Plans**: 6 plans
- [ ] 09-01-PLAN.md — ReconSource interface + Engine skeleton + ExampleSource stub
- [x] 09-01-PLAN.md — ReconSource interface + Engine skeleton + ExampleSource stub
- [x] 09-02-PLAN.md — LimiterRegistry per-source rate.Limiter + jitter
- [ ] 09-03-PLAN.md — Stealth UA pool + cross-source dedup
- [x] 09-03-PLAN.md — Stealth UA pool + cross-source dedup
- [x] 09-04-PLAN.md — robots.txt parser with 1h per-host cache
- [ ] 09-05-PLAN.md — cmd/recon.go CLI tree (full, list)
- [x] 09-05-PLAN.md — cmd/recon.go CLI tree (full, list)
- [ ] 09-06-PLAN.md — Integration test + phase summary
### Phase 10: OSINT Code Hosting

View File

@@ -3,14 +3,14 @@ gsd_state_version: 1.0
milestone: v1.0
milestone_name: milestone
status: executing
stopped_at: Completed 09-04-PLAN.md
last_updated: "2026-04-05T21:43:35.883Z"
stopped_at: Completed 09-05-PLAN.md
last_updated: "2026-04-05T21:48:38.558Z"
last_activity: 2026-04-05
progress:
total_phases: 18
completed_phases: 8
total_plans: 53
completed_plans: 49
completed_phases: 7
total_plans: 48
completed_plans: 52
percent: 20
---
@@ -26,7 +26,7 @@ See: .planning/PROJECT.md (updated 2026-04-04)
## Current Position
Phase: 09 (osint-infrastructure) — EXECUTING
Plan: 2 of 6
Plan: 3 of 6
Status: Ready to execute
Last activity: 2026-04-05
@@ -83,6 +83,7 @@ Progress: [██░░░░░░░░] 20%
| Phase 08-dork-engine P03 | 10m | 2 tasks | 10 files |
| Phase 08-dork-engine P07 | 3m | 1 tasks | 1 files |
| Phase 09-osint-infrastructure P04 | 6min | 2 tasks | 4 files |
| Phase 09 P05 | 5m | 2 tasks | 2 files |
## Accumulated Context
@@ -130,6 +131,6 @@ None yet.
## Session Continuity
Last session: 2026-04-05T21:43:35.879Z
Stopped at: Completed 09-04-PLAN.md
Last session: 2026-04-05T21:48:38.555Z
Stopped at: Completed 09-05-PLAN.md
Resume file: None

View File

@@ -0,0 +1,92 @@
---
phase: 09-osint-infrastructure
plan: 05
subsystem: cli
tags: [cli, recon, cobra, osint]
requires: [09-01, 09-02, 09-03, 09-04]
provides:
- "cmd/recon.go with `recon full` and `recon list` subcommands"
- "ExampleSource registered at process start so Phase 9 ships a demonstrable pipeline"
- "Flag surface: --stealth, --respect-robots (default true), --query"
affects:
- "cmd/stubs.go (reconCmd stub removed)"
tech-stack:
added: []
patterns:
- "cobra subcommand tree with package-level flag vars + init() wiring"
- "buildReconEngine() factory so future phases (10-16) extend a single registration site"
key-files:
created:
- "cmd/recon.go"
- ".planning/phases/09-osint-infrastructure/09-05-SUMMARY.md"
modified:
- "cmd/stubs.go"
decisions:
- "Engine is built per-invocation (not a package global) to keep source registration explicit and testable"
- "ExampleSource is registered inline in buildReconEngine rather than via init() side-effect so Phases 10-16 have one obvious place to add real sources"
- "Dedup runs on the caller side (per engine.go comment) — Phase 05 calls recon.Dedup after SweepAll"
metrics:
duration: "~5m"
completed: "2026-04-05"
tasks: 2
files-changed: 2
---
# Phase 09 Plan 05: Recon CLI Command Tree Summary
Wired `pkg/recon` into the Cobra CLI with `keyhunter recon full` and `keyhunter recon list`, replacing the Phase 9 stub so the recon pipeline is demonstrable end-to-end on a fresh clone.
## What Shipped
- **`cmd/recon.go`** — new file declaring `reconCmd`, `reconFullCmd`, `reconListCmd`, and the `buildReconEngine()` factory. Flags `--stealth` (default false), `--respect-robots` (default true), and `--query` (default "") are registered on `reconFullCmd`. `buildReconEngine` constructs a fresh `recon.Engine`, registers `recon.ExampleSource{}`, and returns it.
- **`recon full`** — calls `Engine.SweepAll(ctx, Config{...})`, pipes results through `recon.Dedup`, then prints a header line (`recon: swept N sources, M findings (K after dedup)`) followed by one line per deduped finding.
- **`recon list`** — prints registered source names one per line (sorted by `Engine.List`).
- **`cmd/stubs.go`** — `var reconCmd` stub removed; a comment points at `cmd/recon.go`. `rootCmd.AddCommand(reconCmd)` in `cmd/root.go` now resolves to the real declaration (same package).
## Verification Run
```
$ go run . recon list
example
$ go run . recon full
recon: swept 1 sources, 2 findings (2 after dedup)
[recon:example] openai sk-examp...AAAA https://example.invalid/a
[recon:example] anthropic sk-ant-e...BBBB https://example.invalid/b
$ go run . recon full --stealth --query=test
recon: swept 1 sources, 2 findings (2 after dedup)
...
```
`go build ./...` clean. `go test ./...` green across all packages (cmd, pkg/recon, pkg/dorks, pkg/engine, etc.).
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 3 - Blocker] Merged wave-1 dependencies into worktree**
- **Found during:** Task 2 build verification
- **Issue:** The worktree branch was behind master and missing commits from plans 09-01, 09-02, 09-04 (`pkg/recon/engine.go`, `source.go`, `example.go`, `limiter.go`, `robots.go`). `go build ./...` failed with `undefined: recon.Engine / recon.Config / recon.NewEngine / recon.ExampleSource`.
- **Fix:** Ran `git merge master --no-edit --no-verify` — fast-forward from `1eb86ca` to `c2137ed` pulling in the four dependent plans' artifacts. No conflicts (my Task 1 edit to `cmd/stubs.go` and new `cmd/recon.go` were untouched by the merge).
- **Files modified:** merge brought in `pkg/recon/{engine,source,example,limiter,robots,engine_test,limiter_test,robots_test}.go`, `go.mod`, `go.sum`, and the 09-01/02/04 SUMMARY files.
- **Commit:** merge commit via fast-forward (no new hash)
## Known Stubs
- **`pkg/recon.ExampleSource`** — intentional per plan 09-01 / Phase 9 CONTEXT. It emits two deterministic fake findings (`openai`/`anthropic` with `recon:example` SourceType) so the CLI has visible output before Phases 10-16 land real sources. Will be superseded as Shodan/GitHub/Pastebin/etc. sources are added in Phases 10-16 via additional `e.Register(...)` calls in `buildReconEngine`.
## Commits
- `86a6bb8` — feat(09-05): add recon full/list commands and remove stub
## Self-Check: PASSED
- FOUND: cmd/recon.go
- FOUND: cmd/stubs.go (reconCmd stub removed; verified `! grep -q 'var reconCmd' cmd/stubs.go`)
- FOUND: commit 86a6bb8
- VERIFIED: `go build ./...` clean
- VERIFIED: `go run . recon list` prints `example`
- VERIFIED: `go run . recon full` prints "recon: swept 1 sources, 2 findings (2 after dedup)" + 2 finding lines
- VERIFIED: `go run . recon full --stealth --query=test` runs cleanly
- VERIFIED: `go test ./...` green