docs(09-05): complete recon CLI command tree plan
This commit is contained in:
@@ -207,7 +207,7 @@ Requirements for initial release. Each maps to roadmap phases.
|
||||
- [x] **RECON-INFRA-05**: Per-source rate limiter with configurable limits
|
||||
- [ ] **RECON-INFRA-06**: Stealth mode (--stealth) with UA rotation and increased delays
|
||||
- [x] **RECON-INFRA-07**: robots.txt respect (--respect-robots, default on)
|
||||
- [ ] **RECON-INFRA-08**: Recon full command — parallel sweep across all sources with deduplication
|
||||
- [x] **RECON-INFRA-08**: Recon full command — parallel sweep across all sources with deduplication
|
||||
|
||||
### Dork Engine
|
||||
|
||||
|
||||
@@ -198,11 +198,11 @@ Plans:
|
||||
3. `keyhunter recon full --respect-robots` (default on) respects robots.txt for web-scraping sources before making any requests
|
||||
4. `keyhunter recon full` fans out to all enabled sources in parallel and deduplicates findings before persisting to the database
|
||||
**Plans**: 6 plans
|
||||
- [ ] 09-01-PLAN.md — ReconSource interface + Engine skeleton + ExampleSource stub
|
||||
- [x] 09-01-PLAN.md — ReconSource interface + Engine skeleton + ExampleSource stub
|
||||
- [x] 09-02-PLAN.md — LimiterRegistry per-source rate.Limiter + jitter
|
||||
- [ ] 09-03-PLAN.md — Stealth UA pool + cross-source dedup
|
||||
- [x] 09-03-PLAN.md — Stealth UA pool + cross-source dedup
|
||||
- [x] 09-04-PLAN.md — robots.txt parser with 1h per-host cache
|
||||
- [ ] 09-05-PLAN.md — cmd/recon.go CLI tree (full, list)
|
||||
- [x] 09-05-PLAN.md — cmd/recon.go CLI tree (full, list)
|
||||
- [ ] 09-06-PLAN.md — Integration test + phase summary
|
||||
|
||||
### Phase 10: OSINT Code Hosting
|
||||
|
||||
@@ -3,14 +3,14 @@ gsd_state_version: 1.0
|
||||
milestone: v1.0
|
||||
milestone_name: milestone
|
||||
status: executing
|
||||
stopped_at: Completed 09-04-PLAN.md
|
||||
last_updated: "2026-04-05T21:43:35.883Z"
|
||||
stopped_at: Completed 09-05-PLAN.md
|
||||
last_updated: "2026-04-05T21:48:38.558Z"
|
||||
last_activity: 2026-04-05
|
||||
progress:
|
||||
total_phases: 18
|
||||
completed_phases: 8
|
||||
total_plans: 53
|
||||
completed_plans: 49
|
||||
completed_phases: 7
|
||||
total_plans: 48
|
||||
completed_plans: 52
|
||||
percent: 20
|
||||
---
|
||||
|
||||
@@ -26,7 +26,7 @@ See: .planning/PROJECT.md (updated 2026-04-04)
|
||||
## Current Position
|
||||
|
||||
Phase: 09 (osint-infrastructure) — EXECUTING
|
||||
Plan: 2 of 6
|
||||
Plan: 3 of 6
|
||||
Status: Ready to execute
|
||||
Last activity: 2026-04-05
|
||||
|
||||
@@ -83,6 +83,7 @@ Progress: [██░░░░░░░░] 20%
|
||||
| Phase 08-dork-engine P03 | 10m | 2 tasks | 10 files |
|
||||
| Phase 08-dork-engine P07 | 3m | 1 tasks | 1 files |
|
||||
| Phase 09-osint-infrastructure P04 | 6min | 2 tasks | 4 files |
|
||||
| Phase 09 P05 | 5m | 2 tasks | 2 files |
|
||||
|
||||
## Accumulated Context
|
||||
|
||||
@@ -130,6 +131,6 @@ None yet.
|
||||
|
||||
## Session Continuity
|
||||
|
||||
Last session: 2026-04-05T21:43:35.879Z
|
||||
Stopped at: Completed 09-04-PLAN.md
|
||||
Last session: 2026-04-05T21:48:38.555Z
|
||||
Stopped at: Completed 09-05-PLAN.md
|
||||
Resume file: None
|
||||
|
||||
92
.planning/phases/09-osint-infrastructure/09-05-SUMMARY.md
Normal file
92
.planning/phases/09-osint-infrastructure/09-05-SUMMARY.md
Normal file
@@ -0,0 +1,92 @@
|
||||
---
|
||||
phase: 09-osint-infrastructure
|
||||
plan: 05
|
||||
subsystem: cli
|
||||
tags: [cli, recon, cobra, osint]
|
||||
requires: [09-01, 09-02, 09-03, 09-04]
|
||||
provides:
|
||||
- "cmd/recon.go with `recon full` and `recon list` subcommands"
|
||||
- "ExampleSource registered at process start so Phase 9 ships a demonstrable pipeline"
|
||||
- "Flag surface: --stealth, --respect-robots (default true), --query"
|
||||
affects:
|
||||
- "cmd/stubs.go (reconCmd stub removed)"
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- "cobra subcommand tree with package-level flag vars + init() wiring"
|
||||
- "buildReconEngine() factory so future phases (10-16) extend a single registration site"
|
||||
key-files:
|
||||
created:
|
||||
- "cmd/recon.go"
|
||||
- ".planning/phases/09-osint-infrastructure/09-05-SUMMARY.md"
|
||||
modified:
|
||||
- "cmd/stubs.go"
|
||||
decisions:
|
||||
- "Engine is built per-invocation (not a package global) to keep source registration explicit and testable"
|
||||
- "ExampleSource is registered inline in buildReconEngine rather than via init() side-effect so Phases 10-16 have one obvious place to add real sources"
|
||||
- "Dedup runs on the caller side (per engine.go comment) — Phase 05 calls recon.Dedup after SweepAll"
|
||||
metrics:
|
||||
duration: "~5m"
|
||||
completed: "2026-04-05"
|
||||
tasks: 2
|
||||
files-changed: 2
|
||||
---
|
||||
|
||||
# Phase 09 Plan 05: Recon CLI Command Tree Summary
|
||||
|
||||
Wired `pkg/recon` into the Cobra CLI with `keyhunter recon full` and `keyhunter recon list`, replacing the Phase 9 stub so the recon pipeline is demonstrable end-to-end on a fresh clone.
|
||||
|
||||
## What Shipped
|
||||
|
||||
- **`cmd/recon.go`** — new file declaring `reconCmd`, `reconFullCmd`, `reconListCmd`, and the `buildReconEngine()` factory. Flags `--stealth` (default false), `--respect-robots` (default true), and `--query` (default "") are registered on `reconFullCmd`. `buildReconEngine` constructs a fresh `recon.Engine`, registers `recon.ExampleSource{}`, and returns it.
|
||||
- **`recon full`** — calls `Engine.SweepAll(ctx, Config{...})`, pipes results through `recon.Dedup`, then prints a header line (`recon: swept N sources, M findings (K after dedup)`) followed by one line per deduped finding.
|
||||
- **`recon list`** — prints registered source names one per line (sorted by `Engine.List`).
|
||||
- **`cmd/stubs.go`** — `var reconCmd` stub removed; a comment points at `cmd/recon.go`. `rootCmd.AddCommand(reconCmd)` in `cmd/root.go` now resolves to the real declaration (same package).
|
||||
|
||||
## Verification Run
|
||||
|
||||
```
|
||||
$ go run . recon list
|
||||
example
|
||||
|
||||
$ go run . recon full
|
||||
recon: swept 1 sources, 2 findings (2 after dedup)
|
||||
[recon:example] openai sk-examp...AAAA https://example.invalid/a
|
||||
[recon:example] anthropic sk-ant-e...BBBB https://example.invalid/b
|
||||
|
||||
$ go run . recon full --stealth --query=test
|
||||
recon: swept 1 sources, 2 findings (2 after dedup)
|
||||
...
|
||||
```
|
||||
|
||||
`go build ./...` clean. `go test ./...` green across all packages (cmd, pkg/recon, pkg/dorks, pkg/engine, etc.).
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
**1. [Rule 3 - Blocker] Merged wave-1 dependencies into worktree**
|
||||
- **Found during:** Task 2 build verification
|
||||
- **Issue:** The worktree branch was behind master and missing commits from plans 09-01, 09-02, 09-04 (`pkg/recon/engine.go`, `source.go`, `example.go`, `limiter.go`, `robots.go`). `go build ./...` failed with `undefined: recon.Engine / recon.Config / recon.NewEngine / recon.ExampleSource`.
|
||||
- **Fix:** Ran `git merge master --no-edit --no-verify` — fast-forward from `1eb86ca` to `c2137ed` pulling in the four dependent plans' artifacts. No conflicts (my Task 1 edit to `cmd/stubs.go` and new `cmd/recon.go` were untouched by the merge).
|
||||
- **Files modified:** merge brought in `pkg/recon/{engine,source,example,limiter,robots,engine_test,limiter_test,robots_test}.go`, `go.mod`, `go.sum`, and the 09-01/02/04 SUMMARY files.
|
||||
- **Commit:** merge commit via fast-forward (no new hash)
|
||||
|
||||
## Known Stubs
|
||||
|
||||
- **`pkg/recon.ExampleSource`** — intentional per plan 09-01 / Phase 9 CONTEXT. It emits two deterministic fake findings (`openai`/`anthropic` with `recon:example` SourceType) so the CLI has visible output before Phases 10-16 land real sources. Will be superseded as Shodan/GitHub/Pastebin/etc. sources are added in Phases 10-16 via additional `e.Register(...)` calls in `buildReconEngine`.
|
||||
|
||||
## Commits
|
||||
|
||||
- `86a6bb8` — feat(09-05): add recon full/list commands and remove stub
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
- FOUND: cmd/recon.go
|
||||
- FOUND: cmd/stubs.go (reconCmd stub removed; verified `! grep -q 'var reconCmd' cmd/stubs.go`)
|
||||
- FOUND: commit 86a6bb8
|
||||
- VERIFIED: `go build ./...` clean
|
||||
- VERIFIED: `go run . recon list` prints `example`
|
||||
- VERIFIED: `go run . recon full` prints "recon: swept 1 sources, 2 findings (2 after dedup)" + 2 finding lines
|
||||
- VERIFIED: `go run . recon full --stealth --query=test` runs cleanly
|
||||
- VERIFIED: `go test ./...` green
|
||||
Reference in New Issue
Block a user