From c40e9f087fb420302706810e78878602f73afa99 Mon Sep 17 00:00:00 2001 From: salvacybersec Date: Sun, 5 Apr 2026 15:24:46 +0300 Subject: [PATCH] docs(04-05): complete scan command source wiring plan --- .planning/REQUIREMENTS.md | 2 +- .planning/ROADMAP.md | 4 +- .planning/STATE.md | 16 +- .../phases/04-input-sources/04-05-SUMMARY.md | 188 ++++++++++++++++++ 4 files changed, 200 insertions(+), 10 deletions(-) create mode 100644 .planning/phases/04-input-sources/04-05-SUMMARY.md diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 149ff6d..15a0cff 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -37,7 +37,7 @@ Requirements for initial release. Each maps to roadmap phases. - [ ] **INPUT-03**: Git scanning supports --since flag for time-scoped history scan - [ ] **INPUT-04**: stdin/pipe input support (cat file | keyhunter scan stdin) - [ ] **INPUT-05**: URL fetching — scan content from any remote URL -- [ ] **INPUT-06**: Clipboard content scanning +- [x] **INPUT-06**: Clipboard content scanning ### Verification diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 84173e0..c75a0cd 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -107,8 +107,8 @@ Plans: - [x] 04-01-PLAN.md — Wave 0: add go-git/v5, atotto/clipboard, golang.org/x/exp/mmap dependencies - [x] 04-02-PLAN.md — DirSource: recursive walk, glob exclusion, binary skip, mmap for large files (INPUT-01, CORE-07) - [x] 04-03-PLAN.md — GitSource: full-history scan across branches/tags with blob dedup and --since (INPUT-02) -- [ ] 04-04-PLAN.md — StdinSource, URLSource, ClipboardSource (INPUT-03, INPUT-04, INPUT-05) -- [ ] 04-05-PLAN.md — cmd/scan.go source-selection dispatch wiring all new sources (INPUT-06) +- [x] 04-04-PLAN.md — StdinSource, URLSource, ClipboardSource (INPUT-03, INPUT-04, INPUT-05) +- [x] 04-05-PLAN.md — cmd/scan.go source-selection dispatch wiring all new sources (INPUT-06) ### Phase 5: Verification Engine **Goal**: Users can opt into active key verification with a consent prompt, legal documentation, and per-provider API calls that confirm whether a found key is live and return metadata about the key's access level diff --git a/.planning/STATE.md b/.planning/STATE.md index 2122619..4ab2545 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,14 +3,14 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone status: executing -stopped_at: Completed 04-02-PLAN.md -last_updated: "2026-04-05T12:19:39.522Z" +stopped_at: Completed 04-05-PLAN.md +last_updated: "2026-04-05T12:24:42.157Z" last_activity: 2026-04-05 progress: total_phases: 18 - completed_phases: 3 + completed_phases: 4 total_plans: 23 - completed_plans: 21 + completed_plans: 23 percent: 20 --- @@ -26,7 +26,7 @@ See: .planning/PROJECT.md (updated 2026-04-04) ## Current Position Phase: 04 (input-sources) — EXECUTING -Plan: 4 of 5 +Plan: 5 of 5 Status: Ready to execute Last activity: 2026-04-05 @@ -68,6 +68,7 @@ Progress: [██░░░░░░░░] 20% | Phase 04 P01 | 1m | 1 tasks | 2 files | | Phase 04-input-sources P03 | 6m | 1 tasks | 2 files | | Phase 04 P02 | 4min | 1 tasks | 3 files | +| Phase 04 P05 | 3min | 1 tasks | 2 files | ## Accumulated Context @@ -89,6 +90,7 @@ Recent decisions affecting current work: - [Phase 03-tier-3-9-providers]: Keyword-only detection for providers without documented key prefixes (You.com, Unstructured, Runway, Midjourney) to avoid false positives. - [Phase 04]: Use 'go mod download' instead of 'go mod tidy' when bootstrapping dependencies ahead of their consumers - [Phase 04-input-sources]: GitSource walks heads+tags+remotes+stash with per-OID blob dedup +- [Phase 04]: Introduced selectSource dispatcher with sourceFlags struct for testable CLI source routing ### Pending Todos @@ -103,6 +105,6 @@ None yet. ## Session Continuity -Last session: 2026-04-05T12:19:39.519Z -Stopped at: Completed 04-02-PLAN.md +Last session: 2026-04-05T12:24:42.153Z +Stopped at: Completed 04-05-PLAN.md Resume file: None diff --git a/.planning/phases/04-input-sources/04-05-SUMMARY.md b/.planning/phases/04-input-sources/04-05-SUMMARY.md new file mode 100644 index 0000000..9a2f341 --- /dev/null +++ b/.planning/phases/04-input-sources/04-05-SUMMARY.md @@ -0,0 +1,188 @@ +--- +phase: 04-input-sources +plan: 05 +subsystem: cli +tags: [cli, cobra, scan, source-dispatch, integration] + +requires: + - phase: 04-input-sources + provides: DirSource, GitSource, StdinSource, URLSource, ClipboardSource (plans 04-02, 04-03, 04-04) +provides: + - selectSource dispatcher routing CLI args+flags to the correct Source implementation + - --git, --url, --clipboard, --since, --max-file-size, --insecure CLI flags on `scan` + - Mutual-exclusion validation for conflicting source selectors +affects: [phase 05 verification integration, CLI UX for all Phase 4 input modes] + +tech-stack: + added: [] + patterns: + - Struct-parameter dispatcher (sourceFlags) for testable CLI flag fan-in + - cobra.MaximumNArgs(1) to support both positional and flag-only invocations + - Stat-based auto-dispatch between FileSource and DirSource on a single positional + +key-files: + created: + - cmd/scan_sources_test.go + modified: + - cmd/scan.go + +key-decisions: + - "selectSource takes a sourceFlags struct (not individual params) so the test suite can drive every combination without touching cobra flag state" + - "Args changed from ExactArgs(1) to MaximumNArgs(1) because --url and --clipboard legitimately have no positional target" + - "Stdin tokens 'stdin' and '-' are both accepted (spec from 04-04) and selectSource maps either to StdinSource" + - "--since is parsed inside selectSource (not cobra) so the error path is covered by a unit test" + - "--git + stdin is explicitly rejected even though --git is not in the explicit-source count, because scanning git history of a pipe is nonsensical" + +patterns-established: + - "CLI dispatcher pattern: positional-first, then flag-overrides, with mutual-exclusion checks up front" + - "Test-friendly flag fan-in: wrap cobra flags in a plain struct passed to the logic function" + +requirements-completed: [INPUT-06] + +duration: ~3min +completed: 2026-04-05 +--- + +# Phase 4 Plan 5: Scan Command Source Wiring Summary + +**Wires every Phase 4 input adapter (Dir, Git, Stdin, URL, Clipboard) through `cmd/scan.go` via a new `selectSource` dispatcher and six new CLI flags, making INPUT-01 through INPUT-05 reachable from the command line and satisfying the INPUT-06 integration requirement.** + +## Performance + +- **Duration:** ~3 min (135s) +- **Started:** 2026-04-05T12:21:08Z +- **Completed:** 2026-04-05T12:23:23Z +- **Tasks:** 1 (TDD: RED → GREEN) +- **Files created:** 1 (cmd/scan_sources_test.go) +- **Files modified:** 1 (cmd/scan.go) + +## Accomplishments + +- Added `selectSource(args, sourceFlags) (sources.Source, error)` with full dispatch logic for all five Phase 4 sources plus legacy FileSource +- Added six new cobra flags: `--git`, `--url`, `--clipboard`, `--since`, `--max-file-size`, `--insecure` +- Relaxed `scanCmd.Args` from `ExactArgs(1)` to `MaximumNArgs(1)` so `--url` and `--clipboard` can run without a positional target +- Added clear mutual-exclusion errors when `--git`, `--url`, `--clipboard` are combined, and when `--url`/`--clipboard` are paired with a positional path +- 13-case unit test suite covering every dispatch branch, including `--since` parse-error path +- Downstream pipeline (engine.Scan, storage.SaveFinding, output, exit codes) untouched — source selection is the only change + +## Example Invocations + +```bash +# Directory (DirSource, default) +keyhunter scan ./pkg + +# Single file (FileSource, unchanged behavior) +keyhunter scan main.go + +# Git history (GitSource with optional --since) +keyhunter scan --git ./some-repo --since 2024-01-01 + +# Stdin (StdinSource) +echo "API_KEY=xxx" | keyhunter scan - +cat creds.env | keyhunter scan stdin + +# Remote URL (URLSource, no positional) +keyhunter scan --url https://example.com/config.js + +# Clipboard (ClipboardSource, no positional) +keyhunter scan --clipboard + +# Exclusions forwarded to DirSource +keyhunter scan ./src --exclude "*.log" --exclude "tmp/**" +``` + +## selectSource Dispatch Branches + +| Input | Result | +| --------------------------------------- | -------------------------------- | +| `--clipboard` (no args) | `*sources.ClipboardSource` | +| `--url https://...` (no args) | `*sources.URLSource` | +| `--git ` | `*sources.GitSource` | +| `--git --since YYYY-MM-DD` | `*sources.GitSource` with `Since` set | +| positional `stdin` or `-` | `*sources.StdinSource` | +| positional directory | `*sources.DirSource` (+ excludes) | +| positional file | `*sources.FileSource` | +| Two of {`--git`, `--url`, `--clipboard`} | error: "mutually exclusive" | +| `--url` or `--clipboard` + positional | error: "does not accept a positional argument" | +| no args and no selector flag | error: "missing target" | +| `--since` not `YYYY-MM-DD` | error: "must be YYYY-MM-DD" | +| `--git` + stdin token | error: "cannot be combined with stdin" | + +## Task Commits + +1. **Task 1 RED** — `9105ca1` test(04-05): add failing tests for selectSource dispatcher +2. **Task 1 GREEN** — `b151e88` feat(04-05): wire all Phase 4 sources through scan command + +## Files Created/Modified + +- **Created:** `cmd/scan_sources_test.go` (112 lines, 13 test cases) +- **Modified:** `cmd/scan.go` (+121, -15) — added flags, selectSource helper, dispatch call site, `time` import + +## Test Results + +Full `selectSource` suite under `-race`: + +| Test | Result | +| ---------------------------------------- | ------ | +| TestSelectSource_Directory | PASS | +| TestSelectSource_File | PASS | +| TestSelectSource_Git | PASS | +| TestSelectSource_GitSince | PASS | +| TestSelectSource_GitSinceBadFormat | PASS | +| TestSelectSource_URL | PASS | +| TestSelectSource_URLRejectsPositional | PASS | +| TestSelectSource_Clipboard | PASS | +| TestSelectSource_ClipboardRejectsPositional | PASS | +| TestSelectSource_Stdin (stdin + -) | PASS | +| TestSelectSource_MutuallyExclusive | PASS | +| TestSelectSource_MissingTarget | PASS | +| TestSelectSource_DirForwardsExcludes | PASS | + +Full repo suite: `go test ./... -race -count=1` → **all packages ok** (cmd, engine, engine/sources, providers, storage). Phase 1-3 tests still green. + +## Decisions Made + +- **sourceFlags struct over many parameters:** Keeps the test surface tiny (callers fabricate a struct literal) and makes adding future selectors (e.g. `--archive`) trivial without breaking signatures. +- **Stat-based Dir vs File:** A single positional auto-dispatches on `os.Stat().IsDir()`. No new `--dir` flag needed — users just point at a path and it "does the right thing". +- **`--since` parsing happens in selectSource:** So the invalid-format error is unit-testable and produces a CLI-friendly message rather than a raw time.Parse error. + +## Deviations from Plan + +None — plan executed exactly as written. RED/GREEN TDD cycle completed in one iteration with no fixes needed beyond the initial plan specification. + +## Issues Encountered + +None. + +## Known Stubs + +None. `--max-file-size` and `--insecure` are wired as cobra flag declarations (per plan requirement) but are not yet consumed by any source — they are forward-compatibility hooks for Phase 5+ and documented as such in the flag help text. The plan explicitly requested the flag declarations; the downstream wiring is out of scope. + +## User Setup Required + +None. + +## Next Phase Readiness + +- All Phase 4 input adapters are now reachable from the CLI. +- `keyhunter scan --help` lists every new flag with descriptive help text. +- Phase 5 (verification) can build on top of `scanCfg.Verify` which was already present and remains untouched. +- Future wiring of `--max-file-size` and `--insecure` into DirSource/URLSource respectively is a small follow-up (one line each). + +## Self-Check: PASSED + +- cmd/scan.go: FOUND (modified) +- cmd/scan_sources_test.go: FOUND +- Commit 9105ca1 (RED): FOUND in git log +- Commit b151e88 (GREEN): FOUND in git log +- `go build ./...`: exit 0 +- `go test ./... -race -count=1`: all packages ok +- `grep -c selectSource cmd/scan.go`: 4 (≥2 required) +- `grep sources.New(Dir|Git|Stdin|URL|Clipboard)Source cmd/scan.go`: 5 hits +- `grep "mutually exclusive" cmd/scan.go`: 1 hit +- `go run . scan --help` lists --git, --url, --clipboard, --since, --insecure: confirmed + +--- +*Phase: 04-input-sources* +*Plan: 05* +*Completed: 2026-04-05*