docs(04-05): complete scan command source wiring plan
This commit is contained in:
salvacybersec
@@ -37,7 +37,7 @@ Requirements for initial release. Each maps to roadmap phases.
|
||||
- [ ] **INPUT-03**: Git scanning supports --since flag for time-scoped history scan
|
||||
- [ ] **INPUT-04**: stdin/pipe input support (cat file | keyhunter scan stdin)
|
||||
- [ ] **INPUT-05**: URL fetching — scan content from any remote URL
|
||||
- [ ] **INPUT-06**: Clipboard content scanning
|
||||
- [x] **INPUT-06**: Clipboard content scanning
|
||||
|
||||
### Verification
|
||||
|
||||
|
||||
@@ -107,8 +107,8 @@ Plans:
|
||||
- [x] 04-01-PLAN.md — Wave 0: add go-git/v5, atotto/clipboard, golang.org/x/exp/mmap dependencies
|
||||
- [x] 04-02-PLAN.md — DirSource: recursive walk, glob exclusion, binary skip, mmap for large files (INPUT-01, CORE-07)
|
||||
- [x] 04-03-PLAN.md — GitSource: full-history scan across branches/tags with blob dedup and --since (INPUT-02)
|
||||
- [ ] 04-04-PLAN.md — StdinSource, URLSource, ClipboardSource (INPUT-03, INPUT-04, INPUT-05)
|
||||
- [ ] 04-05-PLAN.md — cmd/scan.go source-selection dispatch wiring all new sources (INPUT-06)
|
||||
- [x] 04-04-PLAN.md — StdinSource, URLSource, ClipboardSource (INPUT-03, INPUT-04, INPUT-05)
|
||||
- [x] 04-05-PLAN.md — cmd/scan.go source-selection dispatch wiring all new sources (INPUT-06)
|
||||
|
||||
### Phase 5: Verification Engine
|
||||
**Goal**: Users can opt into active key verification with a consent prompt, legal documentation, and per-provider API calls that confirm whether a found key is live and return metadata about the key's access level
|
||||
|
||||
@@ -3,14 +3,14 @@ gsd_state_version: 1.0
|
||||
milestone: v1.0
|
||||
milestone_name: milestone
|
||||
status: executing
|
||||
stopped_at: Completed 04-02-PLAN.md
|
||||
last_updated: "2026-04-05T12:19:39.522Z"
|
||||
stopped_at: Completed 04-05-PLAN.md
|
||||
last_updated: "2026-04-05T12:24:42.157Z"
|
||||
last_activity: 2026-04-05
|
||||
progress:
|
||||
total_phases: 18
|
||||
completed_phases: 3
|
||||
completed_phases: 4
|
||||
total_plans: 23
|
||||
completed_plans: 21
|
||||
completed_plans: 23
|
||||
percent: 20
|
||||
---
|
||||
|
||||
@@ -26,7 +26,7 @@ See: .planning/PROJECT.md (updated 2026-04-04)
|
||||
## Current Position
|
||||
|
||||
Phase: 04 (input-sources) — EXECUTING
|
||||
Plan: 4 of 5
|
||||
Plan: 5 of 5
|
||||
Status: Ready to execute
|
||||
Last activity: 2026-04-05
|
||||
|
||||
@@ -68,6 +68,7 @@ Progress: [██░░░░░░░░] 20%
|
||||
| Phase 04 P01 | 1m | 1 tasks | 2 files |
|
||||
| Phase 04-input-sources P03 | 6m | 1 tasks | 2 files |
|
||||
| Phase 04 P02 | 4min | 1 tasks | 3 files |
|
||||
| Phase 04 P05 | 3min | 1 tasks | 2 files |
|
||||
|
||||
## Accumulated Context
|
||||
|
||||
@@ -89,6 +90,7 @@ Recent decisions affecting current work:
|
||||
- [Phase 03-tier-3-9-providers]: Keyword-only detection for providers without documented key prefixes (You.com, Unstructured, Runway, Midjourney) to avoid false positives.
|
||||
- [Phase 04]: Use 'go mod download' instead of 'go mod tidy' when bootstrapping dependencies ahead of their consumers
|
||||
- [Phase 04-input-sources]: GitSource walks heads+tags+remotes+stash with per-OID blob dedup
|
||||
- [Phase 04]: Introduced selectSource dispatcher with sourceFlags struct for testable CLI source routing
|
||||
|
||||
### Pending Todos
|
||||
|
||||
@@ -103,6 +105,6 @@ None yet.
|
||||
|
||||
## Session Continuity
|
||||
|
||||
Last session: 2026-04-05T12:19:39.519Z
|
||||
Stopped at: Completed 04-02-PLAN.md
|
||||
Last session: 2026-04-05T12:24:42.153Z
|
||||
Stopped at: Completed 04-05-PLAN.md
|
||||
Resume file: None
|
||||
|
||||
188
.planning/phases/04-input-sources/04-05-SUMMARY.md
Normal file
188
.planning/phases/04-input-sources/04-05-SUMMARY.md
Normal file
@@ -0,0 +1,188 @@
|
||||
---
|
||||
phase: 04-input-sources
|
||||
plan: 05
|
||||
subsystem: cli
|
||||
tags: [cli, cobra, scan, source-dispatch, integration]
|
||||
|
||||
requires:
|
||||
- phase: 04-input-sources
|
||||
provides: DirSource, GitSource, StdinSource, URLSource, ClipboardSource (plans 04-02, 04-03, 04-04)
|
||||
provides:
|
||||
- selectSource dispatcher routing CLI args+flags to the correct Source implementation
|
||||
- --git, --url, --clipboard, --since, --max-file-size, --insecure CLI flags on `scan`
|
||||
- Mutual-exclusion validation for conflicting source selectors
|
||||
affects: [phase 05 verification integration, CLI UX for all Phase 4 input modes]
|
||||
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- Struct-parameter dispatcher (sourceFlags) for testable CLI flag fan-in
|
||||
- cobra.MaximumNArgs(1) to support both positional and flag-only invocations
|
||||
- Stat-based auto-dispatch between FileSource and DirSource on a single positional
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- cmd/scan_sources_test.go
|
||||
modified:
|
||||
- cmd/scan.go
|
||||
|
||||
key-decisions:
|
||||
- "selectSource takes a sourceFlags struct (not individual params) so the test suite can drive every combination without touching cobra flag state"
|
||||
- "Args changed from ExactArgs(1) to MaximumNArgs(1) because --url and --clipboard legitimately have no positional target"
|
||||
- "Stdin tokens 'stdin' and '-' are both accepted (spec from 04-04) and selectSource maps either to StdinSource"
|
||||
- "--since is parsed inside selectSource (not cobra) so the error path is covered by a unit test"
|
||||
- "--git + stdin is explicitly rejected even though --git is not in the explicit-source count, because scanning git history of a pipe is nonsensical"
|
||||
|
||||
patterns-established:
|
||||
- "CLI dispatcher pattern: positional-first, then flag-overrides, with mutual-exclusion checks up front"
|
||||
- "Test-friendly flag fan-in: wrap cobra flags in a plain struct passed to the logic function"
|
||||
|
||||
requirements-completed: [INPUT-06]
|
||||
|
||||
duration: ~3min
|
||||
completed: 2026-04-05
|
||||
---
|
||||
|
||||
# Phase 4 Plan 5: Scan Command Source Wiring Summary
|
||||
|
||||
**Wires every Phase 4 input adapter (Dir, Git, Stdin, URL, Clipboard) through `cmd/scan.go` via a new `selectSource` dispatcher and six new CLI flags, making INPUT-01 through INPUT-05 reachable from the command line and satisfying the INPUT-06 integration requirement.**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** ~3 min (135s)
|
||||
- **Started:** 2026-04-05T12:21:08Z
|
||||
- **Completed:** 2026-04-05T12:23:23Z
|
||||
- **Tasks:** 1 (TDD: RED → GREEN)
|
||||
- **Files created:** 1 (cmd/scan_sources_test.go)
|
||||
- **Files modified:** 1 (cmd/scan.go)
|
||||
|
||||
## Accomplishments
|
||||
|
||||
- Added `selectSource(args, sourceFlags) (sources.Source, error)` with full dispatch logic for all five Phase 4 sources plus legacy FileSource
|
||||
- Added six new cobra flags: `--git`, `--url`, `--clipboard`, `--since`, `--max-file-size`, `--insecure`
|
||||
- Relaxed `scanCmd.Args` from `ExactArgs(1)` to `MaximumNArgs(1)` so `--url` and `--clipboard` can run without a positional target
|
||||
- Added clear mutual-exclusion errors when `--git`, `--url`, `--clipboard` are combined, and when `--url`/`--clipboard` are paired with a positional path
|
||||
- 13-case unit test suite covering every dispatch branch, including `--since` parse-error path
|
||||
- Downstream pipeline (engine.Scan, storage.SaveFinding, output, exit codes) untouched — source selection is the only change
|
||||
|
||||
## Example Invocations
|
||||
|
||||
```bash
|
||||
# Directory (DirSource, default)
|
||||
keyhunter scan ./pkg
|
||||
|
||||
# Single file (FileSource, unchanged behavior)
|
||||
keyhunter scan main.go
|
||||
|
||||
# Git history (GitSource with optional --since)
|
||||
keyhunter scan --git ./some-repo --since 2024-01-01
|
||||
|
||||
# Stdin (StdinSource)
|
||||
echo "API_KEY=xxx" | keyhunter scan -
|
||||
cat creds.env | keyhunter scan stdin
|
||||
|
||||
# Remote URL (URLSource, no positional)
|
||||
keyhunter scan --url https://example.com/config.js
|
||||
|
||||
# Clipboard (ClipboardSource, no positional)
|
||||
keyhunter scan --clipboard
|
||||
|
||||
# Exclusions forwarded to DirSource
|
||||
keyhunter scan ./src --exclude "*.log" --exclude "tmp/**"
|
||||
```
|
||||
|
||||
## selectSource Dispatch Branches
|
||||
|
||||
| Input | Result |
|
||||
| --------------------------------------- | -------------------------------- |
|
||||
| `--clipboard` (no args) | `*sources.ClipboardSource` |
|
||||
| `--url https://...` (no args) | `*sources.URLSource` |
|
||||
| `--git <path>` | `*sources.GitSource` |
|
||||
| `--git <path> --since YYYY-MM-DD` | `*sources.GitSource` with `Since` set |
|
||||
| positional `stdin` or `-` | `*sources.StdinSource` |
|
||||
| positional directory | `*sources.DirSource` (+ excludes) |
|
||||
| positional file | `*sources.FileSource` |
|
||||
| Two of {`--git`, `--url`, `--clipboard`} | error: "mutually exclusive" |
|
||||
| `--url` or `--clipboard` + positional | error: "does not accept a positional argument" |
|
||||
| no args and no selector flag | error: "missing target" |
|
||||
| `--since` not `YYYY-MM-DD` | error: "must be YYYY-MM-DD" |
|
||||
| `--git` + stdin token | error: "cannot be combined with stdin" |
|
||||
|
||||
## Task Commits
|
||||
|
||||
1. **Task 1 RED** — `9105ca1` test(04-05): add failing tests for selectSource dispatcher
|
||||
2. **Task 1 GREEN** — `b151e88` feat(04-05): wire all Phase 4 sources through scan command
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- **Created:** `cmd/scan_sources_test.go` (112 lines, 13 test cases)
|
||||
- **Modified:** `cmd/scan.go` (+121, -15) — added flags, selectSource helper, dispatch call site, `time` import
|
||||
|
||||
## Test Results
|
||||
|
||||
Full `selectSource` suite under `-race`:
|
||||
|
||||
| Test | Result |
|
||||
| ---------------------------------------- | ------ |
|
||||
| TestSelectSource_Directory | PASS |
|
||||
| TestSelectSource_File | PASS |
|
||||
| TestSelectSource_Git | PASS |
|
||||
| TestSelectSource_GitSince | PASS |
|
||||
| TestSelectSource_GitSinceBadFormat | PASS |
|
||||
| TestSelectSource_URL | PASS |
|
||||
| TestSelectSource_URLRejectsPositional | PASS |
|
||||
| TestSelectSource_Clipboard | PASS |
|
||||
| TestSelectSource_ClipboardRejectsPositional | PASS |
|
||||
| TestSelectSource_Stdin (stdin + -) | PASS |
|
||||
| TestSelectSource_MutuallyExclusive | PASS |
|
||||
| TestSelectSource_MissingTarget | PASS |
|
||||
| TestSelectSource_DirForwardsExcludes | PASS |
|
||||
|
||||
Full repo suite: `go test ./... -race -count=1` → **all packages ok** (cmd, engine, engine/sources, providers, storage). Phase 1-3 tests still green.
|
||||
|
||||
## Decisions Made
|
||||
|
||||
- **sourceFlags struct over many parameters:** Keeps the test surface tiny (callers fabricate a struct literal) and makes adding future selectors (e.g. `--archive`) trivial without breaking signatures.
|
||||
- **Stat-based Dir vs File:** A single positional auto-dispatches on `os.Stat().IsDir()`. No new `--dir` flag needed — users just point at a path and it "does the right thing".
|
||||
- **`--since` parsing happens in selectSource:** So the invalid-format error is unit-testable and produces a CLI-friendly message rather than a raw time.Parse error.
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None — plan executed exactly as written. RED/GREEN TDD cycle completed in one iteration with no fixes needed beyond the initial plan specification.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
None.
|
||||
|
||||
## Known Stubs
|
||||
|
||||
None. `--max-file-size` and `--insecure` are wired as cobra flag declarations (per plan requirement) but are not yet consumed by any source — they are forward-compatibility hooks for Phase 5+ and documented as such in the flag help text. The plan explicitly requested the flag declarations; the downstream wiring is out of scope.
|
||||
|
||||
## User Setup Required
|
||||
|
||||
None.
|
||||
|
||||
## Next Phase Readiness
|
||||
|
||||
- All Phase 4 input adapters are now reachable from the CLI.
|
||||
- `keyhunter scan --help` lists every new flag with descriptive help text.
|
||||
- Phase 5 (verification) can build on top of `scanCfg.Verify` which was already present and remains untouched.
|
||||
- Future wiring of `--max-file-size` and `--insecure` into DirSource/URLSource respectively is a small follow-up (one line each).
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
- cmd/scan.go: FOUND (modified)
|
||||
- cmd/scan_sources_test.go: FOUND
|
||||
- Commit 9105ca1 (RED): FOUND in git log
|
||||
- Commit b151e88 (GREEN): FOUND in git log
|
||||
- `go build ./...`: exit 0
|
||||
- `go test ./... -race -count=1`: all packages ok
|
||||
- `grep -c selectSource cmd/scan.go`: 4 (≥2 required)
|
||||
- `grep sources.New(Dir|Git|Stdin|URL|Clipboard)Source cmd/scan.go`: 5 hits
|
||||
- `grep "mutually exclusive" cmd/scan.go`: 1 hit
|
||||
- `go run . scan --help` lists --git, --url, --clipboard, --since, --insecure: confirmed
|
||||
|
||||
---
|
||||
*Phase: 04-input-sources*
|
||||
*Plan: 05*
|
||||
*Completed: 2026-04-05*
|
||||
Reference in New Issue
Block a user